Publications

2022

1. High Fidelity Deep Learning-based MRI Reconstruction with Instance-wise Discriminative Feature Matching Loss

   Wang, Ke, Tamir, Jonathan I, De Goyeneche, Alfredo, Wollner, Uri, Brada, Rafi, Yu, Stella, and Lustig, Michael

   Magnetic Resonance in Medicine, 2022

   Abs HTML

   Purpose: To improve reconstruction fidelity of fine structures and textures in deep learning- (DL) based reconstructions. Methods: A novel patch-based Unsupervised Feature Loss (UFLoss) is proposed and incorporated into the training of DL-based reconstruction frameworks in order to preserve perceptual similarity and high-order statistics. The UFLoss provides instance-level discrimination by mapping similar instances to similar low-dimensional feature vectors and is trained without any human annotation. By adding an additional loss function on the low-dimensional feature space during training, the reconstruction frameworks from under-sampled or corrupted data can reproduce more realistic images that are closer to the original with finer textures, sharper edges, and improved overall image quality. The performance of the proposed UFLoss is demonstrated on unrolled networks for accelerated two- (2D) and three-dimensional (3D) knee MRI reconstruction with retrospective under-sampling. Quantitative metrics including normalized root mean squared error (NRMSE), structural similarity index (SSIM), and our proposed UFLoss were used to evaluate the performance of the proposed method and compare it with others. Results: In vivo experiments indicate that adding the UFLoss encourages sharper edges and more faithful contrasts compared to traditional and learning-based methods with pure loss. More detailed textures can be seen in both 2D and 3D knee MR images. Quantitative results indicate that reconstruction with UFLoss can provide comparable NRMSE and a higher SSIM while achieving a much lower UFLoss value. Conclusion: We present UFLoss, a patch-based unsupervised learned feature loss, which allows the training of DL-based reconstruction to obtain more detailed texture, finer features, and sharper edges with higher overall image quality under DL-based reconstruction frameworks. (Code available at: https://github.com/mikgroup/UFLoss)

2. ResoNet: Physics Informed Deep Learning based Off-Resonance Correction Trained on Synthetic Data

   De Goyeneche, Alfredo, Ramachandran, Shreya, Wang, Ke, Karasan, Ekin, Yu, Stella, and Lustig, Michael

   Proceedings of the 31st Annual Meeting of ISMRM, 2022

   Abs HTML

   We propose a physics-inspired, unrolled-deep-learning framework for off-resonance correction. Our forward model includes coil sensitivities, multi-frequency bins, and non-uniform Fourier transforms hence compatible with fat/water imaging and parallel imaging acceleration. The network, which includes data-consistency terms and CNN modules serving as proximal operators, is trained end-to-end using only synthetic random field maps, coil sensitivities, and noise-like images with statistics (smoothness) mimicking natural signals. Our aim is to train the network to reverse off-resonance irrespective of the type of imaging, and hence generalizable to any anatomy and contrast without retraining. We demonstrate initial results in simulations, phantom, and in-vivo data.

3. Rigorous Uncertainty Estimation for MRI Reconstruction

   Wang, Ke, Angelopoulos, Anastasios, De Goyeneche, Alfredo, Kohli, Amit, Shimron, Efrat, Yu, Stella X, Malik, Jitendra, and Lustig, Michael

   Proceedings of the 31st Annual Meeting of ISMRM, 2022

   Abs HTML

   Deep-learning (DL)-based MRI reconstructions have shown great potential to reduce scan time while maintaining diagnostic image quality. However, their adoption has been plagued with fears that the models will hallucinate or eliminate important anatomical features. To address this issue, we develop a framework to identify when and where a reconstruction model is producing potentially misleading results. Specifically, our framework produces confidence intervals at each pixel of a reconstruction image such that 95% of these intervals contain the true pixel value with high probability. In-vivo 2D knee and brain reconstruction results demonstrate the effectiveness of our proposed uncertainty estimation framework.

4. BladeNet: Rapid PROPELLER Acquisition and Reconstruction for High spatio-temporal Resolution Abdominal MRI

   Shimron, Efrat, De Goyeneche, Alfredo, Wang, Ke, Syed, Ali Bin, Vasanawala, Shreyas, and Lustig, Michael

   Proceedings of the 31st Annual Meeting of ISMRM, 2022

   Abs HTML

   To improve bowel wall imaging in abdominal pediatric MRI scans, we propose a multi-phase single-shot fast spin echo (SSFSE) PROPELLER acquisition with novel deep learning reconstruction. This acquisition offers shorter scan time and thus higher temporal resolution, PROPELLER built-in motion correction, and alias-free images; these however exhibit spatial blurring. Our approach leverages the blurring-axis temporal rotation and data redundancy; we train a network to recover high-frequency spatial details from consecutive frames. Retrospective simulations with data from balanced SSFP scans show that this approach yields reconstructions with high spatio-temporal resolution and motion-correction, which are essential for pediatric abdominal imaging.

2021

1. Deep-Learning-Based Motion Correction For Quantitative Cardiac MRI

   De Goyeneche, Alfredo, Tang, Shuyu, Addy, Nii Okai, Hu, Bob S, Overall, William, and Santos, Juan M

   Proceedings of the 29th Annual Meeting of ISMRM, 2021

   Abs HTML

   We developed a deep-learning-based approach for motion correction in quantitative cardiac MRI, including perfusion, T1 mapping, and T2 mapping. The proposed approach consists of a segmentation network and a registration network. The segmentation network was trained using 2D short-axis images for each of the three sequences, while the same registration network was shared between all three sequences. The proposed approach was faster and more accurate than a popular traditional registration method. Our work is beneficial for building a faster and more robust automated processing pipeline to obtain CMR parametric maps.

2020

1. Explaining VQA predictions using visual grounding and a knowledge base

   Riquelme, Felipe*, De Goyeneche, Alfredo*, Zhang, Yundong, Niebles, Juan Carlos, and Soto, Alvaro

   Image and Vision Computing, 2020

   Abs HTML

   In this work, we focus on the Visual Question Answering (VQA) task, where a model must answer a question based on an image, and the VQA-Explanations task, where an explanation is produced to support the answer. We introduce an interpretable model capable of pointing out and consuming information from a novel Knowledge Base (KB) composed of real-world relationships between objects, along with labels mined from available region descriptions and object annotations. Furthermore, this model provides a visual and textual explanations to complement the KB visualization. The use of a KB brings two important consequences: enhance predictions and improve interpretability. We achieve this by introducing a mechanism that can extract relevant information from this KB, and can point out the relations better suited for predicting the answer. A supervised attention map is generated over the KB to select the relevant relationships from it for each question-image pair. Moreover, we add image attention supervision on the explanations module to generate better visual and textual explanations. We quantitatively show that the predicted answers improve when using the KB; similarly, explanations improve with this and when adding image attention supervision. Also, we qualitatively show that the KB attention helps to improve interpretability and enhance explanations. Overall, the results support the benefits of having multiple tasks to enhance the interpretability and performance of the model.

2019

1. Rapid Automated Cardiac Imaging

   De Goyeneche, Alfredo, Addy, Nii Okai, Islam, Haisam, Peterson, Eric, Overall, William, Santos, Juan M, and Hu, Bob S

   Frontier of AI-Assisted Care (FAC) Scientific Symposium, 2019

   HTML
2. One-Click Spine MRI

   De Goyeneche, Alfredo, Peterson, Eric, He, J Jason, Addy, N Okai, and Santos, Juan M

   Medical Imaging Meets NeurIPS Workshop at NeurIPS, 2019

   HTML
3. Automated Cardiac Magnetic Resonance Imaging

   De Goyeneche, Alfredo, Addy, Nii O, Santos, Juan M, and Hu, Bob S

   Circulation, 2019

   Abs HTML

   Introduction: Access, affordability, and accuracy of MRI examinations can all be vastly improved with machine learning-assisted automated imaging. We show that cardiac function can be clinically acquired and analyzed in less than 3 minutes or entire cardiac stress studies in less than 15 minutes with an automated workflow. Methods: We developed a flexible software architecture for image acquisition, recognition, and control. Human operator tasks including image recognition, planning, adjustments, and analysis are replaced by neural networks. The network was trained using TensorFlow with data from 58 clinical exams. The trained network sequentially located the complete set of standard views, in real time (Fig 1). A separate U-Net based model was trained to automatically segment the LV mass and volume for ejection fraction and cardiac mass. For the 15 minute scan, the desired inversion time was automatically determined using neural network outputs. Finally, a neural network was trained to detect image artifacts and repeat scans in the case of insufficient image quality. Results: In-vivo imaging was performed on a 1.5 T GE Signa scanner. Real-time spiral imaging was performed at 114 ms temporal resolution, reconstructed at 30 frames per second and 15 inferences per second. During inference, the network continuously analyzed series of real-time frames until a stable prescription was determined to minimize the effects of cardiac and respiratory motion. In a feasibility test of 50 patients, the network successfully found all standard views and segmented both ventricles without operator intervention. All scans, reconstructions, and analyses were completed in under 3 minutes. Conclusion: AI-assisted cardiac MRI examinations can be acquired rapidly with high precision which could profoundly affect the affordability, accessibility, and accuracy of cardiac diagnoses.