Fully Convolutional SVR for Single-Stack MRI
Sean I Young Yaël Balbastre Bruce Fischl Polina Golland Juan Eugenio Iglesias
Harvard Medical School MIT
1. Slice-to-Volume Reconstruction (SVR)
• MRI of subjects who move uncontrollably — 2D slice acquisitions taken
•
Acquired slices are assembled into a 3D volume — referred to as “SVR”
•
We propose a SVR method that can reconstruct from a single slice stack
2. Multiple-Stack SVR Using Optimization
• Reconstruct volume based on previously estimated motion parameters
• Re-estimate motion parameters based
on the previous 3D reconstruction
• Optimization problem becomes
• Objective is non-convex and solving
the problem is time-consuming
3. Previous Deep Learning Reconstructions
• Predict affine/rigid parameters
of each 2D slice in 3D space
• Can fail to reconstruct if
the slice motion is large
• Does not generalize to non-
affine deformable motion
4. From Single-View Depth Estimation …
• DL enables depth estimation from
a single view
• Train a fully convolutional network
model on (view, depth) examples
• Use similar principle for single-stack
slice-to-volume reconstruction?
5. ... To Fully Convolutional SVR
• Unroll iterations of optimization into end-to-end motion prediction
• Registration of an unknown moving image to a known fixed image
6. Supervised Learning
• Train fully convolutional network on
(slice stack, motion stack) examples
• Synthesize slice stacks with arbitrary
random but known motion
• Use rigid compensated
loss for training:
7. Rigid Motion-compensating Loss
• Uncompensated loss penalizes global rotations and
translations
• Translation-compensated loss improves training
• Full rigid motion compensated loss further improves
training
8. Supervised Interpolation
• Splatted volume has regions of
missing intensities due to motion
• Interpolate missing intensities to
produce artifact-free 3D volume
9. Reconstruction Results
10. Current and Future Work
• Fully convolutional deformable SVR for abdominal imaging et cetera
• Margherita’s adaptation to clinical workflows (multiple stacks et cetera)
Figure adapted from Simonyan et al. (2014)
Figure adapted from
Szeliski and Golland (1999)
Splat–Slice
U-Net
