Sean I Young, PhD — Fully Convolutional Slice-to-Volume Reconstruction for Single-Stack MRI

Abstract

In magnetic resonance imaging (MRI), slice-to-volume

reconstruction (SVR) refers to computational reconstruction

of an unknown 3D magnetic resonance volume from stacks

of 2D slices corrupted by motion. While promising, current

SVR methods require multiple slice stacks for accurate 3D

reconstruction, leading to long scans and limiting their use

in time-sensitive applications such as fetal fMRI. Here, we

propose a SVR method that overcomes the shortcomings of

previous work and produces state-of-the-art reconstructions

in the presence of extreme inter-slice motion. Inspired by the

recent success of single-view depth estimation methods, we

formulate SVR as a single-stack motion estimation task and

train a fully convolutional network to predict a motion stack

for a given slice stack, producing a 3D reconstruction as a

byproduct of the predicted motion. Extensive experiments on

the SVR of adult and fetal brains demonstrate that our fully

convolutional method is twice as accurate as previous SVR

methods. Our code is available at github.com/seannz/svr.

1. Introduction

! #$%!&$&'(&)*+(),!(-*.(&.!$/!01,!2%*(&!31,%,!01,!+425,60!

-*7!8$0,&0(*997!,:1(2(0!+,),%,!4&6$&0%$99*29,!-$0($&;+461!

*+!(&!01,!6*+,!$/!01,!/,0*9!8$849*0($&;03$'<(-,&+($&*9!=>?@!

-*.&,0(6!%,+$&*&6,!(-*.(&.!=ABC@!0,61&(D4,+!*%,!078(6*997!

4+,<!0$!*6D4(%,!*!+0*6E!$/!>?!2%*(&!+9(6,+!*&<!*++,-29,!01,-!

(&0$!*!F?!)$94-,G!*!6$-840*0($&*9!8%$6,<4%,!$/0,&!%,/,%%,<!

0$!*+!+9(6,'0$')$94-,!%,6$&+0%460($&!=HIB@J!K&9(E,!F?!ABC!

*6D4(+(0($&+!31,%,!01,! ,&0(%,!𝑘'+8*6,!=#$4%(,%@! 6$,L6(,&0+!

6*&!2,6$-,!6$%%480,<!,),&!/%$-!*!+(&.9,!-$0($&!,),&0G!>?!

0,61&(D4,+!6*&!%,+0%(60!01(+!6$%%480($&!0$!3(01(&!*!+9(6,!31(9,!

9,*)(&.!01,!𝑘'+8*6,!<*0*!/$%!01,!$01,%!+9(6,+!(&0*60J!M,!HIB!

8%$6,<4%,!6*&!01,&!2,!*889(,<!$&!01,!&$&'6$%%480,<!+9(6,+!0$!

,&*29,!*%0(/*60'/%,,!(-*.(&.!$/!01,!2%*(&!*&<!/*6(9(0*0,!$01,%!

<$3&+0%,*-!0*+E+G!+461!*+!2%*(&!-$%81$-,0%7 N"OFPG!31$9,!

2%*(&!+,.-,&0*0($&!NQORP!*&<!*09*+!6%,*0($&!NSOTPJ

! U!9*%.,!&4-2,%!$/!69*++(6*9!HIB!0,61&(D4,+!N"VO"WP!*%,!

<,+6,&<,<!/%$-!01,!3$%E!$/!B$4++,*4!,0!*9J!N"TPG!31,%,!01,!

HIB!0*+E!(+!8$+,<!*+!*&!$80(-(X*0($&!8%$29,-!0$!%,6$),%!*&!

4&E&$3&!F?!)$94-,!/%$-!$2+,%),<!>?!+9(6,+J!H461!(&),%+,!

-,01$<+!<$!&$0!1*),!01,!6*8*2(9(07!0$!9,*%&!01,!9*0,&0!+8*6,!

$/!%,6$&+0%460($&+G!*&<!*+!*!%,+490G!078(6*997!%,D4(%,!-490(89,!

$%01$.$&*9!+9(6,!+0*6E+!0$!6%$++'%,.49*%(X,!01,!,+0(-*0($&!$/!

01,!-$0($&!$/!,*61!+0*6EG!*<<(&.!*!6$-840*0($&*9!24%<,&!0$!

*&!*9%,*<7!(0,%*0(),!&4-,%(6*9!-,01$<J!A$%,!%,6,&0!-,01$<+!

N>VO>YP!$),%6$-,!01,!24%<,&!$/!&4-,%(6*9!$80(-(X*0($&!27!

<(%,6097!8%,<(60(&.!01,!8$+(0($&!8*%*-,0,%+!$/!>?!+9(6,+!3(01!

&,4%*9!&,03$%E!-$<,9+J!Z$3,),%G!01,+,!*88%$*61,+!8%$6,++!

+9(6,+!(&<,8,&<,&097!$/!,*61!$01,%!*&<!6*&&$0!6$&+0%*(&!01,!

%,6$&+0%460($&!(&!F?!+8*6,!N>VO>>P!$%!%,D4(%,!-490(89,!+9(6,!

+0*6E+!/$%!*664%*0,!%,6$&+0%460($&!N>YPG!8%,694<(&.!01,(%!4+,!

(&!0(-,'+,&+(0(),!*889(6*0($&+!+461!*+!/,0*9!/ABC!N>RG!>SPJ!

! C&!01,!69$+,97!%,9*0,<!0*+E+!$/!8*(%3(+,!(-*.,!%,.(+0%*0($&!

N>WOF>PG!-$0($&!NFFOFTP!*&<!<,801!,+0(-*0($&!NQVOQFPG!/4997!

6$&)$940($&*9!&,03$%E!=#[\@!*%61(0,604%,+!1*),!2,,&!4+,<!

Figure 1: Method overview. Inspired by monocular disparity estimation (a), we tackle slice-to-volume reconstruction (b) by predicting

slice

motion from a single stack of 2D slices using a fully convolutional network model. ?e intensities of the 2D slices are splatted using the slice

motion predicted by our network and the splatting is then interpolated to produce an artifact-free 3D MR reconstruction.!

(a) Monocular Depth (Disparity) Estimation [38–41] (b) Single-Stack Slice-to-Volume Reconstruction (Proposed)

Splat and

Interpolate

Standard

U-Net

Splat–Slice

U-Net

Single View Disparity Map Single Slice Stack Motion Stack 3D Reconstruction

Fully Convolutional Slice-to-Volume Reconstruction for Single-Stack MRI

Sean I. Young* Yaël Balbastre*

Harvard Medical School Harvard Medical School

siyoung@mit.edu ybalbastre@mgh.harvard.edu!

Bruce Fischl Polina Golland Juan Eugenio Iglesias

Harvard Medical School MIT Harvard Medical School

fischl@nmr.mgh.harvard.edu polina@csail.mit.edu jei@mit.edu

*?ese authors contributed equally to this work.

0$!,&*29,!*664%*0,!&$&'(0,%*0(),!8%,<(60($&!$/!*&!4&<,%97(&.!

<,/$%-*0($&G!-$0($&!*&<!<(+8*%(07!/%$-!*!8*(%!$/!(-*.,+J!]7!

/$%-49*0(&.!01,!HIB!0*+E!*+!*!+48,%)(+,<!-$0($&!,+0(-*0($&!

8%$29,-G!3,!6*&!%,<46,!01,!0*+E!*0!1*&<!0$!*!8%$29,-!01*0!(+!

-$%,!%,*<(97!0*6E9,<!4+(&.!*!6$&)$940($&*9!/%*-,3$%EJ!M(+!

6$&)$940($&*9!*88%$*61!0$!HIB!2,*%+!*!6,%0*(&!%,+,-29*&6,!

0$!-$&$649*%!<,801!,+0(-*0($&G!31,%,!<(+8*%(07!(+!8%,<(60,<!

/%$-!*!+(&.9,!)(,3!NQ>PJ!M(+!(+!(994+0%*0,<!(&!#(.4%,!"J!

! C&!01(+!3$%EG!3,!8%$8$+,!*!/4997!6$&)$940($&*9!-,01$<!/$%!

HIB!3(01!*889(6*0($&!0$!+(&.9,'+0*6E!ABCJ!^,!8$+,!01,!HIB!

0*+E!*+!*!+48,%)(+,<!(-*.,!%,.(+0%*0($&!8%$29,-!*&<!0%*(&!*!

6$&)$940($&*9!&,03$%E!$&!8*(%,<!=+9(6,!+0*6EG!-$0($&!+0*6E@!

<*0*!0$!8%,<(60!+9(6,!-$0($&!<(%,6097G!8%$<46(&.!*!F?!)$94-,!

%,6$&+0%460($&!*+!*!%,+490!=#(.4%,!>@J!_4%!*88%$*61!%,D4(%,+!

$&97!*!+(&.9,!+9(6,!+0*6E!/$%!%,6$&+0%460($&G!(+!-*&7!$%<,%+!$/!

-*.&(04<,!/*+0,%!01*&!$80(-(X*0($&!*88%$*61,+!N"VO"WPG!*&<!

%,<46,+!,%%$%!(&!01,!8%,<(60,<!-$0($&!27!2×!6$-8*%,<!0$!01,!

%,6,&0!0%*&+/$%-,%'2*+,<!-$<,9!N>YPJ!_4%!6$&0%(240($&+!*%,`!

• Splat–Slice U-Net Model. ^,!(&0,%8%,0!01,!(0,%*0($&+!$/!

69*++(6*9!HIB!*+!*!+,%(,+!$/!+89*0!*&<!+9(6,!$8,%*0($&+!0$!

<,+(.&!*!F?!#[\!-$<,9!/$%!/*+0!+9(6,!-$0($&!8%,<(60($&J!

• Supervised Learning. ^,! /$%-49*0,!*! 8*%*-,0,%'/%,,!

9$++!/4&60($&!0$!/*6(9(0*0,!0%*(&(&.!3(01!0%4,!+9(6,!-$0($&!

/$%!+9(6,!-$0($&!8%,<(60($&!3(01!+42')$:,9!*664%*67J!

• Artifact-Free Reconstruction.!^,! $20*(&!*%0(/*60'/%,,!

F?!%,6$&+0%460($&+!27!+89*00(&.!+9(6,!<*0*!*&<!(-840(&.!

-(++(&.!(&0,&+(0(,+!)(*!+48,%)(+,<!(&0,%8$9*0($&J!

2. Related Work

! C&!/,0*9!(-*.(&.G!+9(6,'0$')$94-,!%,6$&+0%460($&!=HIB@!(+!

*&!(-8$%0*&0!(&),%+,!8%$29,-!3(01!%,6$&+0%460($&!-,01$<+!

%*&.(&.!/%$-!0%*<(0($&*9!$80(-(X*0($&'2*+,<!$&,+!N"VO"TP!0$!

%,6,&0!9,*%&(&.'2*+,<!$&,+!N>VO>YPG!31(61!3,!&$3!%,)(,3J!

2.1. Optimization Approaches

! _80(-(X*0($&'2*+,<!HIB!N"VO"WP!0%*6,+!(0+!3*7!2*6E!0$!

01,!+,-(&*9!3$%E!$/!B$4++,*4!,0!*9J!N"TPG!31$!8%,<(60,<!01,!

4&<,%97(&.!F?!ABC!)$94-,!27!*90,%&*0(&.97!,+0(-*0(&.!01,!

+9(6,!-$0($&!*&<!)$94-,!/%$-!01%,,!$%01$.$&*9!+9(6,!+0*6E+!

*&<!+48,%'%,+$9)(&.!01,!a&*9!8%,<(60,<!)$94-,J!M,!-,01$<!

$/!b(*&.!,0!*9J!N"VP!(+!69$+,97!%,9*0,<!0$!N"TP!240!*<$80+!*!.%(<!

+,*%61!0$!,+0(-*0,!01,!+9(6,!-$0($&!8*%*-,0,%+!0$!$),%6$-,!

01,!&$&'6$&),:(0(,+!$/!01,!$80(-(X*0($&!8%$29,-J!c1$9(8$4%!

,0!*9J!N""P!*&<!d4E9(+$)*'A4%.*+$)*!,0!*9J!N"FP!*00%(240,!01,!

6*4+,!$/!8$$%!%,6$&+0%460($&+!0$!(&*664%*0,!+9(6,!*9(.&-,&0+!

*&<!8%$8$+,!%,6$&+0%460($&!+0%*0,.(,+!0$!6$-2*0!-(+*9(.&,<!

+9(6,+!8$+0'1$6J!K4+!,0!*9J!N"WP!,:0,&<!N"FP!0$!*66$--$<*0,!

<,/$%-*29,!-$0($&!-$<,9+!*+!3$49<!2,!%,D4(%,<G!,J.JG!(&!01,!

%,6$&+0%460($&!$/!01,!89*6,&0*J!^1(9,!01,!/$64+!$/!$4%!3$%E!

(+!$&!2%*(&!%,6$&+0%460($&G!$4%!+48,%)(+,<!9,*%&(&.!*88%$*61!

+(-(9*%97!*66$--$<*0,+!<(e,%,&0!-$0($&!-$<,9+!=%(.(<!*&<!

<,/$%-*29,@!27!,&6$<(&.!%,.49*%(07!$/!-$0($&!<(%,6097!(&0$!

01,!0%*(&(&.!-$0($&!a,9<+!3(01$40!*<<(0($&*9!%,.49*%(X*0($&J!!

! H,),%*9!HIB!*88%$*61,+!,:0,&<!N"TP!27!%,8*%*-,0%(X(&.!

+9(6,!-$0($&!/$%!*!-$%,!/*)$%*29,!6$-840*0($&*9!6$-89,:(07!

$%!f,:(2(9(07!$/!01,!+9(6,!-$0($&J!d(-!,0!*9J!N">P!8%$8$+,!0$!

9$3,%!01,!6$+0!$/!-$0($&!,+0(-*0($&!27!*9(.&(&.!"?!9(&,+!$/!

(&0,%+,60($&!2,03,,&!>?!+9(6,+J!U9*&+*%7!,0!*9J!N"YP!8%$8$+,!

0$!1*&<9,!<,/$%-*29,!+9(6,!-$0($&!27!8*%0(0($&(&.!+9(6,+!(&0$!

+D4*%,!8*061,+!*&<!,+0(-*0(&.!*!+,8*%*0,!+,0!$/!%(.(<!-$0($&!

8*%*-,0,%+!/$%!,*61!8*061J!Z$3,),%G!01(+!+0%*0,.7!8%$<46,+!

*!8*061'3(+,!9(&,*%!*88%$:(-*0($&!$/!01,!4&<,%97(&.!-$0($&!

*&<!9,*<+!0$!29$6E!*%0(/*60+!(&!01,!%,6$&+0%460,<!)$94-,J!C&!

$4%!3$%EG!3,!%,8%,+,&0!+9(6,!-$0($&!*+!<,&+,!-$0($&!a,9<+!

0$!,&*29,!+0%*(.10'/$%3*%<!*%0(/*60'/%,,!F?!%,6$&+0%460($&J!

! C&!$80(-(X*0($&'2*+,<!HIB!-,01$<+G!.$$<!(&(0(*9(X*0($&!

$/!01,!F?!)$94-,!*&<!01,!+9(6,!-$0($&!6*&!1,98!-(0(.*0,!01,!

<,9,0,%($4+!,e,60+!$/!9$6*9!-(&(-*!(&!01,!$25,60(),!/4&60($&!

$&!01,!%,6$&+0%460($&J!d*(&X!,0!*9J!N"QP!8%$8$+,!0,61&(D4,+!

0$!*40$-*0(6*997!+,9,60!*!%,/,%,&6,!+9(6,!+0*6E!0$!(-8%$),!01,!

(&(0(*9(X*0($&!$/!01,!F?!)$94-,J!g$4%2(,%!,0!*9J!N"RP!(&(0(*9(X,!

01,!F?!)$94-,!3(01!*&!*.,'-*061,<!0,-89*0,!(-*.,G!31(61!

*60+!*+!8%($%!E&$39,<.,!(&!01,!%,6$&+0%460($&!/%*-,3$%EJ!]7!

6$&0%*+0G!3,!0*+E!$4%!/4997!6$&)$940($&*9!&,03$%E!-$<,9!3(01!

%$24+097!<,0,%-(&(&.!+9(6,!-$0($&!(&!01,!8%,+,&6,!$/!(-*.,!

&$&'6$&),:(0(,+G!*99$3(&.!(0!0$!<(+8,&+,!3(01!0,-89*0,+!*&<!

8%$<46,!+425,60'!*&<!8*01$9$.7'+8,6(a6!%,6$&+0%460($&+J!

2.2. Learning-Based Approaches

! [4%%,&097G!01,! -*5$%(07!$/! -*61(&,'9,*%&(&.'2*+,<! HIB!

-,01$<+!N>VO>YP!8%,<(60!01,!8$+(0($&!8*%*-,0,%+!$/!+9(6,+!=T!

Figure 2: Supervised learning. We train our splat–slice U-Net (a) on paired (slice stack, motion stack) examples, which are generated by

slicing 3D volumes with randomized slice-wise motion (motion stack). We additionally train a fully convolutional interpolation network (b)

on paired (splat volume, 3D volume) examples, where splat volumes are generated by splatting slice stacks with ground truth motion.

(a) Single-Stack Motion Estimation (b) Supervised Interpolation

Slice Stack

Splat Volume

3D Volume

𝐲

ℒ

Interpolator

𝐱

𝐲

ℒ

𝐱

3D Volume

Slice Stack

Motion Stack

Splat–Slice

U-Net

Slice

Splat

&4-2,%+@!3(01(&!01,!4&<,%97(&.!ABC!)$94-,!27!4+(&.!*!>?!

6$&)$940($&*9!&,4%*9!&,03$%E!=[\\@J!M,!8%,<(60,<!8$+(0($&!

8*%*-,0,%+!*%,!01,&!4+,<!0$!3*%8!01,!+9(6,+!*&<!%,6$),%!01,!

4&<,%97(&.!ABC!)$94-,J!M,![\\!-$<,9!$/!Z$4!,0!*9J!N>VP!

6$&+(+0+!$/!01,!6$&)$940($&*9!9*7,%+!$/!Icc'"R!8%,'0%*(&,<!

$&!C-*.,\,0!NQQP!*&<!*!<,&+,!6$&&,60,<!1,*<!0$!8%,<(60!01,!

*&61$%!8$(&0+!=Uh@!$/!01,!(&<()(<4*9!+9(6,+J!i, 4& .!,0!*9 J!N>QP!

,:0,&<!N>VP!*&<!8%,<(60!01,!8$+(0($&!8*%*-,0,%+!$/!*99!+9(6,+!

5$(&097G!(&0%$<46(&.!*&!(&0,%'+9(6,!*00,&0($&!9*7,%!2,03,,&!01,!

,:0%*60,<!/,*04%,+!*&<!01,!$40840!0$!6$&<(0($&*997!8%,<(60!01,!

-$0($&!/$%!,*61!+9(6,!.(),&!01,!/,*04%,+!$/!$01,%!+9(6,+J!

! A$%,!%,6,&097G! *!&$&'6$&)$940($&*9!-$<,9!/$%!HIB!3*+!

8%$8$+,<!27!j4!,0!*9J!N>YP!(&!*&!4&+48,%)(+,<!+,00(&.!4+(&.!

*!)(+($&!0%*&+/$%-,%!-$<,9!NQYOQSPJ!U901$4.1!01(+!8%$<46,+!

8%$-(+(&.!%,+490+G!(0!+6*9,+!D4*<%*0(6*997!3(01!01,!&4-2,%!$/!

+9(6,+!<4,!0$!(&0,%'+9(6,!*00,&0($&!6$-840*0($&+!*&<!%,D4(%,+!

178,%8*%*-,0,%!04&(&.!0$!2*9*&6,!<(e,%,&0!0,%-+!(&!01,!9$++!

/4&60($&J!]7!8$+(&.!01,!HIB!0*+E!*+!*!+48,%)(+,<!%,.(+0%*0($&!

8%$29,-!3(01(&!*!6$&)$940($&*9!/%*-,3$%EG!3,!+(.&(a6*&097!

9,++,&!01,!6$-89,:(07!$/!-$<,9(&.!8*(%3(+,!+9(6,!(&0,%*60($&!

01%$4.1!6$&)$940($&*9!3,(.10!+1*%(&.!*&<!$2)(*0,!01,!&,,<!

/$%!178,%8*%*-,0,%!$80(-(X*0($&!(&!01,!9$++!/4&60($&J!

! M,!8%$29,-!$/!%,.%,++(&.!+9(6,!8$+(0($&!=$%!,),&!-$0($&@!

*.*(&+0!(&0,&+(07!<*0*!(+!1(.197!&$&'6$&),:!<4,!0$!01,!(-*.,!

9*&<+6*8,!N"TPJ!^1(9,!01,%,!1*),!2,,&!,e$%0+!0$!$),%6$-,!

(-*.,!&$&'6$&),:(0(,+!(&!9,*%&(&.'2*+,<!(-*.,!%,.(+0%*0($&!

NF"G!F>PG!01,+,!6$&6,%&+!*%,!&$0!*<<%,++,<!(&!9,*%&(&.'2*+,<!

8$+,!8%,<(60($&!3$%E+!N>QG!>YP!*&<!*%,!,:*6,%2*0,<!/4%01,%!

27!+,8*%*0,!8%$6,++(&.!$/!01,!(&<()(<4*9!+9(6,+!N>VPJ!h$+(&.!

HIB!*+!(-*.,!%,.(+0%*0($&!*99$3+!4+!0$!*88,*9!0$!,J.J!NF"G!F>P!

0$!*<<%,++!01,!/4&<*-,&0*997!&$&'6$&),:!&*04%,!$/!HIBJ!

2.3. Pre- and Post-Processing

! HIB!0,61&(D4,+!078(6*997!(&)$9),!+9(6,!8%,'8%$6,++(&.!*+!

*!a%+0!+0,8!0$!%,5,60!+9(6,+!,:1(2(0(&.!6$&+(<,%*29,!(&0%*'+9(6,!

-$0($&!*&<!+,.-,&0!%,9,)*&0G!,J.JG!01,!2%*(&G!+0%4604%,+!/%$-!

01,!+6*&J!U&D4,X!,0!*9J!NQWP!8%$8$+,!*!-,01$<!0$!+E499'+0%(8!

/,0*9!)$94-,+G!31(61!6*&!2,!4+,<!*9+$!0$!-*+E!$40!&$&'2%*(&!

+0%4604%,+!(&!AB!+9(6,+!/$%!+-*99!(&0,%'+9(6,!-$0($&J!g$4%2(,%!

,0!*9J!N"RP!8%$8$+,!0$!9$6*9(X,!2%*(&!%,.($&+!(&!AB!+9(6,+!27!

29$6E'-*061(&.!*&!*.,'-*061,<!0,-89*0,!0$!01,!+9(6,+!01,&!

%,.(+0,%(&.!01,!-*061,<!%,.($&+!2*6E!0$!01,!0,-89*0,J!d*(&X!

,0!*9J!NQTP!9$6*9(X,!2%*(&!%,.($&+!2*+,<!$&!%$0*0($&'(&)*%(*&0!

<,+6%(80$%+!/$%!9*2,9(&.!2%*(&J!d,%*4<%,&!,0!*9J!NYVG!Y"P!4+,!*!

<,&+,!HC#g!*&<!%*&<$-!/$%,+0!69*++(a6*0($&!/$%!)$:,9'9,),9!

+,.-,&0*0($&J!H$-,!+1$%06$-(&.+!$/!01,+,!-,01$<+!(&694<,!

1(.1!6$-840*0($&*9!6$-89,:(07!N"RG!QWP!*+!3,99!*+!01,!&,,<!

0$!1*&<6%*/0!/,*04%,+!NYVG!Y"P!/$%!+,.-,&0*0($&J!!

! k,*%&(&.'2*+,<!*88%$*61,+!0$!/,0*9!+,.-,&0*0($&!(&694<,!

01,!-,01$<+!$/!B*5619!,0!*9J!NY>P!*&<!H*9,1(!,0!*9J!NYFPG!2$01!

$/!31(61!4+,!#[\!-$<,9+!NYQP!0$!8%,<(60!/$%,.%$4&<!-*+E+!

<(%,6097!$&!+9(6,+J!#$%!8$+0'8%$6,++(&.!$/!01,!%,6$&+0%460,<!

F?!)$94-,G!j4!,0!*9J!NYYP!8%$8$+,!&,4%*9!)$94-,!%,&<,%(&.!

+(-(9*%!0$!\,B#!NYRP!0$!+48,%'%,+$9),!%,6$&+0%460($&+!*0!*&!

*%2(0%*%7! %,+$940($&J! C&! 01(+! 3$%EG! 3,! *<<(0($&*997! 0%*(&! *&!

(&0,%8$9*0($&!&,03$%E!0$!8$+08%$6,++!01,!%*3!%,6$&+0%460($&+!

*&<!(&0,%8$9*0,!$),%!%,.($&+!$/!-(++(&.!(&0,&+(0(,+!01*0!-*7!

*88,*%!(&!01,!+(&.9,'+0*6E!%,6$&+0%460($&!6*+,G!8%$<46(&.!*&!

*%0(/*60'/%,,!%,6$&+0%460($&!/$%!$01,%!<$3&+0%,*-!0*+E+J!

3. Mathematical Preliminaries

3.1. Slice Acquisition Model

! C&!1(.1'%,+$940($&!F?!ABC!$/!+425,60+!,:1(2(0(&.!+,),%,!

4&6$&0%$99*29,!-$0($&G!/*+0!+9(6,!*6D4(+(0($&!+,D4,&6,+!+461!

*+!+(&.9,!+1$0!/*+0!+8(&!,61$!*%,!4+,<!0$!l/%,,X,m!01,!-$0($&!

(&'89*&,G!+42+0*&0(*997!-(0(.*0(&.!-$0($&!*%0(/*60+!6$-8*%,<!

0$!-490('+1$0!-,01$<+!NYSPJ!HIB!8%$6,<4%,+!*(-!0$!*9(.&!*&!

*6D4(%,<!+0*6E!$/!+9(6,+!01,%,27!%,-$)(&.!01,!%,-*(&(&.!=(&'

89*&,!*&<!01%$4.1'89*&,@!-$0($&!*6%$++!+9(6,+J!

! k,0!4+!<,&$0,!01,!6$$%<(&*0,+!(&+(<,!01,!ABC!+6*&&,%!27!

(𝑥, 𝑦, 𝑧)!*&<!*++4-,!*!+0*6E!𝑓!$/!>?!+9(6,+!(+!*6D4(%,<!*9$&.!

01,!𝑧!*:(+!3(01!+8*6(&.!𝑠!/%$-!*&!4&<,%97(&.!)$94-,!𝑣J!M,!

-$<,9!/$%!01,!*6D4(+(0($&!$/!01,!𝑘 01!AB!+9(6,!(+!N""G!"TP`!

𝑓

𝑘

(𝑥, 𝑦) = 𝜀 + ∫ (𝑣 ∘ 𝑢

𝑘

)(𝑥, 𝑦, 𝑧)ℎ(𝑠𝑘 − 𝑧) d𝑧

∞

−∞

="@!

/$%!𝑘 = 1, . . . , 𝐾G! 31,%,!𝑢

𝑘

: (𝑥, 𝑦, 𝑧) ↦ (𝑥, 𝑦, 𝑧)

′

!<,&$0,+!

*&!4&$2+,%),<!+9(6,')*%7(&.!%(.(<!+8*0(*9!+425,60!-$0($&G!ℎ!

<,&$0,+!01,!8$(&0!+8%,*<!/4&60($&!$/!+9(6,!*6D4(+(0($&G!*&<!𝜀!

6*804%,+!*99!AB'(&<46,<!,%%$%+!+461!*+!&$(+,G!2(*+G!,06J!

! g$!<(+6%,0(X,!-$<,9!="@G!+488$+,!𝑣!(+!+*-89,<!$&!*!642(6!

9*00(6,!$/!𝑁

!8$(&0+!*&<!𝑓

𝑘

!$&!*!>?!9*00(6,!$/!𝑁

!8$(&0+J!^,!

6*&!3%(0,!01,!<(+6%,0(X,<!-$<,9!=48!0$!%*&<$-!&$(+,@!*+!

𝐟 =

[

𝐟

; . . . ; 𝐟

𝐾

]

∈ ℝ

𝑁

𝐾

, 𝐟

𝑘

= 𝐇

𝑘

𝐔

𝑘

𝐯 ∈ ℝ

𝑁

=>@!

(&!31(61!𝐔

𝑘

!*&<!𝐇

𝑘

!%,8%,+,&0!<(+6%,0(X*0($&+!$/!(∘ 𝑢

𝑘

)!*&<!

ℎ(𝑠𝑘 − ⋅)G!%,+8,60(),97J!C&!01,!+-*99!-$0($&!%,.(-,G!3,!6*&!

/4%01,%!*88,*9!0$!g*79$%n+!*88%$:(-*0($&!*&<!<,a&,!

𝐔

𝑘

𝐯 = 𝐯 +

[

𝐝𝐢𝐚𝐠(𝐯

𝑥

) , 𝐝𝐢𝐚𝐠(𝐯

𝑦

) , 𝐝𝐢𝐚𝐠(𝐯

𝑧

)

]

𝐒𝐮

𝑘

=F@!

(&!31(61!𝐒𝐮

𝑘

%,8%,+,&0+!01,!,:8*&+($&!$/!01,!6$,L6(,&0+!$/!

-$0($&!𝐮

𝑘

(&!*!2*+(+!𝐒 *&<!𝐯

{𝑥,𝑦,𝑧}

!*%,!01,!%,+8,60(),!+8*0(*9!

<,%()*0(),+!$/!01,!F?!)$94-,J!^,!*66$--$<*0,!<,/$%-*29,!

-$0($&!-$<,9+!27!+,00(&.!𝐒!0$!01,!(<,&0(07!-*0%(:G!(&!31(61!

6*+,G!𝐮

𝑘

<,&$0,+!01,!),60$%(X*0($&!$/!*!F?!-$0($&!a,9<J!

3.2. Classical SVR Methods

! #$%! 2%,)(07G! 9,0! 4+! 3%(0,!𝐔 =

[

𝐇

𝐔

; . . . ; 𝐇

𝐾

𝐔

𝐾

]

!*&<!

01,!*++$6(*0,<!-$0($&!8*%*-,0,%+!*+!𝐮 =

[

𝐮

; . . . ; 𝐮

𝐾

]

J!^,!

6*&!3%(0,!01,!HIB!8%$29,-!3(01!01%,,!$%01$.$&*9!+0*6E+!*+!

minimize 𝐷(𝐮

{1,2,3}

, 𝐯

{1,2,3}

) =

‖

𝐔

𝐯

− 𝐟

‖

𝐔

𝐯

− 𝐟

‖

𝐔

𝐯

− 𝐟

‖

subject to 𝐯

− 𝐯

= 𝐯

− 𝐯

= 𝐯

− 𝐯

= 𝟎!

=Q@!

(&!31(61!𝐮

{1,2,3}

<,&$0,!01,!8*%*-,0,%+!$/!-$0($&!(&!,*61!$/!!

01,!+0*6E+J![$&0%*+0'(&)*%(*&0!9$++!/4&60($&+!+461!*+!-404*9!

(&/$%-*0($&!9$++!NYWP!*%,!*9+$!8$849*%97!4+,<!(&!89*6,!$/!01,!

D4*<%*0(6!$&,!+1$3&!1,%,!/$%!+(-89(6(07J!Z,%,G!3,!(&0%$<46,!

$80(-(X*0($&! )*%(*29,+!𝐯

{1,2,3}

!3(01! ,D4*9(07! 6$&+0%*(&0+! *+!

$88$+,<!0$!*!+(&.9,!)*%(*29,!𝐯!0$!/*6(9(0*0,!+$9)(&.!=Q@!4+(&.!

*90,%&*0(&.!$80(-(X*0($&!+0%*0,.(,+!+(-(9*%97!0$!N"SG!"TPJ!

! U!+(-89,!*90,%&*0(&.!$80(-(X*0($&!+0%*0,.7!31(61!<$,+!&$0!

(&0%$<46,!<4*9!)*%(*29,+!(+!0$!%,9*:!01,!,D4*9(07!6$&+0%*(&0+!0$!

01,!D4*<%*0(6!8,&*907!

𝑅

(

𝐯

{

1,2,3

}

)

‖[

𝐯

, 𝐯

]

−

[

𝐯

, 𝐯

]‖

=Y@!

*&<!$80(-(X,!01,!%,9*:*0($&!

𝐽(⋅) = 𝐷(𝐮

{1,2,3}

, 𝐯

{1,2,3}

) + 𝜆𝑅(𝐯

{1,2,3}

),!

=R@!

(&!31(61!𝜆!(+!*!8*%*-,0,%!6$&0%$99(&.!01,!%,9*0(),!3,(.10+!$/!

01,!<*0*!a<,9(07!0,%-!𝐷!*&<!01,!6$489(&.!0,%-!𝑅J!_25,60(),!

=R@!(&!/*60!6$%%,+8$&<+!0$!01,!$25,60(),!(-89(6(097!$80(-(X,<!

27!B$4++,*4!,0!*9J!N"TP!31,&!01,!-404*9!(&/$%-*0($&!9$++!(+!

4+,<!(&!01,!<*0*!a<,9(07!0,%-!𝐷J!_80(-(X(&.!=R@!4+(&.!29$6E!

6$$%<(&*0,!<,+6,&0!7(,9<+!48<*0,+!/$%!𝐯

, 𝐮

, 𝐯

, 𝐮

, 𝐯

!*&<!

𝐮

G!31,%,!01,!F?!%,6$&+0%460($&+!=𝑣'48<*0,@!*&<!01,!-$0($&!

8*%*-,0,%+!=𝑢'48<*0,@!$/!,*61!+0*6E!*%,!48<*0,<!(&!04%&!3(01!

01,!$01,%!03$!%,6$&+0%460($&+!a:,<!*0!,*61!04%&J!

! C&!01,!𝑛01!(0,%*0($&!$/!01,!29$6E!6$$%<(&*0,!<,+6,&0G!01,!𝑣'

48<*0,!+0,8!/$%!*!.(),&!+0*6E!*-$4&0+!0$!6$-840(&.!

𝐯

𝑛

← (𝐔

𝑛∗

𝐔

𝑛

+ 2𝜆𝐈)

−1

(𝐔

𝑛∗

𝐟

𝑛

+ 2𝜆𝐯







𝑛

),!

(&!31(61!𝐯

𝑛

!*&<!𝐟

𝑛

<,&$0,!01,!%,6$&+0%460($&!*&<!>?!+9(6,+!

$/!01,!𝑛%3%<!+0*6EG!%,+8,60(),97G!*&<!𝐯





𝑛

<,&$0,+!01,!*),%*.,!

%,6$&+0%460($&!/%$-!01,!03$!+0*6E+!$%01$.$&*9!0$!𝐯

𝑛

J!^,!6*&!

(&0,%8%,0!01(+!48<*0,!*+!*!%,6$&+0%460($&!$/!*!&,3!F?!)$94-,!

/%$-!*!3,(.10,<!+4-!$/!01,!9*+0!*),%*.,!F?!%,6$&+0%460($&!

*&<!3*%8,<!+9(6,+G!/$99$3,<!27!^(,&,%'<,6$&)$940($&J!M(+!

(&0,%8%,0*0($&!$/!01,!𝑣'48<*0,!3(99!.4(<,!$4%!6$&+0%460($&!$/!

*!/4997!6$&)$940($&*9!&,03$%E!9*0,%!(&!H,6!FJFJ!!

! U9.,2%*(6*997G!(𝐔, 𝐔

∗

) <,a&,!*&!*<5$(&0!8*(%!$/!3*%8(&.!

$8,%*0$%+G!31(61!3,!3(99!%,/,%!0$!*+!l+9(6(&.m!*&<!l+89*00(&.m!

%,+8,60(),97!NYTPJ!C&!01,!6*+,!31,%,!01,%,!(+!(&5,60(),!-$0($&!

%,9*0(&.!*!+9(6,!+0*6E!𝐟

𝑛

!0$!*!F?!)$94-,!𝐯G!3,!6*&!$20*(&!𝐟

𝑛

27!l+9(6(&.m!𝐯 3(01!𝐔

𝑛

!*&<!$20*(&!𝐯 27!l+89*00(&.m!𝐟

𝑛

!3(01!

𝐔

𝑛∗

J!#(.4%,!F!(994+0%*0,+!01,!03$!*<5$(&0!3*%8(&.!$8,%*0$%+!

(&!01,!<(+6%,0,!6*+,!31,&!-490('9(&,*%!(&0,%8$9*0($&!3,(.10+!

*%,!4+,<!/$%!/%*60($&*9!+9(6(&.!*&<!+89*00(&.J!

! M,!𝑢'48<*0,!/$%!01,!-$0($&!$/!01,!𝑘01!+9(6,!(&!*!+0*6E!(+!

𝐮

𝑘

𝑛

← (𝐒

∗

𝐕

𝑘

𝑛∗

𝐕

𝑘

𝑛

𝐒)

−1

𝐒

∗

𝐕

𝑘

𝑛∗

(𝐇

𝑘

𝐯

𝑛

− 𝐟

𝑘

𝑛

),!

=W@!

(&!31(61!𝐕

𝑘

𝑛

= 𝐇

𝑘

[

𝐝𝐢𝐚𝐠(𝐯

𝑥

𝑛

) , 𝐝𝐢𝐚𝐠(𝐯

𝑦

𝑛

) , 𝐝𝐢𝐚𝐠(𝐯

𝑧

𝑛

)

]

G 3(01!

01,!𝐯

{𝑥,𝑦,𝑧}

𝑛

<,&$0(&.!01,!%,+8,60(),!+8*0(*9!<,%()*0(),+!$/!01,!

.(),&!F?!)$94-,!%,6$&+0%460,<!(&!01,!(𝑛 − 1)01!(0,%*0($&J!

! C&!8%*60(6,G!+425,60!-$0($&!(&!*&!AB!+9(6,!9(E,97!,:6,,<+!

$&,!)$:,9!(&!-*.&(04<,!240!01,!9(&,*%!-$<,9!$/!3*%8(&.!=F@!

+0*7+!)*9(<!$&97!(/!01,!+9(6,!(+!a%+0!294%%,<!*0!01,!+6*9,!$/!01,!

-$0($&J!?4,!0$!01(+G!-$0($&!(+!$/0,&!,+0(-*0,<!$&!*!87%*-(<!

(&!*!6$*%+,'0$'a&,!-*&&,%G!4+(&.!01,!-$0($&!a,9<!,+0(-*0,<!

*0!6$*%+,%!9,),9+!0$!(&(0(*9(X,!-$0($&!*0!a&,%!$&,+!NRVPJ!ABC!

*6D4(+(0($&+!*9+$!+4e,%!/%$-!*!2(*+!a,9<!<4,!0$!&$&'4&(/$%-!

-*(&!-*.&,0(6!a,9<!*&<!%*<($/%,D4,&67!6$(9+G!(&0%$<46(&.!*!

.9$2*9!(&0,&+(07!+1(/0!2,03,,&!01,!4&E&$3&!)$94-,!*&<!01,!

+9(6,+J!U!E,7!61*99,&.,!(&!$80(-(X*0($&'2*+,<!HIB!-,01$<+!

(+!<,0,%-(&(&.G!*0!,*61!+6*9,G!-$0($&!(&!01,!8%,+,&6,!$/!2(*+!

*&<!,:0%,-,!+9(6,!*%0(/*60+J!A(+0*E(&.!,(01,%!/$%!-$0($&!6*&!

D4(6E97!9,*<!0$!8(0/*99+!$/!9$6*9!-(&(-*!(&!01,!$80(-(X*0($&!

$25,60(),!,+8,6(*997!*0!6$*%+,!+6*9,+J!Z$3,),%G!(0!(+!<(L6490!

0$!1*&<'6%*/0!*!%,6$&+0%460($&!8(8,9(&,!31(61!6*&!$),%6$-,!

(-*.,!&$&'6$&),:(0(,+!<4,!0$!+1,,%!&4-2,%+!$/!*6D4(+(0($&'

+8,6(a6!<,+(.&!61$(6,+!0$!2,!-*<,!=,J.JG!&$&'9(&,*%!a90,%+@J!

4. Fully Convolutional SVR

! K+(&.!*!#[\!-$<,9!/$%!HIB!,&*29,+!4+!0$!278*++!1*&<'

6%*/0,<!*6D4(+(0($&'+8,6(a6!%,6$&+0%460($&!8(8,9(&,+!*+!3,99!

*+!&4-,%(6*9!$80(-(X*0($&J!B*01,%!01*&!/$%-49*0(&.!HIB!*+!

01,!8%,<(60($&!$/!*2+$940,!+9(6,!6$$%<(&*0,+!(&!F?!+8*6,!*+!(+!

078(6*997!<$&,!(&!$01,%!9,*%&(&.'2*+,<!-,01$<+!N>WOF>PG!3,!

6*+0!HIB!*+!01,!%,.(+0%*0($&!$/!+$-,!4&$2+,%),<!F?!)$94-,!

0$!*&!$2+,%),<!+0*6E!$/!>?!+9(6,+J!^,!0%*(&!*!#[\!-$<,9!0$!

8%,<(60!-$0($&!%,9*0(&.!01,!+9(6,+!0$!*!F?!)$94-,!.(),&!$&97!

01,!+9(6,+!*+!(&840G!8%$<46(&.!*!F?!)$94-,!*+!*!27'8%$<460!

$/!%,.(+0%*0($&J![$&6,804*997G!01(+!(+!+(-(9*%!0$!01,!8%$29,-!

$/!-$&$649*%!<,801!,+0(-*0($&!NQVOQFPG!31,%,!$&,n+!.$*9!(+!

0$!8%,<(60!<(+8*%(07!=-$0($&!*9$&.!01,!,8(8$9*%!9(&,@!%,9*0(&.!

*!+(&.9,!>?!(-*.,!0$!01,!<,801!$/!01,!4&<,%97(&.!F?!+6,&,J!

4.1. Neural Network Architecture

! [\\!*%61(0,604%,+!+461!*+!01$+,!4+,<!(&!+,-*&0(6!(-*.,!

+,.-,&0*0($&!6*&!8%$<46,!+42$80(-*9!$406$-,+!/$%!-$0($&!

,+0(-*0($&!0*+E+!(/!01,!-$0($&!(+!9*%.,!$%!(-*.,+!,:1(2(0!a&,!

0,:04%,!NFFOFTPJ!^1(9,!/4997!6$&)$940($&*9!&,03$%E+!+461!*+!

01,!K'\,0!NYQP!*%,!,&<$3,<!3(01!*!-490('+6*9,!*%61(0,604%,!

*&<!6$49<! 01,$%,0(6*997!,+0(-*0,!-$0($&!(&!*! 6$*%+,'0$'a&,!

-*&&,%G!4+(&.!6$&)$940($&!E,%&,9+!0$!8*%*-,0,%(X,!3*%8(&.!

Figure 3: Slicing and splatting. In slicing (top row), the endpoints

of motion vectors deﬁne coordinate locations in a moving image to

pull data from and construct an image. In splatting (bottom), the

same motion vector endpoints deﬁne coordinate locations in a ﬁxed

image to push data to. Multi-linear weights are used for fractional

splatting and slicing, guaranteeing diﬀerentiability w.r.t grid data.

Moving Image Motion Field Slicing Sliced Image

1¼

1½

Fixed Image Motion Field Splatting Splat Image

Slice (Warp)

Splat (Warp

)

=+8*6,')*%7(&.!0%*&+/$%-+@!(+!(&1,%,&097!(&,L6(,&0G!2$01!(&!

0,%-+!$/!8*%*-,0%(X*0($&!*&<!6$-840*0($&o!+,,G!,J.JG!NFFG!FTPJ!

! _4%!&,03$%E!-$<,9!=#(.4%,!Q@!2,*%+!+(-(9*%(0(,+!0$!01$+,!

4+,<!(&!8*(%3(+,!(-*.,!%,.(+0%*0($&!N>WOF>PG!31(61!8%,<(60!

(-*.,!-$0($&!27!8%$6,++(&.!01,!(-*.,!8*(%!*6%$++!-490(89,!

9,),9+!$/!6$&)$940($&+J!M,+,!&,03$%E!-$<,9+!*%,!078(6*997!

,D4(88,<!3(01!3*%8(&.!9*7,%+!2,03,,&!*<5*6,&0!9,),9+!+461!

01*0!$&97!01,!%,+(<4*9!-$0($&!&,,<+!0$!2,!,+0(-*0,<!*0!,),%7!

9,),9J!\$0($&*997G!01,!+9(6,!+0*6E!*++4-,+!01,!%$9,!$/!*!a:,<!

(-*.,!31(9,!01,!F?!)$94-,G!31(61!3,!+,,E!0$!,+0(-*0,G!6*&!

2,!+,,&!*+!*!-$)(&.!(-*.,J!U+!<(+64++,<!(&!H,6J!FJ>G!3,!6*&!

$20*(&!01,!/,*04%,+!$/!01,!>?!+9(6,+!=a:,<!(-*.,@!27!+9(6(&.!

01,!/,*04%,+!$/!01,!F?!)$94-,!3(01!01,!,+0(-*0,<!-$0($&J!g$!

$20*(&!01,!/,*04%,+!$/!01,!F?!)$94-,G!3,!+89*0!01,!/,*04%,+!

$/!01,!+9(6,!+0*6E!4+(&.!01,!+*-,!-$0($&!,+0(-*0,J!

! \$0,!01*0!(&!$4%!6*+,G!$&97!01,!a:,<!(-*.,!=+9(6,!+0*6E@!(+!

.(),&G!+$!01,!<$3&3*%<!0%*&+/$%-!&,,<+!0$!8%$6,++!$&97!01,!

a:,<!(-*.,J!M,!483*%<!0%*&+/$%-!+7&01,+(X,+!a:,<!(-*.,!

/,*04%,+!27!4&8$$9(&.!9$3,%'%,+$940($&!$&,+G!6$&6*0,&*0(&.!

01,-!3(01!01,!+E(8!/,*04%,+!*&<!8%$6,++(&.!01,-!)(*!*!+,%(,+!

$/!6$&)$940($&+J!M,!-$)(&.!(-*.,!=$%!F?!)$94-,@!/,*04%,+!

*%,!+7&01,+(X,<!(&!8*%*99,9!27!4&8$$9(&.!01,(%!9$3'%,+$940($&!

6$4&0,%8*%0+G!6$&6*0,&*0(&.!01,-!3(01!01,!+89*00(&.+!$/!+E(8!

/,*04%,+!*&<!8%$6,++(&.!01,-!)(*!6$&)$940($&+J!M,!-$)(&.!

(-*.,!/,*04%,+!*%,!a&*997!+9(6,<G!6$&6*0,&*0,<!3(01!01,!a:,<!

(-*.,!/,*04%,+!*&<!8%$6,++,<!4+(&.!*!+,%(,+!$/!6$&)$940($&+!

0$!$40840!%,+(<4*9!+9(6,!-$0($&G!31(61!(+!*<<,<!0$!01,!9$3,%'

%,+$940($&!-$0($&!01*0!3*+!4+,<!<4%(&.!+89*00(&.!*&<!+9(6(&.J!

! C&!0,%-+!$/!(-89,-,&0*0($&G!2$01!01,!+9(6,!+0*6E!*&<!01,!

F?!)$94-,!/,*04%,+!*%,!+,,&!*+!=]G[G?GZG^@!0,&+$%+G!31,%,!

01,!?!<(-,&+($&!*60+!*+!01,!+9(6(&.!<(-,&+($&!(&!01,!6*+,!$/!

+9(6,!+0*6E!/,*04%,+J!F?!)$94-,!/,*04%,+!*%,!8%$6,++,<!4+(&.!

6$&)$940($&!*&<!8$$9(&.!E,%&,9+!$/!+1*8,!=FGFGF@!*&<!=>G>G>@!

%,+8,60(),97G!31(9,!+9(6,!/,*04%,+!*%,!8%$6,++,<!4+(&.!E,%&,9+!

$/!+1*8,+!="GFGF@!*&<!="G>G>@G!%,+8,60(),97G!+$!01*0!,*61!+9(6,!

6*&!2,!8%$6,++,<!(&<,8,&<,&097J!^(01(&!01,!%,+(<4*9!-$0($&!

,:0%*60($&!+0*.,+G!01,!a%+0!6$&)$940($&+!*%,!8,%/$%-,<!3(01!

E,%&,9+!$/!+1*8,!=QGFGF@!*&<!+0%(<,+!$/!=QG"G"@!+(&6,!+9(6,+!*%,!

(&0,%&*997!%,8%,+,&0,<!*+!F?!+9*2+!3(01!01(6E&,++!$/!Q!)$:,9+!

*&<!01,!-$0($&!&,,<+!0$!2,!%,.49*%(X,<!*6%$++!,*61!+9*2J!^,!

1*&<9,!<(e,%,&0!+9(6,!+8*6(&.+!(&!%,*9!<*0*!27!+6*9(&.!01,!>?!

(&840!+9(6,+!0$!*!Q`"!+9*2!01(6E&,++!0$!)$:,9!+8*6(&.!%*0($J!!

4.2. Rigid Motion-Compensating Loss

! C&,:*60!8$+(0($&(&.!$/!+425,60+!(&!ABC!+6*&&,%+!078(6*997!

(&0%$<46,+!*!.9$2*9!+425,60!$e+,0!=%$0*0($&+!*&<!0%*&+9*0($&+@!

(&!01,!*6D4(%,<!+9(6,+J!#$%!%,6$&+0%460($&G!1$3,),%G!3,!&,,<!

$&97!%,6$),%!01,!-$0($&!$/!01,!+9(6,+!%,9*0(),97!0$!,*61!$01,%!

(.&$%(&.!.9$2*9!+1(/0+J!g$!,&+4%,!01*0!3,!8,&*9(X,!8%,<(60($&!

,%%$%+!$&97!(&!01,!%,9*0(),!+9(6,!-$0($&+G!3,!6$-8,&+*0,!/$%!

*&7!.9$2*9!%(.(<!-$0($&!+1(/0!01*0!-*7!,:(+0!(&!$4%!8%,<(60,<!

-$0($&!+0*6E!2,/$%,!6$-840(&.!01,!0%*(&(&.!9$++!*.*(&+0!01,!

8%,<(60($&!0*%.,0J!U++4-(&.!*&!AHp!0%*(&(&.!$25,60(),G!3,!

6*&!,:8%,++!$4%!-$0($&'6$-8,&+*0(&.!0%*(&(&.!9$++!2,03,,&!

01,!8%,<(60,<!𝐮!*&<!.%$4&<!0%401!𝐲 ∈ ℝ

𝑁×3

-$0($&!a,9<+!*+!

𝓛

(

𝐮, 𝐲

)

= min

[

𝐑,𝐭

]

‖

𝐮 + 𝐩 −

(

𝐲 + 𝐩

)

𝐑 − 𝐭

‖

𝐹

(&!31(61!𝐑!%,8%,+,&0+!*!3 × 3!%$0*0($&!-*0%(:G!𝐭!%,8%,+,&0+!*!

1 × 3!0%*&+9*0($&!),60$%G!*&<!𝐩 ∈ ℝ

𝑁×3

(+!*!-*0%(:!$/!)$:,9!

6$$%<(&*0,+J!C&!3$%<+G!01(+!9$++!a%+0!8,%/$%-+!*!.9$2*9!%(.(<!

*9(.&-,&0!2,03,,&!01,!8$(&0!69$4<+!𝐮 + 𝐩 *&<!𝐲 + 𝐩G!+461!

01*0!$&97!01,!&$&'%(.(<!-$0($&!6$-8$&,&0!2,03,,&!01,!03$!

8$(&0!69$4<+!(+!8,&*9(X,<J!k$++!𝓛!%,<46,+!0$!01,!4+4*9!AHp!

9$++!(&!01,!*2+,&6,!$/!.9$2*9!%$0*0($&!𝐑!*&<!0%*&+9*0($&!𝐭J!

! g$!+$9),!01,!$80(-(X*0($&!8%$29,-!=T@G!3,!a%+0!-(&(-(X,!

01,!4&6$&+0%*(&,<!),%+($&!$/!8%$29,-!=T@!$),%!01,!+8*6,!$/!

*99!

[

𝐑, 𝐭

]

∈ ℝ

3×4

0$!$20*(&!*!-(&(-(X,%!

[

𝐑

′

, 𝐭

⋆

]

G!01,&!/*60$%!

𝐑

′

(&0$!(0+!%$0*0($&!*&<!+1,*%!-*0%(6,+!*+!𝐑

′

= 𝐑

⋆

𝐒

⋆

J!#%$-!

01,!+(&.49*%!)*94,!<,6$-8$+(0($&!𝐑

′

= 𝐔𝚺𝐕

∗

G!3,!+,,!01*0!

𝐑

⋆

= 𝐔sign

(

𝚺

)

𝐕

∗

, 𝐒

⋆

= 𝐕abs

(

𝚺

)

𝐕

∗

="V@!

*!/*60$%(X*0($&!E&$3&!*+!01,!8$9*%!<,6$-8$+(0($&!NR"PJ!M,!

%,+490(&.!%$0*0($&!-*0%(:!𝐑

⋆

!(+!01,!-(&(-4-!-,*&!+D4*%,<'!

,%%$%!*88%$:(-*0($&!$/!𝐑

′

G!*&<!6$&+,D4,&097G!01,!+$940($&!$/!

01,!$%(.(&*9!%(.(<!-$0($&'6$&+0%*(&,<!8%$29,-!=T@!(+!.(),&!27!

01,! 8*(%!

[

𝐑

⋆

, 𝐭

⋆

]

J!M,! 9$++!𝓛(𝐮, 𝐲) (+! 01,&! 2*6E8%$8*.*0,<!

*&<!01,!3,(.10+!$/!$4%!HIB!&,03$%E!-$<,9!48<*0,<J!#(.4%,!

Figure 4: Splat–Slice U-Net. ?e stack of 2D slices goes through

2D feature extraction (downward path) and reconstruction (upward

path) with 2D skip features. 3D volume features are reconstructed

by splatting 2D skip features with previous motion to form 3D skip

features. At each level, the sliced 3D feature volume and the slice

feature stack are convolved jointly to extract residual slice motion.!

Splat

Concat

Splat

Concat

2D Feat

3D Feat

Motion

Conv

Add

Slice-Conv

Figure 5: Rigid motion-compensated loss. Validation MSE and

EPE (end-point error) [33] of the predicted motion shown. Our loss

(blue), which compensates for oﬀsets in global rigid motion, trains

faster and attains higher ﬁnal prediction accuracy than the MSE loss

(orange) and the loss that compensates for translations only (gray). !

Rigid-Comp

No Comp

Trans-Comp

Motion MSE (voxels)

Motion EPE (voxels)

Rigid-Comp

No Comp

Trans-Comp

Epochs

Y!89$0+!01,!)*9(<*0($&!64%),+!/$%!$4%!&,03$%E!-$<,9!0%*(&,<!

4+(&.!-$0($&'6$-8,&+*0,<!*&<!%,.49*%!9$++,+G!+1$3(&.!01*0!

$4%!%(.(<!6$-8,&+*0(&.!9$++!(+!E,7!0$!(-8%$)(&.!8%,<(60($&J!

4.3. Interpolating Reconstructions

! C&!.,&,%*9G!*!F?!)$94-,!%,6$&+0%460,<!/%$-!*!+(&.9,!+9(6,!

+0*6E!6*&!6$&0*(&!1$9,+!(/!%,.($&+!$/!01,!4&<,%97(&.!)$94-,!

*%,!-(++,<!(&!*99!+9(6,!*6D4(+(0($&+!<4,!0$!+425,60!-$0($&J!C&!

6$-8*%(+$&!3(01!&*04%*9!(-*.,+G!AB!+6*&+!078(6*997!,:1(2(0!

*!9(-(0,<!<(),%+(07!(&!(-*.,%7!*&<!1*),!*!+0%$&.!8%($%!$&!01,!

<(+0%(240($&!$/!4&<,%97(&.!F?!)$94-,+J!M(+!54+0(a,+!01,!4+,!

$/!(&0,%8$9*0($&!0,61&(D4,+!0$!a99!1$9,+!(&!%,6$&+0%460($&+!0$!

+*0(+/7!%,.49*%(07!6$&+0%*(&0+!=,J.JG!$&!8*01$9$.7@J!^,!04%&!0$!

*!/4997!6$&)$940($&*9!(&0,%8$9*0($&!-$<,9!0%*(&,<!(&!*<)*&6,!

$&!=0%4,!%,6$&+0%460($&G!)$94-,@!8*(%+J!g%4,!%,6$&+0%460($&+!

=3(01!1$9,+@!6*&!2,!$20*(&,<!%,*<(97!*0!0%*(&!0(-,!27!3*%8(&.!

+9(6,!+0*6E+!3(01!.%$4&<!0%401!+9(6,!-$0($&J!^1(9,!01,!+,*%61!

/$%!01,!2,+0!(&0,%8$9*0($&!&,03$%E!-$<,9!(+!&$0!01,!/$64+!$/!

01(+!3$%EG!3,!a&<!01*0!*!+0*&<*%<!K'\,0!-$<,9!0%*(&,<!3(01!

0%4,!%,6$&+0%460($&+!*+!01,!0*%.,0!*&<!4+(&.!01,!-,*&!+D4*%,<!

,%%$%!9$++!8%$<46,+!.$$<!%,+490+J!#(.4%,!>!=%(.10@!+1$3+!01,!

0%*(&(&.!8%$6,<4%,!/$%!$4%!/4997!6$&)$940($&*9!(&0,%8$9*0$%J!!

5. Experimental Results

! ^,! %4&!,:0,&+(),!,:8,%(-,&0+!$&!2$01!*<490!ABC!+0*6E+!

=/$%!31(61!.%$4&<!0%401!AB!)$94-,+!*%,!*)*(9*29,@!*&<!/,0*9!

$&,+G!/$%!31(61!$&97!%,/,%,&6,!)$94-,+!*%,!.(),&J!#$%!,*61!

,:8,%(-,&0G!3,!0%*(&!*!HIB!&,03$%E!(&!*!+48,%)(+,<!/*+1($&!

$&!8*(%,<!=+9(6,!+0*6EG!-$0($&!+0*6E@!<*0*!*&<!*&!(&0,%8$9*0$%!

&,03$%E!$&!8*(%,<!=+9(6,O+89*0!)$94-,G!4&<,%97(&.!)$94-,@!

<*0*!=H,6J!QJF@J!M,!<*0*!*4.-,&0*0($&!178,%8*%*-,0,%+!*&<!

0%*(&(&.!<,0*(9+!*%,!.(),&!(&!U88,&<(:!UJ!^,! 4+,!01,!-,*&!

+D4*%,<!,%%$%!=AHp@!*&<!01,!*),%*.,!,&<'8$(&0!,%%$%!=php@!

NFFP!0$!-,*+4%,!8%,<(60($&!,%%$%o!+,,!U88,&<(:!UJ!

5.1. Single-Stack SVR of Adult Brains

! HIB!,:8,%(-,&0+!$&!+7&01,0(6!+9(6,<!ABC!)$94-,+!6*&!

%,),*9!D4*9(0*0(),!*&<!D4*&0(0*0(),!61*%*60,%(+0(6+!$/!*!HIB!

-,01$<!27!/*6(9(0*0(&.!6$-8*%(+$&!*6%$++!01,!%,6$&+0%460,<!

*&<!01,!.%$4&<!0%401!F?!AB!)$94-,+J!^,!64%*0,!""VV!*<490!

2%*(&!AB!+6*&+!/%$-!U]C?pG!'>!NR>PG!U?Z?!NRFPG![_]Bp!

NRQPG!cHh!NRYPG!A[C[!NRRPG!_UHCH!NRSPG!hhAC!NRWPG!Kd]!

NRTP!*&<!]FT!NSVPG!*&<!+89(0!01,!+6*&+!(&0$!"VVV!0%*(&(&.!*&<!

"VV!)*9(<*0($&!,:*-89,+J!U99!+6*&+!1*),!*!>YR

!)$94-,!3(01!

"--

!%,+$940($&J!M,!+6*&+!*%,!*9(.&,<!0$!01,!g*9*(%*61!*09*+!

NS"P!0$!/*6(9(0*0,!+9(6(&.!*9$&.!01,!+*.(00*9G!*:(*9!*&<!6$%$&*9!

<(%,60($&+o!+,,!U88,&<(:!U!/$%!<,0*(9+!$&!0%*(&(&.!*&<!+*-89,!

.,&,%*0($&J!!

Slicing Directions and Reconstruction. H(&.9,'+0*6E!HIB!

6*&!*61(,),!<(e,%,&0!%,6$&+0%460($&!*664%*67!<,8,&<(&.!$&!

01,!<(%,60($&!$/!+9(6(&.!=+*.(00*9G!*:(*9!*&<!6$%$&*9@J!g$!+04<7!

01,!,e,60!$/!+9(6,!<(%,60($&!$&!01,!*664%*67!$/!01,!8%,<(60,<!

-$0($&G!3,!0%*(&!*!+,8*%*0,!&,03$%E!/$%!,*61!+9(6,!<(%,60($&!

*&<!89$0!(&!#(.4%,!RG!01,!)*9(<*0($&!AHp!*&<!p&<8$(&0!p%%$%!

$/!01,!-$0($&!8%,<(60,<!27!,*61!&,03$%EJ!M,!a&*9!)*9(<*0($&!

,%%$%!*00*(&,<!(+!01,!9$3,+0!/$%!+*.(00*9!+0*6E+!*0!VJWT!=AHp@!

*&<!01,!1(.1,+0!/$%!*:(*9!+0*6E+!*0!"JFF!=AHp@J!H461!*!+9(.10!

(&6%,*+,!(&!01,!8%,<(60($&!,%%$%!/$%!*:(*9!+0*6E+!-$+0!9(E,97!

+0,-+!/%$-!/,3,%!l6$%&,%m!/,*04%,+!0$!0%*6E!*&<!*9(.&!(&!*:(*9!

+9(6,+G!9,*<(&.!0$!*!9,++!8%,6(+,!*9(.&-,&0J!

Motion Prediction Error.!^,! 89$0!<(+0%(240($&+!$/!01,!8,%'!

+425,60!,%%$%+!$/!01,!8%,<(60,<!-$0($&!+0*6E+!*6%$++!YVV!1,9<'

$40!+6*&+!(&!#(.4%,!SJ!U+!,:8,60,<!/%$-!01,!)*9(<*0($&!,%%$%!

64%),+!+,,&!(&!#(.4%,!RG!+*.(00*9!8%,<(60($&+!1*),!01,!9$3,+0!

-,<(*&!-$0($&!AHp!*&<!01,!0(.10,+0!(&0,%D4*%0(9,!%*&.,J!_&!

*),%*.,G!$4%!8%,<(60,<!-$0($&!(+!*664%*0,!0$!9,++!01*&!*!)$:,9!

*901$4.1!01,!hhAC!<*0*+,0!,:1(2(0+!+9(.1097!1(.1,%!,%%$%+J!

Reconstruction and Interpolation.!#(.4%,!W!)(+4*9(X,+!01,!

F?!%,6$&+0%460($&+!$20*(&,<!4+(&.!01,!8%,<(60,<!-$0($&J!^,!

*9(.&!*99!%,6$&+0%460($&+!2*6E!0$!01,!g*9*(%*61!*09*+!NS"P!/$%!

6$-8*%(+$&!*6%$++!<(e,%,&0!%,6$&+0%460($&+J!^,!*9+$!(&694<,!

%,6$&+0%460($&+!3(01!01,!-(++(&.!(&0,&+(07!<*0*!(&0,%8$9*0,<!

4+(&.!$4%!(&0,%8$9*0$%!<,+6%(2,<!(&!H,6J!QJFJ!M,!+89*0!%,+490+!

+1$3!01*0!$4%!&,03$%E!8%,<(60+!9(&,*%!-$0($&!a,9<+!3(01$40!

01,!&,,<!/$%!*<<(0($&*9!9(&,*%!8%$5,60($&+J!M,!(&0,%8$9*0($&!

%,-$),+!1$9,+!*+!3,99!*+!+9(6(&.!*%0(/*60+!01*0!(&0,%/,%,!3(01!

<$3&+0%,*-!0*+E+!+461!*+!2%*(&!-$%81$-,0%7J!k*%.,%!1$9,+!

*%,!+,,&!31,&!+89*00(&.+!*%,!)(,3,<!*9$&.!01,!+9(6(&.!*:(+J!

Figure 6: Accuracy of predicted motion. Slicing direction has an

impact on the accuracy of predicted motion. We plot the validation

MSE (left), and EPE (end-point error, right) of the predicted motion

on adult brain slices for sagittal, axial, and coronal acquisitions.

Sagittal

Axial

Coronal

Motion MSE (voxels)

Motion EPE (voxels)

Sagittal

Axial

Coronal

Epochs

Motion MSE (voxels)

All AB2 AB ADH B39 CO GSP MC OAS PP UKB

Figure 7: Motion accuracy across datasets. We plot the MSE of

the motion stacks predicted on 500 held-out slice stacks (1mm

!"#$$"

%$&%'(")*%+*,"-.$/+0%+*,"+#"1002.1%$"%*"3"455"*,"16$.17$("817+%%19"

1,/"0*.*,19"5*%+*,"-.$/+0%+*,#"%$,/"%*":$"5*.$"1002.1%$"%;1,"1&+19(!

Sagittal

Axial

Coronal

5.2. Single-Stack SVR of Fetal Brains

! Z,%,G!3,!64%*0,!TW!g>3!/,0*9!2%*(&!*09*+,+!*&<!%,/,%,&6,!

%,6$&+0%460($&+!*6%$++!01,![Bk!#,0*9!U09*+,+!NS>P!*&<!#,gU!

NSFP!=VJW--

G!*9(.&,<!0$!*.,'-*061,<![Bk!*09*+,+@!*&<!+89(0!

01,-!(&0$!WR!0%*(&(&.!*&<!">!)*9(<*0($&!#,gU!,:*-89,+J!^,!

4+,!03$!%,*9!g>3!+9(6,!+0*6E+!/%$-!ACUk!N"RP!/$%!0,+0J!C&!01,!

6*+,!$/!/,0*9!HIBG!3,!0%*(&!*!+(&.9,!+89*0O+9(6,!&,03$%E!$&!

*99!8$++(29,!+9(6(&.!<(%,60($&+!27!*8897(&.!*!%*&<$-!%$0*0($&!

=p49,%!*&.9,+!2,03,,&!q"WVr@!0$!$4%!0%*(&(&.!,:*-89,+J!^,!

6$-8*%,!$4%!-,01$<!3(01!+0*0,'$/'01,'*%0!HIB&,0!N>VP!*&<!

HI$Bg!N>YPo!+,,!U88,&<(:!UJ!_80(-(X*0($&'2*+,<!-,01$<+!

<$!&$0!3$%E!$&!+(&.9,!+0*6E+!*&<!*%,!&$0!6$-8*%,<!*.*(&+0J!

3D Reconstruction and Interpolation.!#(.4%,!T!)(+4*9(X,+!

01,!%,6$&+0%460($&+!$20*(&,<!4+(&.!$4%!8%,<(60,<!-$0($&J!^,!

*9(.&!#,gU!%,6$&+0%460($&+!0$!01,(%!.,+0*0($&*9!*.,'-*061,<!

*09*+!NS>P!/$%!6$-8*%(+$&!*6%$++!<(e,%,&0!-,01$<+J!HIB&,0!

N>VP!6$-89,0,97!/*(9+!0$!*9(.&!+9(6,+!(&!-$+0!$/!01,!6*+,+!*&<!

HI$Bg!=)>@!(&0%$<46,+!9*%.,!+9(6,!-(+*9(.&-,&0+G!9,*<(&.!0$!

+42$80(-*9!F?!%,6$&+0%460($&!3(01!*88*%,&0!<(+0$%0($&J!=M,!

HI$Bg!*&<!HIB&,0!%,6$&+0%460($&+!*%,!8%$6,++,<!4+(&.!$4%!

(&0,%8$9*0($&J@!_4%!%,6$&+0%460($&+!*%,!-$%,!/*(01/49!0$!01,!

4&<,%97(&.!F?!)$94-,+!48!0$!*-2(.4(0(,+!/%$-!+9(6(&.J!M,!

9*+0!03$!%$3+!)(+4*9(X,!%,6$&+0%460($&!$/!01,!03$!%,*9!ACUk!

*6D4(+(0($&+!/$%!31(61!&$!.%$4&<!0%401!,:(+0+!*&<!-$0($&!(+!

+-*99,%J!K&9(E,!HIB&,0G!01,!8%$8$+,<!-,01$<!.,&,%*9(X,+!0$!

%,*9!*6D4(+(0($&+!,),&!31,&!01,!&,03$%E!-$<,9!(+!0%*(&,<!$&!

+7&01,0(6!+9(6,!+0*6E+!3(01!<(e,%,&0!-$0($&!*&<!*6D4(+(0($&!

8*%*-,0,%+G!,J.JG!hH#J!H,,!U88,&<(:!]!/$%!-$%,!,:*-89,+J!

Quantitative Results.!g*29,! "!9(+0+!01,!8%,<(60($&!*664%*67!

$/!<(e,%,&0!-,01$<+!$&!)*9(<*0($&!+425,60+!3(01!Q!/$9<+J!C&!

*<<(0($&!0$!-$0($&!phpG!3,!%,8$%0!01,!+9(6,!hH\BG!$20*(&,<!

27!+9(6(&.!01,!0%4,!)$94-,!3(01!01,!8%,<(60,<!-$0($&G!*&<!01,!

hH\B!$/!01,!a&*9!%,6$&+0%460($&J!M,!01%,,'+0*6E!%,+490+!*%,!

/%$-!N>YPG!31(61!9(+0+!01,!*&61$%!8$(&0!,%%$%+!=Uhp@J!C&!01,!

+(&.9,'+0*6E!6*+,G!$4%!-,01$<!%,<46,+!,%%$%!(&!01,!8%,<(60,<!

-$0($&!27!SSJRs!*&<!QQJRs!%,9*0(),!0$!01,!HIB&,0!N>VP!*&<!

HI$Bg!=)>@!8%,<(60($&+G!%,+8,60(),97J!M,!+9(6,!*&<!)$94-,!

hH\B!1*),!*9+$!(-8%$),<!240!4&9(E,!01,!phpG!01,+,!-,0%(6+!

<,8,&<!$&!01,!hH#!$/!*6D4(+(0($&!*&<!%,6$&+0%460($&!*&<!<$!

&$0!<,a&(0(),97!-,*+4%,!*9(.&-,&0!*664%*67J!

ADHD200 8628223

ABIDE 50975

MCIC A00036476

Buckner39 990921

(a) Slice Stack

(b) Splat (Ours, True Motion)

(d) True Volume

Figure 8: SVR of adult brain scans. <$"6+#219+=$"*2."8>?".$#29%#"*,"#17+%%19"@.*A#"4BC'!"1&+19"@.*A"D'!"1,/"0*.*,19"@91#%".*A'"#9+0$"#%10E#"

#F,%;$#+=$/"2#+,7".1,/*5"#9+0$"5*%+*,"@1'("G#+,7"%;$"5*%+*,"#%10E"-.$/+0%$/":F"*2.",$%A*.E!"A$"#-91%"#9+0$"/1%1"%*".$0*,#%.20%"%;$"2,/$.9F+,7"

DH"6*925$"@:'("G#+,7"*2."-.$I%.1+,$/"+,%$.-*91%*.!"A$"%;$,"+,%$.-*91%$"%;$"5+##+,7"+,%$,#+%+$#"@;*9$#'"+,"*2.".$0*,#%.20%+*,"@0'("J$".$#29%"+#"

#+5+91."%*"%;$"%.2$"DH"6*925$"@/'("<$"1//+%+*,199F"6+#219+=$"+,"@:'"1,/"@0'"%;$"#-91%"1,/"+,%$.-*91%$/".$#29%#"A;$,"%;$"%.2$"5*%+*,"+#"2#$/(""

Sagittal Acquisition EPE: 1.66mm EPE: 0.00mm Sagittal View

Axial Acquisition EPE: 2.20mm EPE: 0.00mm Coronal View

Sagittal Acquisition EPE: 2.04mm EPE: 0.00mm Axial View

Coronal Acquisition EPE: 3.36mm EPE: 0.00mm Sagittal View

6. Discussion

Limitations.!_4%!/,0*9!HIB!&,03$%E!(+!64%%,&097!0%*(&,<!$&!

*40$-*0(6*997!+,.-,&0,<!2%*(&!+9(6,+J!g%*(&(&.!3(01!$%(.(&*9!

4&+,.-,&0,<!+9(6,!+0*6E+!6*&!(-8%$),!01,!%$24+0&,++!$/!$4%!

*88%$*61!*.*(&+0!(-8,%/,6097!.,&,%*0,<!+,.-,&0*0($&+J!

Future Work.!c(),&!01*0!$4%!*88%$*61!8%,<(60+!<,&+,!+9(6,!

-$0($&!/$%!%,6$&+0%460($&G!3,!89*&!0$!,:0,&<!$4%!*88%$*61!0$!

8%$29,-+!31,%,!01,!*6D4(%,<!+9(6,+!6*&!4&<,%.$!<,/$%-*29,!

-$0($&;,J.J!(&!/,0*9!0$%+$!$%!89*6,&0*9!%,6$&+0%460($&!N"WP;

*&<!6*&&$0!2,!0*6E9,<!4+(&.!%(.(<!-$0($&!HIBJ!U9+$G!2,00,%!

(&0,%8$9*0($&!&,03$%E+G!+461!*+!01$+,!01*0!*%,!,D4()*%(*&0!0$!

%(.(<!-$0($&G!6*&!/4%01,%!(-8%$),!%,6$&+0%460($&o!+,,!NSQPJ!!

7. Conclusion

C&!2%*(&!(-*.(&.G!+9(6,'0$')$94-,!%,6$&+0%460($&! =HIB@!

(+!*&!(-8$%0*&0!6$-840*0($&*9!0,61&(D4,!/$%!01,!(-*.(&.!$/!

+425,60+!31$+,!-$0($&!6*&&$0!,*+(97!2,!6$&0%$99,<J!i,0G!01,!

-*5$%(07!$/!HIB!0,61&(D4,+!8%,)($4+97!8%$8$+,<!<$!&$0!%,*8!

01,!2,&,a0+!$/!#[\!-$<,9+!01*0!1*),!.%*<4*997!2,6$-,!01,!

-*(&+0*7!(&!69$+,97!%,9*0,<!0*+E+G!+461!*+!(-*.,!%,.(+0%*0($&!

*&<!+,.-,&0*0($&J!M,!/,3!9,*%&(&.'2*+,<!HIB!0,61&(D4,+!

%,+$%0!0$!8%,0%*(&,<!C-*.,\,0!69*++(a6*0($&!*&<!,),&!)(+($&!

0%*&+/$%-,%!-$<,9+J!\$0!$&97!<$,+!$4%!3$%E!a99!01,!.*8!(&!

01,!HIB!-,01$<!<,),9$8-,&0!0(-,9(&,!240!(0!*9+$!+1$3+!01*0!

*!#[\!-$<,9!6*&!8%$<46,!+0*0,'$/'01,'*%0!HIB!$406$-,+J!

Acknowledgments. HCi!01*&E+!A*%.1,%(0*!#(%,&X,!/$%!1,%!

01$4.10'8%$)$E(&.!D4,+0($&+J!h%(-*%7!+488$%0!.(),&!27!01,!

\CZ!.%*&0+!dTTUcVW"QTF!*&<!B#"AZ">F"TYJ!H488$%0,<!

27!\CZ!BV"UcVRQV>SG!B>"UcVW>VW>!*&<!hQ"p]VFVVVRJ!

FeTA Sub 073

FeTA Sub 053

MIAL001

Run

MIAL001

Run 1

(a) Slice Stack

(b) SVRnet

†

[20]

(d) Interp (Ours)

(f) True Volume

Figure 9: Single-stack fetal SVR. <$"6+#219+=$"%;$"8>?".$#29%#"*,"619+/1%+*,"#2:K$0%#"L.*5"%;$"M$NO"/1%1#$%"PQDR"1,/"%A*".$19"10S2+#+%+*,#""

L.*5")TOU"P4VR("W**5$/"+,"X&"L*.":$%%$."6+#+:+9+%F("Y2.".$#29%#"09*#$9F".$#$5:9$"%;$"7.*2,/"%.2%;"6*925$#"A;+9$"8>*?N6C"1,/"8>?,$%"

.$0*,#%.20%+*,#"@A+%;"*2."+,%$.-*91%+*,'"$&;+:+%"#-1%+19"/+#%*.%+*,"L.*5"+,1002.1%$"#9+0$"19+7,5$,%("

†

Y2."+5-9$5$,%1%+*,Z"#$$"O--$,/+&"O(""

Random Acquisition EPE: 8.69mm EPE: 3.70mm EPE: 1.49mm Pathological

Sagittal Acquisition EPE: Unknown EPE: Unknown EPE: Unknown

Random Acquisition EPE: 5.72mm EPE: 2.61mm EPE: 1.45mm Neurotypical

Sagittal Acquisition EPE: Unknown EPE: Unknown EPE: Unknown

Unknown

)$%;*/"

O[\])*%+*,"

\[\"@55'

89+0$""

[8^?"@/_'"

>* 9 2 5 $ "

[8^?"@/_'"

N+5$"

@#$0'"

D"8%10E#

8>?,$%`"PCaR"

4C(bCcd(Ve"

Ca(dDc4(VC"

4e(dXc4(dC"

[91,$T,>*9`"PCXR"

4C(XecV(QD"

4e(eVc4(QD"

4b(ebc4(VC"

8>*?N`"PCdR"

X(Ddca(ea"

Cd(CVc4(bV"

CD(DCc4(XC"

4"8%10E

8>?,$%

†

"PCaR"

b(abcC(Dd"

4D(XVc4(dV"

4V(aDc4(Cb"

0.011s

8>*?N"@6C'"PCdR"

D(CQca(Q4"

Ca(QQc4(Cb"

4b(Xec4(VD"

a(4XC#"

[.*-*#$/"@Y2.#'"

1.81

0.40

23.69c1.39

23.43c1.40

a(CCX#"

Table 1: Validati o n a ccuracy."<$"9+#%"%;$"5*%+*,"1,/"DH"6*925$

.$0*,#%.20%+*,"1002.10F"*L"/+f$.$,%"5$%;*/#("`?$#29%#"%1E$,"L.*5

PCdR("N+5$/"*,"?Ngbaaa("

†

Y2."+5-9$5$,%1%+*,Z"#$$"O--$,/+&"O("

References

" P4R" )+%#2;+.*"^+#;+/1"$%"19("H$%1+9$/"#$5+12%*51%$/")?T":1#$/"

5*.-;*5$%.F" *L" %;$" ,$*,1%19" :.1+,h" [.$9+5+,1.F" .$#29%#("

NeuroImage!"DC@D'h4aX4B4aXe!"CaaV("

" PCR" i;1*"j("U+2"$%"19("k21,%+l01%+*,"*L"6*925$%.+0"5*.-;*5$%.F"

1,/" *-%+019" -.*-$.%F" +," %;$" 0*.%$&" *L" ;251," 0$.$:$9925" 1%"

5+0.*5$%$.".$#*92%+*,("NeuroImage!"CXXh44bVCQ!"CaC4("

" PDR" )1&A$99"U("\99+*%%"$%"19("_.1+,"5*.-;*5$%.F"+,"*9/$."1/29%#"

A+%;"1,/"A+%;*2%" /$5$,%+1"2#+,7"$&%.$5$9F".1-+/" #%.20%2.19"

#01,#("NeuroImage!"CQVh4Ca4QD!"CaCD("

" PXR" _.20$"M+#0;9("M.$$82.L$.("NeuroImage!"VC@C'hQQXBQb4!"Ca4C("

" PdR" O5F" W;1*!" m2;1" _191E.+#;,1,!" M.$/*" H2.1,/!" j*;," >("

m2%%17!" 1,/" O/.+1," >(" H1901(" H1%1" 1275$,%1%+*," 2#+,7"

9$1.,$/" %.1,#L*.51%+*,#" L*." *,$I#;*%" 5$/+019" +517$"

#$75$,%1%+*,("T,"Proc. CVPR!"Ca4e("

" PVR" U+991" Wn99$+!" j21," \27$,+*" T79$#+1#!"o1,75+,7" Y2!" [(" \99$,"

m.1,%!" 1,/" _.20$" M+#0;9(" T,L1,%" M.$$82.L$.h"O," 12%*51%$/"

#$75$,%1%+*,"1,/"#2.L10$"$&%.10%+*,"-+-$9+,$"L*."N4IA$+7;%$/"

,$2.*+517+,7" /1%1" *L" +,L1,%#" aBC" F$1.#(" NeuroImage!"

C4bh44VeXV!"CaCa("

" PQR" O91," i(" \61,#!" O,/.$A" U(" j1,E$!" H(" U*2+#" i*99+,#!" 1,/"

8F961+," _1+99$%(" _.1+," %$5-91%$#" 1,/" 1%91#$#(" NeuroImage!"

VC@C'he44BeCC!"Ca4C("

" PbR" 8;1/1:" p;1," $%" 19(" M$%19" :.1+," 7.*A%;" -*.%.1F$/" :F" 1"

#-1%+*%$5-*.19"/+f2#+*,"%$,#*.")?T"1%91#"0*5-2%$/"L.*5"+,"

2%$.*"+517$#("NeuroImage!"4bdhdeDBVab!"Ca4e("

" PeR" O/.+q" i1#15+%K1,1!" 1,/" j21," \27$,+*" T79$#+1#(" r+7;I

.$#*92%+*," 1%91#+,7" 1,/" #$75$,%1%+*," *L" %;$" #2:0*.%$&h"

?$6+$A" 1,/" -$.#-$0%+6$" *," 0;199$,7$#" 1,/" *--*.%2,+%+$#"

0.$1%$/" :F" 510;+,$" 9$1.,+,7(" NeuroImage!" CVDh44eV4V!"

CaCC("

"P4aR" 8;2=;*2" j+1,7!" r2+" g2$!" O91," m9*6$.!" )1.F" ?2%;$.L*./!"

H1,+$9" ?2$0E$.%!" 1,/" j*#$-;" >(" r1K,19(" )?T" *L" 5*6+,7"

#2:K$0%#" 2#+,7" 529%+#9+0$" #,1-#;*%" +517$#" A+%;" 6*925$"

.$0*,#%.20%+*," @8>?'h" O--9+01%+*," %*" L$%19!" ,$*,1%19!" 1,/"

1/29%":.1+," #%2/+$#(" IEEE Trans. Med. Imaging!"CV@Q'heVQB

eba!"CaaQ("

"P44R" O9+" m;*9+-*2.!" j2/F" O(" \#%.*f!" 1,/" 8+5*," p(" <1.l$9/("

?*:2#%" #2-$.I.$#*92%+*," 6*925$" .$0*,#%.20%+*," L.*5" #9+0$"

10S2+#+%+*,#h" O--9+01%+*," %*" L$%19" :.1+," )?T(" IEEE Trans.

Med. Imaging!"Ce@4a'h4QDeB4Qdb!"Ca4a("

"P4CR" p+*"p+5!"[+*%."O("r1:1#!"M.1,0*+#"?*2##$12!"Y.+%"O("m9$,,!"

O,%;*,F" j(" _1.E*6+0;!" 1,/" i*9+," 8%2/;*95$(" T,%$.#$0%+*,"

:1#$/"5*%+*,"0*..$0%+*,"*L"529%+#9+0$")?T"L*."DIH"+,"2%$.*"

L$%19" :.1+," +517$" L*.51%+*,(" IEEE Trans. Med. Imaging!"

Ce@4'h4XVB4db!"Ca4a("

"P4DR" )1.+1" p2E9+#*61I)2.71#*61!" m$.1./+,$" k217;$:$2.!" )1.F"

O(" ?2%;$.L*./!" j*#$-;" >(" r1K,19!" 1,/" j29+1" O(" 80;,1:$9("

?$0*,#%.20%+*," *L" L$%19" :.1+," )?T" A+%;" +,%$,#+%F" 51%0;+,7"

1,/" 0*5-9$%$" *2%9+$." .$5*619(" Med. Image Anal.!"

4V@b'h4ddaB4dVX!"Ca4C("

"P4XR" _$.,;1./" p1+,=" $%" 19(" M1#%" 6*925$" .$0*,#%.20%+*," L.*5"

5*%+*," 0*..2-%$/" #%10E#" *L" CH" #9+0$#(" IEEE Trans. Med.

Imaging!"DX@e'h4ea4B4e4D!"Ca4d("

"P4dR" O5+."O91,#1.F"$%"19("[>?h"[1%0;I%*I>* 9 2 5 $ " ? $ 0 * , # % . 2 0 % + * , "

L*."91.7$"1.$1"5*%+*," 0*..$0%+*,"*L" L$%19" )?T("IEEE Trans.

Med. Imaging!"DV@4a'hCaD4BCaXX!"Ca4Q("

"P4VR" 8s:1#%+$," N*2.:+$." $%" 19(" O2%*51%$/" %$5-91%$I:1#$/" :.1+,"

9*019+=1%+*,"1,/"$&%.10%+*,"L*."L$%19":.1+,")?T".$0*,#%.20%+*,("

NeuroImage!"4ddhXVaBXQC!"Ca4Q("

"P4QR" )+0;1$9" \:,$." $%" 19(" O," 12%*51%$/" L.15$A*.E" L*."

9*019+=1%+*,!" #$75$,%1%+*," 1,/" #2-$.I.$#*92%+*,"

.$0*,#%.20%+*,"*L"L$%19":.1+,")?T("NeuroImage!"CaVh44VDCX!"

CaCa("

"P4bR" O9$,1" G2#!" N*,7" W;1,7!" U12.$,0$" r(" j10E#*,!" J*51#"O("

?*:$.%#!" )1.F"O(" ?2%;$.L*./!" j*#$-;">(" r1K,19!" 1,/" )1.+1"

H$-.$=("H$L*.51:9$"#9+0$I%*I6*925$".$7+#%.1%+*,"L*."5*%+*,"

0*..$0%+*,"*L"L$%19":*/F"1,/"-910$,%1")?T("IEEE Trans. Med.

Imaging!"De@e'hCQdaBCQde!"CaCa("

"P4eR" M.1,0*+#"?*2##$12!"Y.+%"O("m9$,,!"_+#%.1"T*./1,*61!"i912/+1"

?*/.+72$=Ii1..1,=1!" H1,+$9" _(" >+7,$.*,!" j15$#" O("

_1.E*6+0;!" 1,/" i*9+," 8%2/;*95$(" ?$7+#%.1%+*,I:1#$/"

1--.*10;"L*.".$0*,#%.20%+*,"*L";+7;I.$#*92%+*,"+,"2%$.*"L$%19"

)?":.1+,"+517$#("Acad. Radiol.!"4D@e'h4aQCB4ab4!"CaaV("

"PCaR" _$,K15+," r*2" $%" 19(" DIH" ?$0*,#%.20%+*," +," 01,*,+019" 0*I

*./+,1%$" #-10$" L.*5" 1.:+%.1.+9F" *.+$,%$/" CIH" T517$#(" IEEE

Trans. Med. Imaging!"DQ@b'h4QDQB4Qda!"Ca4b("

"PC4R" _$,K15+,"r*2"$%"19("i*5-2%+,7"i^^"9*##"1,/"7.1/+$,%#"L*."

-*#$" $#%+51%+*," A+%;" ?+$51,,+1," 7$*5$%.F(" T," Proc.

MICCAI!"Ca4b("

"PCCR" 8$F$/" 81/$7;" )*;#$,+" 819$;+!" 8;1/1:" p;1,!" H$,+="

\./*752#!" 1,/" O9+" m;*9+-*2.(" ?$19I%+5$" /$$-" -*#$"

$#%+51%+*," A+%;" 7$*/$#+0" 9*##" L*." +517$I%*I%$5-91%$" .+7+/"

.$7+#%.1%+*,(" IEEE Trans. Med. Imaging!" Db@C'hXQaBXb4!"

Ca4e("

"PCDR" o2 0; $, " [ $ +! " U + #; $, 7" <1 ,7 !" M $, S+ 1 ,7 " W ;1 *! " N1 *" W ;* ,7 !"

U2L1,"U+1*!"H+,771,7"8;$,!"1,/"m1,7"U+("O,1%*5FI72+/$/"

0*,6*92%+*,19",$2.19" ,$%A*.E" L*."5*%+*,"0*..$0%+*," +,"L$%19"

:.1+,")?T("T,"Proc. MLMI!"CaCa("

"PCXR" [1EIr$+" o$2,7!" )*#E1" O9+1#+!" O.+#" N(" [1-17$*.7;+*2!"

)*,+S2$" r11E!" <$+/+" g+$!" 1,/" O,1" T(" U(" ^15:2.$%$("

U$1.,+,7" %*"51-" CH"29%.1#*2,/"+517$#" +,%*"DH" #-10$"A+%;"

5+,+519" ;251," 1,,*%1%+*,(" Med. Image Anal.!" Qah4a4eeb!"

CaC4("

"PCdR" j2,#;$,"g2!"H1,+$9")*F$.!"[("\99$,"m.1,%!"[*9+,1"m*991,/!"

j21," \27$,+*" T79$#+1#!" 1,/" \9L1." O/19#%$+,##*,(" 8>*?Nh"

T%$.1%+6$"%.1,#L*.5$."L*."#9+0$I%*I6*925$".$7+#%.1%+*,"+,"L$%19"

:.1+,")?T("T,"Proc. MICCAI!"CaCC("

"PCVR" \9+#$"N2.E"$%"19("M2,0%+*,19"0*,,$0%*5$"*L"%;$"L$%19":.1+,("J.

Neurosci.!"De@Xe'heQ4VBeQCX!"Ca4e("

"PCQR" H1,+$9" 8*:*%E1" $%" 19(" )*%+*," 0*..$0%+*," 1,/" 6*925$%.+0"

.$0*,#%.20%+*," L*." L$%19" L2,0%+*,19" 517,$%+0" .$#*,1,0$"

+517+,7"/1%1("NeuroImage!"Cddh44eC4D!"CaCC("

"PCbR" g+1*;21," i1*!" j+1,;21" o1,7!" j2," W;1,7!" H*,7" ^+$!"

)+,K$*,7"p+5!"k+1,"<1,7!"1,/"H+,771,7"8;$,("H$L*.51:9$"

+517$" .$7+#%.1%+*," :1#$/" *," #+5+91.+%FI#%$$.$/" i^^"

.$7.$##+*,("T,"Proc. MICCAI!"Ca4Q("

"PCeR" m2;1" _191E.+#;,1,!" O5F" W;1*!" )$.%" ?(" 81:2,02!" j*;,"

m2%%17!"1,/"O/.+1,">("H1901("O,"2,#2-$.6+#$/"9$1.,+,7"5*/$9"

L*."/$L*.51:9$"5$/+019"+517$".$7+#%.1%+*,("T," Proc. CVPR!"

Ca4b("

"PDaR" N*,F" i(" <(" )*E!" 1,/"O9:$.%" i(" 8(" i;2,7(" M1#%" #F55$%.+0"

/+f$*5*.-;+0" +517$" .$7+#%.1%+*," A+%;" 0*,6*92%+*,19" ,$2.19"

,$%A*.E#("T,"Proc. CVPR!"CaCa("

"PD4R" 8$1,"T("o*2,7!"o1t9"_19:1#%.$!"O/.+1,">("H1901!"<+99+15")("

"V!

<$99#!" j21," \27$,+*" T79$#+1#!" 1,/" _.20$" M+#0;9(" 82-$.<1.-h"

82-$.6+#$/" 9$1.,+,7" 1,/" A1.-+,7" *," GI^$%" L*." +,61.+1,%"

#2:6*&$9I-.$0+#$".$7+#%.1%+*,("T,"Proc. WBIR!"CaCC("

"PDCR" )+1*" p1,7!" g+1*K2,"r2!"<$+9+," r21,7!" )1%%;$A"?(" 80*%%!"

1,/" )12.+0+*" ?$F$#(" H219I#%.$15" -F.15+/" .$7+#%.1%+*,"

,$%A*.E("Med. Image Anal.!"Qbh4aCDQe!"CaCC("

"PDDR" O9$&$F" H*#*6+%#E+F" $%"19(" M9*A^$%h" U$1.,+,7" Y-%+019" M9*A"

<+%;"i*,6*92%+*,19"^$%A*.E#("T,"Proc. ICCV!"Ca4d("

"PDXR" j1#*,"j("o2!"O/15"<("r1.9$F!"1,/"p*,#%1,%+,*#"m("H$.-1,+#("

_10E" %*" :1#+0#h" G,#2-$.6+#$/" 9$1.,+,7" *L" *-%+019" u*A" 6+1"

:.+7;%,$##" 0*,#%1,0F" 1,/" 5*%+*," #5**%;,$##(" T," Proc.

ECCV!"Ca4V("

"PDdR" \//F"T97!"^+E*912#")1F$.!"N*,5*F"81+E+1!")1.7.$%"p$2-$.!"

O9$&$F" H*#*6+%#E+F!" 1,/" J*51#" _.*&(" M9*A^$%" C(ah"

\6*92%+*,"*L"*-%+019"u*A"$#%+51%+*,"A+%;"/$$-",$%A*.E#("T,"

Proc. CVPR!"Ca4Q("

"PDVR" H$S+,7" 82,!"g+1*/*,7"o1,7!" )+,7Io2" U + 2! " 1 , /" j 1 ," p 1 2% =( "

[<iI^$%h" i^^#" L*." *-%+019" u*A" 2#+,7" -F.15+/!" A1.-+,7!"

1,/"0*#%"6*925$("T,"Proc. CVPR!"Ca4b("

"PDQR" O,2.17" ?1,K1,!" 1,/" )+0;1$9" j(" _910E(" Y-%+019" u*A"

$#%+51%+*,"2#+,7"1"#-1%+19"-F.15+/",$%A*.E("T,"Proc. CVPR!"

Ca4Q("

"PDbR" [$,7-$,7" U+2!" )+0;1$9" UF2!" T.A+," p+,7!" 1,/" j+1" g2("

8$9M9*Ah" 8$9LI#2-$.6+#$/" 9$1.,+,7" *L" *-%+019" u*A(" T," Proc.

CVPR!"Ca4e("

"PDeR" j2,;A1"r2.!"1,/"8%$L1," ?*%;("T%$.1%+6$" .$#+/219".$l,$5$,%"

L*." K*+,%" *-%+019" u*A" 1,/" *0092#+*," $#%+51%+*,(" T," Proc.

CVPR!"Ca4e("

"PXaR" M1F1*" U+2!" i;2,;21" 8;$,!" 1,/" m2*#;$,7" U+,(" H$$-"

0*,6*92%+*,19",$2.19"l$9/#"L*."/$-%;"$#%+51%+*,"L.*5"1"#+,79$"

+517$("T,"Proc. CVPR!"Ca4d("

"PX4R" j1$Ir1," U$$!" 1,/" i;1,7I82" p+5(" )*,*0291." /$-%;"

$#%+51%+*,"2#+,7".$91%+6$"/$-%;"51-#("T,"Proc. CVPR!"Ca4e("

"PXCR" i9$5$,%" m*/1./!" Y+#+," )10"O*/;1!" )+0;1$9" M+.51,!" 1,/"

m1:.+$9"j("_.*#%*A("H+77+,7"+,%*"#$9LI#2-$.6+#$/"5*,*0291."

/$-%;"$#%+51%+*,("T,"Proc. ICCV!"Ca4e("

"PXDR" 8;1.+S"M1.**S" _;1%!"T:.1;$$5"O9;1#;+5!"1,/"[$%$."<*,E1("

O/1_+,#h" H$-%;" $#%+51%+*," 2#+,7" 1/1-%+6$" :+,#(" T," Proc.

CVPR!"CaC4("

"PXXR" j+1"H$,7!"<$+"H*,7!"?+0;1./"8*0;$.!"U+Ij+1"U+!"p1+"U+!"1,/"

U+" M$+IM$+(" T517$^$%h" 1" 91.7$I#019$" ;+$.1.0;+019" +517$"

/1%1:1#$("T,"Proc. CVPR!"Caae("

"PXdR" O9$&$F" H*#*6+%#E+F"$%"19("O," +517$"+#" A*.%;" 4V&4V"A*./#h"

N.1,#L*.5$.#"L*."+517$".$0*7,+%+*,"1%"#019$("T,"Proc. ICLR!"

CaCa("

"PXVR" O,2.17"O.,1:!")*#%1L1"H$;7;1,+!"m$*.7"r$+7*9/!"i;$,"82,!"

)1.+*" U2v+w!" 1,/" i*./$9+1" 80;5+/(" >+>+Nh"O" 6+/$*" 6+#+*,"

%.1,#L*.5$.("T,"Proc. ICCV!"CaC4("

"PXQR" W$" U+2" $%" 19(" 8A+," N.1,#L*.5$.h" r+$.1.0;+019" 6+#+*,"

%.1,#L*.5$."2#+,7"#;+L%$/"A+,/*A#("T,"Proc. ICCV!"CaC4("

"PXbR" j$.$5+$" O,S2$=!" \9#1" H(" O,7$9+,+!" 1,/" T#1:$99$" _9*0;("

O2%*51%+0"#$75$,%1%+*,"*L";$1/"#%.20%2.$#"*,"L$%19")?T("T,"

Proc. ISBI!"Caae("

"PXeR" _$.,;1./"p1+,=!"p$6+,"p$.12/.$,!">1,$##1"pF.+1E*-*29*2!"

)1.F" ?2%;$.L*./!" j*#$-;" >(" r1K,19!" 1,/" H1,+$9" ?2$0E$.%("

M1#%"L299F"12%*51%+0":.1+,"/$%$0%+*,"+,"L$%19")?T"2#+,7"/$,#$"

.*%1%+*,"+,61.+1,%"+517$"/$#0.+-%*.#("T,"Proc. ISBI!"Ca4X("

"PdaR" p$6+,"p$.12/.$,!">1,$##1"pF.+1E*-*29*2!")1.F"?2%;$.L*./!"

j*#$-;">(" r1K,19!" 1,/" H1,+$9"?2$0E$.%(" U*019+#1%+*,"*L" %;$"

:.1+," +," L$%19" )?T" 2#+,7" :2,/9$/" 8TMN" L$1%2.$#(" T," Proc.

MICCAI!"Ca4D("

"Pd4R" p$6+,"p$.12/.$,"$%" 19("O2%*51%$/" L$%19":.1+,"#$75$,%1%+*,"

L.*5" CH" )?T" #9+0$#" L*." 5*%+*," 0*..$0%+*,(" NeuroImage!"

4a4hVDDBVXD!"Ca4X("

"PdCR" )1.%+," ?1K0;9" $%" 19(" U$1.,+,7" 2,/$." /+#%.+:2%$/" A$1E"

#2-$.6+#+*,!";%%-h]]1.&+6(*.7]1:#]4VaV(a44aa!"Ca4V("

"PdDR" 8$F$/"81/$7;")*;#$,+"819$;+"$%"19("?$19I%+5$"12%*51%+0"L$%19"

:.1+,"$&%.10%+*,"+,"L$%19")?T":F"/$$-"9$1.,+,7("T,"Proc. ISBI!"

Ca4b("

"PdXR" Y91L"?*,,$:$.7$.!"[;+9+--"M+#0;$.!"1,/"J*51#"_.*&("GI^$%h"

0*,6*92%+*,19",$%A*.E#"L*.":+*5$/+019"+517$"#$75$,%1%+*,("

T,"Proc. MICCAI!"Ca4d("

"PddR" j2,#;$," g2!"H1,+$9" )*F$.!" _*.K1,"m17*#E+!" j21," \27$,+*"

T79$#+1#!" [(" \99$," m.1,%!" [*9+,1" m*991,/!" 1,/" \9L1."

O/19#%$+,##*,(" ^$8>*?h" T5-9+0+%" ,$2.19" .$-.$#$,%1%+*," L*."

#9+0$I%*I6*925$" .$0*,#%.20%+*," +," )?T(" IEEE Trans. Med.

Imaging!"XC@V'h4QaQB4Q4e!"CaCD("

"PdVR" _$," )+9/$,;199!" [.1%29" [(" 8.+,+61#1,!" )1%%;$A" N1,0+E!"

j*,1%;1,"N("_1..*,!"?16+"?1515**.%;+!"1,/"?$,"^7("^$?Mh"

?$-.$#$,%+,7" #0$,$#" 1#" ,$2.19" .1/+1,0$" l$9/#" L*." 6+$A"

#F,%;$#+#!"T,"Proc."ECCV!"CaCa("

"PdQR" 81;1." ^(" 819$$5(" M$%19" )?Th" O," 1--.*10;" %*" -.10%+0$h" O"

.$6+$A("J. Adv. Res.!"d@d'hdaQBdCD!"Ca4X("

"PdbR" r21I5$+"i;$,!"1,/"[(p(">1.#;,$F(")2%219"+,L*.51%+*,I:1#$/"

iNI)?" :.1+," +517$" .$7+#%.1%+*," 2#+,7" 7$,$.19+=$/" -1.%+19"

6*925$" K*+,%" ;+#%*7.15" $#%+51%+*,(" IEEE Trans. Med.

Imaging!"CC@e'h4444B444e!"CaaD("

"PdeR" O,/.$A"O/15#!"j*,75+,"_1$E!"1,/")F$.#"O:.1;15"H16+#("

M1#%" ;+7;I/+5$,#+*,19" l9%$.+,7" 2#+,7" %;$" -$.52%*;$/.19"

91%%+0$("Comput. Graph. Forum!"Ce@C'hQdDBQVC!"Ca4a("

"PVaR" ^+9#"[1-$,:$.7!"O,/.s#"_.2;,!"J*51#"_.*&!"8%$-;1,"H+/1#!"

1,/" j*10;+5" <$+0E$.%(" r+7;9F" 1002.1%$" *-%+0" u*A"

0*5-2%1%+*," A+%;" %;$*.$%+0199F" K2#%+l$/" A1.-+,7(" Int. J.

Comput. Vis.!"VQ@C'h4X4B4db!"CaaV("

"PV4R" ^+0;*91#"j("r+7;15("i*5-2%+,7"%;$"-*91."/$0*5-*#+%+*,x

A+%;" 1--9+01%+*,#(" SIAM J. Sci. Stat. Comput.!" Q@X'h44VaB

44QX!"4ebV("

"PVCR" O("H+")1.%+,*"$%"19("J$"12%+#5":.1+,"+517+,7"/1%1"$&0;1,7$h"

%*A1./#" 1" 91.7$I#019$" $61921%+*," *L" %;$" +,%.+,#+0" :.1+,"

1.0;+%$0%2.$" +," 12%+#5(" Mol. Psychiatry!" 4e@V'hVdeBVVQ!"

Ca4X("

"PVDR" )+0;1$9")+9;15!"H15+$,"M1+.!")11.%$,")$,,$#!"1,/"8%$A1.%"

)*#%*L#EF("J$"OHrHICaa"0*,#*.%+25h"1"5*/$9"%*"1/61,0$"

%;$" %.1,#91%+*,19" -*%$,%+19" *L" ,$2.*+517+,7" +," 09+,+019"

,$2.*#0+$,0$("Front. Syst. Neurosci.!"V!"Ca4C("

"PVXR" >+,0$" i19;*2,!" j+,7" 82+!" p$,%" p+$;9!" j$##+01" N2.,$.!" \9$,1"

O99$,!" 1,/" m*/L.$F" [$1.9#*,(" \&-9*.+,7" %;$" -#F0;*#+#"

L2,0%+*,19" 0*,,$0%*5$h" 1:$..1,%" +,%.+,#+0" ,$%A*.E#" +,"

#0;+=*-;.$,+1" 1,/" :+-*91." /+#*./$.(" Front. Psychiatry!" C!"

Ca4C("

"PVdR" O6.15" j(" r*95$#" $%" 19(" _.1+," m$,*5+0#" 82-$.#%.20%" [.*K$0%"

+,+%+19"/1%1".$9$1#$"A+%;"#%.20%2.19!"L2,0%+*,19!"1,/":$;16+*.19"

5$1#2.$#("Sci. Data!"C@4'h4daaD4!"Ca4d("

"PVVR" ?1,/F" U(" m*992:" $%" 19(" J$" )iTi" i*99$0%+*,h" 1" #;1.$/"

.$-*#+%*.F"*L"529%+I5*/19!"529%+I#+%$":.1+,"+517$"/1%1"L.*5"1"

09+,+019" +,6$#%+71%+*," *L" #0;+=*-;.$,+1(" Neuroinformatics!"

""!

44@D'hDVQBDbb!"Ca4D("

"PVQR" H1,+$9" 8(" )1.02#!" N.10F" r(" <1,7!" j15+$" [1.E$.!" j*;," m("

i#$.,1,#EF!" j*;," i(" )*..+#!" 1,/" ?1,/F" U(" _20E,$.(" Y-$,"

O00$##"8$.+$#"*L"T517+,7"8%2/+$#"@YO8T8'h"i.*##I#$0%+*,19"

)?T" /1%1" +," F*2,7!" 5+//9$" 17$/!" ,*,/$5$,%$/!" 1,/"

/$5$,%$/"*9/$."1/29%#("J. Cogn. Neurosci.!"4e@e'h4XebB4daQ!"

CaaQ("

"PVbR" p$,,$%;" )1.$E" $%" 19(" J$" [1.E+,#*," [.*7.$##+*," )1.E$."

T,+%+1%+6$"@[[)T'("Prog. Neurobiol.!"ed@X'hVCeBVDd!"Ca44("

"PVeR" i1%;+$" 82/9*A"$%"19(" Gp" _+*:1,Eh"1,"*-$," 100$##".$#*2.0$"

L*."+/$,%+LF+,7"%;$"012#$#"*L"1"A+/$".1,7$"*L"0*5-9$&"/+#$1#$#"

*L"5+//9$"1,/"*9/"17$("PLOS Med.!"4C@D'h$4aa4QQe!"Ca4d("

"PQaR" _.20$" M+#0;9" $%" 19(" <;*9$" :.1+," #$75$,%1%+*,h" O2%*51%$/"

91:$9+,7" *L" ,$2.*1,1%*5+019" #%.20%2.$#" +," %;$" ;251," :.1+,("

Neuron!"DD@D'hDX4BDdd!"CaaC("

"PQ4R" j("N191+.10;!"1,/"[+$..$"N*2.,*2&("i*I-91,1."#%$.$*%1&+0"1%91#"

*L"%;$";251,":.1+,h"DI/+5$,#+*,19"-.*-*.%+*,19" #F#%$5yh"1,"

1--.*10;" %*"0$.$:.19" +517+,7(" m("J+$5$yZ"J+$5$")$/+019"

[2:9+#;$.#!"8%2%%71.%!"^$A"o*.E"4ebb("

"PQCR" O9+"m;*9+-*2."$%"19("O",*.51%+6$"#-1%+*%$5-*.19")?T"1%91#"*L"

%;$" L$%19" :.1+," L*." 12%*51%+0" #$75$,%1%+*," 1,/" 1,19F#+#" *L"

$1.9F":.1+,"7.*A%;("Sci. Rep.!"Q@4'hXQV!"Ca4Q("

"PQDR" p$99F" [1F$%%$" $%" 19("O," 12%*51%+0" 529%+I%+##2$" ;251," L$%19"

:.1+," #$75$,%1%+*," :$,0;51.E" 2#+,7" %;$" M$%19" N+##2$"

O,,*%1%+*,"H1%1#$%("Sci. Data!"b@4'h4VQ!"CaC4("

PQXR"" H*,7/*,7"i;$,!")+E$"H16+$#!")1%%;+1#"j("\;.;1./%!"i1.*91I

_+:+1,$" 80;n,9+$:!" M$./+1" 8;$..F!" 1,/" j29+z," N10;$991("

T517+,7" A+%;" $S2+61.+$,%" /$$-" 9$1.,+,7h" L.*5" 2,.*99$/"

,$%A*.E"/$#+7,"%*"L299F"2,#2-$.6+#$/"9$1.,+,7("IEEE Signal

Process. Mag.! Xa@4'h4DXB4XQ!"CaCD("

! !

">!

A. Training and Validation

A.1 Training Details

! #$%!0%*(&(&.G!3,!8%,8*%,!8*(%,<!=+9(6,!+0*6EG!-$0($&!+0*6E@!

*&<!=+89*00,<!)$94-,G!.%$4&<!0%401!)$94-,@!0%*(&(&.!<*0*!27!

*8897(&.!*!%*&<$-!(+$0%$8(6!X$$-!=q>R!)$:,9+@G!1$%(X$&0*9!

f(8G!%$0*0($&!=p49,%!*&.9,+!q>Vr!/$%!*<490!2%*(&+!*&<!q"WVr!

/$%!/,0*9@!*&<!0%*&+9*0($&!=q"F!)$:,9+@!0$!01,!%,.(+0,%,<!>YR

%,/,%,&6,!)$94-,!="--

!*6D4(+(0($&G!g*9*(%*61!/$%!*<490!*&<!

VJW--

!#,gU!%,6$&+0%460($&G![Bk!/$%!/,0*9@!0$!a%+0!+(-49*0,!

01,!<(),%+(07!(&!+6*&&,<!+425,60+!*&<!)*%($4+!8$+,+!01*0!01,7!

-*7!2,!+6*&&,<!4&<,%J!g$!+(-49*0,!+9(6(&.G!3,!*8897!%*&<$-!

%$0*0($&+!=p49,%!*&.9,+!q>Vr@!*&<!0%*&+9*0($&+!=q>R!)$:,9+@!

01*0!)*%7!+-$$0197!*6%$++!+9(6,+!27!%*&<$-97!+*-89(&.!F>!0$!

RQ!%$0*0($&+!*&<!0%*&+9*0($&+!/%$-!01,!%*&.,+!+1$3&!*2$),!

*&<!+-$$0197!(&0,%8$9*0(&.!01,-!4+(&.!642(6!]!+89(&,+J!^,!

(&0,%9,*),!01,!03$!1*9),+!$/!-$0($&!0%*5,60$%(,+!0$!+(-49*0,!

03$'+1$0!+,D4,&6,+!078(6*997!4+,<!(&!>?!ABC!*6D4(+(0($&+J!

! H9(6,!*6D4(+(0($&!(+!+(-49*0,<!27!294%%(&.!01,!+9(6,+!4+(&.!

*!2$:6*%!hH#!=/$4%!)$:,9+!3(<,@!*9$&.!01,!+9(6(&.!<(%,60($&!

=+*.(00*9G!*:(*9!$%!6$%$&*9!/$%!*<490!2%*(&+G!*&<!+*.(00*9!(&!01,!

6*+,!$/!/,0*9G!31,%,!01,!q"WVr!p49,%!*&.9,+!-,*&!01*0!+9(6,+!

*%,!*6D4(%,<!*9$&.!%*&<$-!<(%,60($&+!*&73*7@!*&<!+*-89(&.!

,),%7!/$4%01!+9(6,!*9$&.!01,!+*-,!*:(+J!H9(6,!(&0,&+(0(,+!*%,!

-*&(849*0,<!27!*8897(&.!c*4++(*&!&$(+,!3(01!&$(+,!+0*&<*%<!

<,)(*0($&!𝜎 = 0.01!*&<!.*--*!*4.-,&0*0($&!3(01!,:8$&,&0!

𝛾 ∈

[

0.9,1

]

J!!#(&*997G!01,!*6D4(%,<!+9(6,+!*%,!%,89(6*0,<!*9$&.!

01,!+9(6(&.!<(%,60($&!27!*!/*60$%!$/!/$4%!*.*(&!+$!01*0!01,!+9(6,!

+0*6E+!1*),!(+$0%$8(6!(&'!*&<!01%$4.1'89*&,!%,+$940($&+J!^,!

+42+*-89,!01,!+9(6,!+0*6E+!27!*!/*60$%!$/!03$!=(J,JG!0$!>--

$%!"JR--

@!0$!0%*(&!01,!+9(6,!-$0($&!8%,<(60($&!&,03$%EJ!\$!

+42+*-89(&.!(+!8,%/$%-,<!31,&!0%*(&(&.!01,!(&0,%8$9*0$%J!

! ^,! 0%*(&! $4%!+9(6,!-$0($&!*&<!(&0,%8$9*0($&!&,03$%E+!/$%!

256,000!+0,8+!4+(&.!U?UAG!3(01!*&!(&(0(*9!9,*%&(&.!%*0,!$/!

−4

G!31(61!(+!%,<46,<!0$!0!3(01!8$97!+61,<49(&.!=,:8$&,&0!

$/!0.9@G!3,(.10!<,6*7!$/!0!*&<!-$-,&04-!$/!0.90J!^,!-*+E!

$40!2*6E.%$4&<!)$:,9+!31,&!6$-840(&.!01,!0%*(&(&.!9$++J!C&!

01,!6*+,!$/!/,0*9!HIBG!3,!$),%'+*-89,!01,!#,gU!8$%0($&!$/!

01,!0%*(&(&.!<*0*!a),/$9<G!*&<!01,![Bk!8$%0($&!0,&/$9<J!^,!

+(-897!8(6E!01,!-$<,9!$/!01,!9*+0!,8$61!/$%!-$<,9!+,9,60($&!

240!+0(99!-$&(0$%!)*9(<*0($&!-,0%(6+!/$%!8$0,&0(*9!$),%a00(&.J!

A.2 Validation Metrics

! H(-(9*%!0$!$4%!0%*(&(&.!9$++!=T@G!*99!$4%!)*9(<*0($&!-,0%(6+!

6$-8,&+*0,!/$%!*&7!.9$2*9!%(.(<!-$0($&!$e+,0!-*7!01*0!,:(+0!

2,03,,&!8%,<(60,<!*&<!0%4,!+9(6,!-$0($&o!+,,!=T@J!C&!*<<(0($&!

0$!01,!AHp!𝓛

MSE

(𝐮, 𝐲) = (1 𝑁

⁄

)

‖

𝐮 − 𝐲

‖

𝐹

!$/!01,!8%,<(60,<!

+9(6,!-$0($&!𝐮 ∈ ℝ

𝑁×3

!3J%J0J!0%4,!-$0($&!𝐲 ∈ ℝ

𝑁×3

G!3,!4+,!

01,!*),%*.,!,&<'8$(&0!,%%$%!=php@!-,0%(6!NFFPG!<,a&,<!*+!

𝓛

EPE

(𝐮, 𝐲) = (1 𝑁

⁄

)

‖

𝐮 − 𝐲

‖

2,1

=U"@!

01*0!(+G!01,!-,*&!p469(<,*&!<(+0*&6,!2,03,,&!01,!,&<'8$(&0+!

$/!03$!+9(6,!-$0($&!a,9<+!=2$01!-,0%(6+!+1$3&!1,%,!3(01$40!

%(.(<!6$-8,&+*0($&!/$%!69*%(07@J!C&!8%,)($4+!3$%E!N>VG!>YPG!*!

+(-(9*%!-,0%(6!(+!8%$8$+,<!0$!-,*+4%,!01,!*),%*.,!p469(<,*&!

<(+0*&6,!2,03,,&!01,!8%,<(60,<!*&<!0%4,!+9(6,!8$+(0($&+!*0!01,!

*&61$%!8$(&0+!$/!01,!+9(6,+!=*&61$%!8$(&0!,%%$%G!Uhp@`!

𝓛

APE

(𝐮, 𝐲) = (1 3

⁄

)

‖

𝐮

{0,1,2}

− 𝐲

{0,1,2}

‖

2,1

=U>@!

(&!31(61!𝐮

{0,1,2}

!*&<!𝐲

{0,1,2}

!<,&$0,!01,!8$+(0($&!),60$%+!*0!

01,!*&61$%!8$(&0+!$/!01,!.%(<+!01*0!<,a&,!01,!)$:,9!9$6*0($&+!

$/!01,!%,+8,60(),!+9(6,+!(&!F?!+8*6,J!g78(6*997G!*&61$%!8$(&0+!

*%,!*++4-,<!0$!2,!*0!01,!6,&0,%G!2$00$-!9,/0!*&<!2$00$-!%(.10!

6$%&,%+!$/!*!.(),&!+9(6,J!U++4-(&.!01*0!+9(6,+!*%,!4&<,%.$(&.!

%(.(<!-$0($&G!01,!Uhp!(+!,D4()*9,&0!0$!01,!php!*),%*.,<!$&!

01,!%(.10'0%(*&.49*%!%,.($&!/$%-,<!27!01,!01%,,!*&61$%!8$(&0+!

$&!01,!%,/,%,&6,!+9(6,J!

A.3 SVRnet Model Implementation

! #$%!%,8%$<46(2(9(07G!3,!8$%0!01,!$%(.(&*9!g,&+$%#9$3!"J"F!

(-89,-,&0*0($&!$/!HIB&,0!N>VP!3(01!*&!C&6,80($&!2*6E2$&,!

0$!h7g$%61!"J"FG!31,%,!3,!4+,!*!B,+\,0'FQ!2*6E2$&,!*&<!*!

8%,<(60($&!1,*<! 6$&+(+0(&.!$/! *!512 × 9!<,&+,!9*7,%G!31(61!

8%,<(60+!01,!+9(6,!8$+(0($&!),60$%+!*0!01%,,!*&61$%!8$(&0+J!^,!

a&<!01*0!>?!2*061!&$%-*9(X*0($&!2*+,<!$&!6$99,60,<!+0*0(+0(6+!

<$,+!&$0!8,%/$%-!3,99!*0!0,+0!0(-,!*&<!$80!0$!&$%-*9(X,!,*61!

,:*-89,!2*+,<!$&!01,!+0*0(+0(6+!$/!,*61!+9(6,!+0*6EJ!^,!4+,!

+42+*-89,<!+9(6,+!$/! +(X,!128 × 128!8(:,9+! *&<! (&0,%8$9*0,!

01,!8%,<(60,<!*&61$%!8$(&0!8$+(0($&!),60$%+!0$!*!9(&,*%!-$0($&!

a,9<!*&<!+420%*60!01,!+9(6,!)$:,9!6$$%<(&*0,+!0$!$40840!+9(6,!

-$0($&J!^,!0%*(&!01(+!(-89,-,&0*0($&!$/!HIB&,0!4+(&.!01,!

%,.49*%!AHp!9$++!$&!$40840!+9(6,!-$0($&!a,9<J!!^,!(&(0(*9(X,!

01,!-$<,9!3(01!01,!0$%61)(+($&!C-*.,\,0"dt)"!3,(.10+J!

A.4 SVoRT Model Conﬁguration

! #$%!6$-8*%(+$&!3(01!HI$Bg!=)>@G!3,!4+,!-$<,9!3,(.10+!

8%$)(<,<!27!j4!,0!*9J!N>YP!$&!01,(%!%,8$+(0$%7G!6$&a.4%,!01,!

-$<,9!0$!4+,!$&,!+9(6,!+0*6E!3(01!*!+9(6,!.*8!$/!FJ>--G!*&<!

$80(-(X,!01,!%,6$&+0%460($&!hH#!=+9(6,!01(6E&,++!$/!"JR--@!

/$%!)*9(<*0($&!*664%*67!=(J,JG!*),%*.,!-$0($&!,&<'8$(&0!,%%$%@!

$&!01,!">!#,gU!)*9(<*0($&!+425,60+!4+(&.!*&!,:8$&,&0(*9!.%(<!

+,*%61J!^,! a:,<! 01,! +0*6E! 8$+(0($&*9! ,&6$<(&.! $/! HI$Bg!0$!

V!=%*01,%!01*&!*!%*&<$-!(&0,.,%@!/$%!%,8%$<46(29,!%,+490+J!^,!

6$&),%0!HI$Bgn+!0%*&+/$%-!$40840!0$!<,&+,!-$0($&!a,9<+!0$!

6$-840,!01,!-$0($&!AHp!*&<!php!/$%!)*9(<*0($&!*664%*67J!

B. Additional SVR Results

! Z,%,G!3,!8%$)(<,!*<<(0($&*9!HIB!%,+490+!$&!$4%!*<490!*&<!

/,0*9!<*0*+,0+J!#(.4%,!]"!)(+4*9(X,+!%,6$&+0%460($&+!$/!*<490!

2%*(&!AB!)$94-,+!/$%!01%,,!$/!$4%!)*9(<*0($&!+425,60+G!3(01!

*99!01%,,!$%01$.$&*9!)(,3+!+1$3&!/$%!6$-89,0,&,++J!#(.4%,!

]>!+(-(9*%97!)(+4*9(X,+!/,0*9!%,6$&+0%460($&+J!^,!(&694<,!01,!

6$%%,+8$&<(&.!HI$Bg)>!%,+490+!/$%!6$-8*%(+$&G!&$0(&.!01*0!

HIB&,0!%,6$&+0%460($&+!*%,!.*%29,<!(&!-*&7!6*+,+!=+,,!a%+0!

%$3!$/!#(.4%,!T@!*&<!*%,!9,++!-,*&(&./49!0$!6$-8*%,!*.*(&+0J!