Non-line-of-sight Surface Reconstruction

Abstract!

We propose a joint albedo–normal approach to non-line-

of-sight (NLOS) surface reconstruction using the directional

light-cone transform (D-LCT). While current NLOS imaging

methods reconstruct either the albedo or surface normals of

the hidden scene, the two quantities provide complementary

information of the scene, so an eﬃcient method to estimate

both simultaneously is desirable. We formulate the recovery

of the two quantities as a vector deconvolution problem, and

solve it using the Cholesky–Wiener decomposition. We show

that surfaces ﬁtted non-parametrically using our recovered

normals are more accurate than those produced with NLOS

surface reconstruction methods recently proposed, and are

1,000× faster to compute than using inverse rendering.

1. Introduction

! #$%&'(%)&$*&+(,-.!/#0123!(45,(%,!6)5'+!7(.-!.-)!859.:;)!

5%6!;)%6);(%,!$*!5!+8)%)!.-5.!(+!%$.!(%!.-)!6(;)8.!'(%)!$*!+(,-.!

*;$4!.-)!+)%+$;<!=%!;)8)%.!>)5;+?!#012!(45,(%,!-5+!)4);,)6!

5+!5%!(49$;.5%.!@(+($%!9;$A')4?!7(.-!599'(85.($%+!(%!;)4$.)!

+)%+(%,?!6)*)%+)?!;$A$.(8!@(+($%!5%6!5:.$%$4$:+!6;(@(%,<!B!

8$44$%!(45,(%,!+).:9!(+!.$!C'$$D!5;$:%6!.-)!8$;%);E!:+(%,!

.-)!8$%*$85'!8$%F,:;5.($%!6)9(8.)6!(%!G(,:;)!"<!H>9(85''>?!5!

'(,-.!+$:;8)?!+:8-!5+!5!'5+);!A)54?!(%6(;)8.'>!('':4(%5.)+!.-)!

+8)%)!A>!;)I)8.(%,!$%.$!5!+:;*58)!.-5.!85%!A)!+))%!*;$4!A$.-!

.-)!+)%+$;!5%6!.-)!+8)%)<!J)!+)%+$;!.-)%!859.:;)+!.-)!+8)%)!

;)I)8.($%+!*;$4!.-)!+54)!+:;*58)!'$85.($%?!5%6!;)8$;6+!.-)4!

5+!5!.(4)&;)+$'@)6!+)K:)%8)!$*!.7$&6(4)%+($%5'!(45,)+!/$;!

.;5%+()%.+3?!*;$4!7-(8-!$%)!85%!8$49:.5.($%5''>!;)8$%+.;:8.!

.-)!+8)%)<!B95;.!*;$4!.;5%+()%.&A5+)6!(45,(%,?!$.-);!#012!

(45,(%,!4$65'(.()+!(%8':6)!.-$+)!A5+)6!$%!+9)8D')!L"MNO?!$;!!

(%8$-);)%.!(%.)%+(.>!4)5+:;)4)%.+!LP?!QO?!5+!7)''!5+!95++(@)!

+)%+(%,!LRMSO!5%6!58$:+.(8!(45,(%,!L"TO!.)8-%(K:)+<!

! U);)?!7)!7(''!8$%+(6);!)V8':+(@)'>!#012!(45,(%,!A5+)6!

$%!.;5%+()%.+!L""M"SO?!*;$4!7-(8-!.-)!-(66)%!#012!+8)%)!(+!

.>9(85''>!;)%6);)6!5+!5!+95.(5'?!.-;))&6(4)%+($%5'!@$':4)!$*!

5'A)6$!/@$':4).;(8!5'A)6$3?!$;!5+!5!+).!$*!$AW)8.!+:;*58)+<!=%!

.-)!@$':4).;(8!5'A)6$!95;56(,4?!.-)!$AW)8.(@)!(+!.$!)+.(45.)!

5'A)6$!@5':)+!*$;!+8)%)!@$V)'+!L""M"RO?!7-(')!(%!.-)!+:;*58)!

;)8$%+.;:8.($%!95;56(,4?!$%)!+))D+!4$;)!6(;)8.'>!.$!;)8$@);!

$AW)8.!+:;*58)+!(%!.-)!.-;))&6(4)%+($%5'!+8)%)!A>!)+.(45.(%,!

.-)(;!+:;*58)!%$;45'+!L"XM"SO<!Y-(')!+:;*58)&A5+)6!4).-$6+!

-5@)!.-)!9$.)%.(5'!.$!;)8$%+.;:8.!$AW)8.!,)$4).;>!7(.-!F%);!

6).5('!.-5%!.-)!5'A)6$!$%)+?!8:;;)%.!599;$58-)+!.$!)+.(45.(%,!

+:;*58)!%$;45'+!5;)!+)%+(.(@)!.$!%$(+)?!'(4(.)6!.$!+8)%)+!7(.-!

+(49');!$AW)8.!,)$4).;>!L"ZO?!+)%+(.(@)!.$!(%(.(5'([5.($%!L"SO!

$;!)%.5('!5!-(,-!8$49:.5.($%5'!8$49')V(.>!L"NO?!5''!$*!7-(8-!

4$.(@5.)!$:;!9;)+)%.!7$;D<!\4A)66)6!(%!.;5%+()%.+!(+!5!4(V!

$*!+:;*58)!%$;45'!5%6!5'A)6$!(%*$;45.($%?!+$!.-5.!)V9'(8(.'>!

588$:%.(%,!*$;!.-)!9;)+)%8)!$*!A$.-!(%!.;5%+()%.+!85%!9;)+)%.!

%)7!$99$;.:%(.()+!*$;!;)8$@);(%,!A$.-!K:5%.(.()+!;$A:+.'><!

Non-line-of-sight Surface Reconstruction

Using the Directional Light-cone Transform

Sean I. Young David B. Lindell

Stanford University Stanford University

sean0@stanford.edu lindell@stanford.edu

Bernd Girod David Taubman Gordon Wetzstein

Stanford University UNSW Sydney Stanford University

bgirod@stanford.edu d.taubman@unsw.edu.au gordon.wetzstein@stanford.edu

! "#$!%&'!#()*+,! ")$!%&'!-,'.#(/! "0$!1233*+!/&'4#0*!

125&'*!67!NLOS surface reconstruction via the D-LCT: 892/32-5!

:;%<!2.#52-5! .*3=,+/!3>?20#((>!'*0,@*'!,-(>! 3=*! #()*+,!,4! 3=*!

=2++*-!/0*-*7!A*!B2'*032,-#(!;CD!'*0,@*'/!),3=!3=*!#()*+,!"#$!#-+!

3=*!/&'4#0*!-,'.#(/!")$!,4!3=*!/0*-*E!#((,F2-5!&/!3,!'*0,-/3'&03!3=*!

=2++*-!,)G*03!/&'4#0*!F23=!H-*'!+*3#2(!"0$7!!



′



′

I2++*-!,)G*03!

C,((,0#3*+!(#/*'!

#-+!+*3*03,'!

!

J2/2)(*!/&'4#0*!

%00(&+*'

! !=%!.-(+!7$;D?!7)!9;$9$+)!5!W$(%.!5'A)6$M%$;45'!599;$58-!

.$!#012!+8)%)!;)8$%+.;:8.($%!A5+)6!$%!.-)!6(;)8.($%5'!'(,-.&

8$%)!.;5%+*$;4!/^&0_H3?!7-(8-!(+!5!@)8.$;(5'!,)%);5'([5.($%!

$*!.-)!/+85'5;3!'(,-.&8$%)!.;5%+*$;4!/0_H3!;)8)%.'>!9;$9$+)6!

A>!1`H$$')!et al.!L"]O<!Y)!*$;4:'5.)!.-)!;)8$@);>!$*!%$;45'+!

5%6!5'A)6$+!5+!5!@)8.$;!6)8$%@$':.($%!9;$A')4?!5%6!-5@(%,!

*$:%6!.-)!.7$!K:5%.(.()+?!7)!F.!5!+:;*58)!$%.$!.-)!;)8$@);)6!

%$;45'+<!a>!+$'@(%,!.-(+!@)8.$;(5'!6)8$%@$':.($%!9;$A')4!(%!

.-)!W$(%.!5'A)6$M%$;45'!+958)?!7)!;)8$@);!.-)!.7$!K:5%.(.()+!

;$A:+.'>!5%6!)b8()%.'>?!5''$7(%,!:+!.$!$A.5(%!5!A)..);!+:;*58)!

;)8$%+.;:8.($%<!=%!95;.(8:'5;?!$:;!599;$58-!(+!45%>!$;6);+!$*!

45,%(.:6)!*5+.);!.-5%!;)8)%.!7$;D!L"NO?!7-(8-!85%!+(4('5;'>!

)+.(45.)!5'A)6$+!5%6!+:;*58)!%$;45'+!(%!5!W$(%.!45%%);<!1:;!

9;$9$+)6!599;$58-!5'+$!7$;D+!7(.-!)V(+.(%,!8$%*$85'!#012!

(45,(%,!-5;675;)<!Y)!('':+.;5.)!$:;!$@);5''!4).-$6!9(9)'(%)!

(%!G(,:;)!]<!H$!+:445;([)?!$:;!45(%!8$%.;(A:.($%+!5;)c!

• Directional LCT:!Y)!)V9;)++!%$%&'(%)&$*&+(,-.!+:;*58)!

%$;45'!;)8$@);>!5+!5!@)8.$;!6)8$%@$':.($%!9;$A')4!$%!

.(4)&;)+$'@)6!4)5+:;)4)%.+?!@(5!.-)!^(;)8.($%5'!0(,-.&

8$%)!H;5%+*$;4!/^&0_H3<!

• Cholesky–Wiener Solver:!Y)! +$'@)! .-)! 5A$@)! @)8.$;&!

6)8$%@$':.($%!9;$A')4!)b8()%.'>!(%!.-)!G$:;();!6$45(%!

.$!;)8$@);!.-)!+:;*58)!%$;45'+<!

• Surface Fitting:!Y)!;)8$%+.;:8.!-(,-'>&588:;5.)!$AW)8.!

+:;*58)!6)+8;(9.($%+!A>!F..(%,!+:;*58)!95;54).);+!$%!.-)!

;)8$@);)6!%$;45'+<!!

2. Related Work

! H;5%+()%.!(45,(%,!75+!F;+.!8$%8)9.:5'([)6!A>!d(;45%(!et

al.!LPO?!7-$!A)'()@)6!.-5.!7)!85%!'$$D!5;$:%6!.-)!8$;%);!A>!

9;$A(%,!5!75''!7(.-!5%!:'.;5*5+.!'5+);!5%6!6).)8.$;<!05.);?!.-)!

(6)5!75+!6)4$%+.;5.)6!(%!9;58.(8)!A>!e)'.)%!et al.!L""O!:+(%,!

5!*)4.$+)8$%6!'5+);!5%6!5!+.;)5D!854);5<!17(%,!.$!.-)!8$+.!

$*!+:8-!(45,(%,!-5;675;)?!;)+)5;8-);+!-5@)!5'+$!:+)6!$.-);!

+)%+(%,!.)8-%$'$,()+!+:8-!5+!.(4)&$*&I(,-.!854);5+!L]T?!]"O!

$;!)@)%!;),:'5;!8$%+:4);!854);5+!LS?!]]O<!U$7)@);?!+(%,')&

9-$.$%!5@5'5%8-)!6($6)+!/2fB^+3!L]NM]XO!-5@)!A))%!+-$7%!

.$!A)!95;.(8:'5;'>!@);+5.(')!*$;!+)%+(%,?!5''$7(%,!:+!.$!(45,)!

:%6);!54A()%.!'(,-.(%,?!5.!*5+.!;5.)+!L]ZO?!$;!5.!'$%,!6(+.5%8)+!

L]SO<!B'.45%%!et al.!LNTO!9;$@(6)!5!8$49;)-)%+(@)!;)@()7!$*!

2fB^+!5+!7)''!5+!.-)(;!599'(85.($%+<!

! U5@(%,!859.:;)6!.-)!.;5%+()%.!65.5?!.-)!#012!+8)%)!85%!

A)!;)8$%+.;:8.)6!5+!5!@$':4)!$*!5'A)6$!L""M"RO?!$;!+:;*58)+!

$*!$AW)8.+!L"XM"SO<!g)%);5''>!+9)5D(%,?!)+.(45.($%!$*!5'A)6$!

85%!A)!9$+)6!5+!5%!(%@);+)&F'.);(%,!9;$A')4!7-);)5+!.-5.!$*!

+:;*58)+!(+!$*.)%!9$+)6!5+!5%!(%@);+)&;)%6);(%,!9;$A')4<!Y)!

%$7!9;$@(6)!5!A;()*!;)@()7!$*!.-)!.7$!(%@);+)!599;$58-)+<!

2.1. Inverse Filtering Approaches

! e) '.) %! et al.!L""O!5;)!.-)!F;+.!.$!9$+)!.-)!;)8$@);>!$*!.-)!

5'A)6$!@$':4)!5+!5%!(%@);+)!9;$A')4<!h)'5.(%,!.-)!.;5%+()%.!

4)5+:;)4)%.+!.$!+$4)!#012!+8)%)!:+(%,!-(,-);&$;6);!'(,-.!

.;5%+9$;.?!.-)>!*$;4:'5.)6!+8)%)!;)8$%+.;:8.($%!5+!5!/'(%)5;3!

')5+.&+K:5;)+!9;$A')4!5%6!+$'@)6!.-)!;)+:'.(%,!6)%+)!+>+.)4!

$*!)K:5.($%+!7(.-!F'.);)6!A58D&9;$W)8.($%!/Gaf3<!Y-(')!Gaf!

9;$6:8)+!9;$4(+(%,!;)+:'.+?!(.!$%'>!599;$V(45.)'>!+$'@)+!.-)!

$;(,(%5'!')5+.&+K:5;)+!9;$A')4?!5%6!.-)!;)8$%+.;:8.)6!+8)%)+!

85%!'58D!F%)!6).5('+<!2(%8)!.-)!8$49:.5.($%5'!8$49')V(.>!$*!

Gaf!(+!+.(''!-(,-!5.!(

3!(%!.-)!%:4A);!

!$*!@$V)'+?!'5.);!

5:.-$;+!+$:,-.!.$!;)F%)!Gaf!A>!(49;$@(%,!.-)!K:5'(.>!$*!.-)!

(.);5.(@)!+$'@);+!L"R?!N"O!$;!.-)(;!+9))6!$%!gfi+!L]RO<!

! =%!.-)!8$%*$85'!85+)?!1`H$$')!et al<!L"]O!%$.)!.-5.!-(,-);&!

$;6);!'(,-.!.;5%+9$;.!85%!A)!)V9;)++)6!5+!5!8$%@$':.($%!7(.-!

5!8-5%,)!$*!@5;(5A')+<!J)(;!$@);5''!.;5%+*$;45.($%?!;)*);;)6!

.$!5+!.-)!'(,-.&8$%)!.;5%+*$;4!/0_H3?!)V9;)++)+!.-)!9;$A')4!

$*!e)'.)%!et al.!L""O!5+!5!.-;))&6(4)%+($%5'!+(,%5'!6)A':;;(%,!

9;$A')4<!=%!8$%.;5+.!.$!.-)!Gaf?!.-)!0_H!+$'@)+!.-)!(%@);+)!

9;$A')4!)V58.'>?!5%6!-5+!5!'$7!8$49:.5.($%5'!8$49')V(.>!$*!

(

log )!(%!.-)!%:4A);!

!$*!@$V)'+?!.-5%D+!.$!.-)!:+)!

$*!.-)!G$:;();!.;5%+*$;4<!h)8)%.'>?!B-%!et al.!LN]O!9;$9$+)6!

5%!599;$V(45.)!8$%@$':.($%5'!(45,(%,!.;5%+*$;4!+(4('5;!.$!

.-)!0_H!*$;!.-)!%$%&8$%*$85'!+)..(%,<!

! =%!8$%.;5+.!7(.-!6)8$%@$':.($%?!7-(8-!7)!85%!:'.(45.)'>!

;)'5.)!.$!.-)!6(j:+($%!)K:5.($%?!0(%6)''!et al.!L"PO!+:,,)+.!.$!

!

! "#$!D'#-/2*-3!@,(&.*!𝜏! ")$!:,'.#(/!𝜐!4',.!𝜏!! "0$!<&'4#0*!4233*+!,-3,!𝜐! "+$!K',&-+L3'&3=!/&'4#0*!

125&'*!M7!Method overview:!N!1m × 1m × 2ns!@,(&.*!𝜏!,4!3'#-/2*-3/!"#$!2/!H(3*'*+!&/2-5!3=*!+2'*032,-#(!(25=3L0,-*!3'#-/4,'.!3,!,)3#2-!3=*!

/&'4#0*!-,'.#(/!")$7!O*!2-3*5'#3*!3=*!/&'4#0*!-,'.#(/!3,!,)3#2-!3=*!H-#(!'*0,-/3'&03*+!/&'4#0*!"0$E!F=20=!2/!/2.2(#'!3,!3=*!5',&-+L3'&3=!"+$7!

!



′



′

+$'@)!.-)!.;5%+()%.!(45,(%,!9;$A')4!A>!4$6)'(%,!-(,-&$;6);!

'(,-.!.;5%+9$;.!5+!75@)!9;$95,5.($%!(%!.-)!.-;))&6(4)%+($%5'!

+958)?!5%6!+$'@)!.-)!;)+:'.(%,!(%@);+)!9;$A')4!)b8()%.'>!(%!

.-)!G$:;();!6$45(%!:+(%,!CME!4(,;5.($%<!J(+!4).-$6!5'+$!

-5+!5!(

log )!8$49:.5.($%5'!8$49')V(.>!6:)!.$!.-)!:+)!

$*!.-)!G$:;();!.;5%+*$;4<!k).-$6+!A5+)6!$%!6(j;58.(@)!75@)!

9;$95,5.($%?!)<,<!9-5+$;!F)'6+!LNNO?!5(4!*:;.-);!.$!$@);8$4)!

.-)!'(4(.5.($%+!(%!.-)!(45,(%,!4$6)'!6:)!.$!5++:49.($%+!+:8-!

5+!+(%,')!+85..);(%,?!5%6!.-)!'58D!$*!$88':+($%+!(%!.-)!-(66)%!

+8)%)<!=*!.-)!8$%*$85'!+).:9!(+!:+)6?!.-)!9-5+$;!F)'6!4).-$6!

85%!A)!(49')4)%.)6!(%!.);4+!$*!.-)!0_H!*$;!5!(

log )!

8$49:.5.($%5'!8$49')V(.><!

! Y-);)5+!A$.-!.-)!0_H!5%6!.-)!M!4(,;5.($%!599;$58-)+!

5;)!)V.;)4)'>!)b8()%.?!.-)>!6$!%$.!(%%5.)'>!-5@)!.-)!5A('(.>!

.$!)+.(45.)!.-)!+:;*58)!%$;45'+!$*!+8)%)!$AW)8.+<!H$!F%6!.-)!

+:;*58)!%$;45'+!5'$%,!7(.-!.-)!5'A)6$!@$':4)?!U)(6)!et al.!

L"NO!9$+)!.-)!.;5%+()%.!(45,(%,!9;$A')4!5+!5%!$9.(4([5.($%!

(%!.-)!5'A)6$!5%6!.-)!+:;*58)!%$;45'!@5;(5A')+<!B'.-$:,-!.-)!

;)9$;.)6!;)+:'.+!5;)!9;$4(+(%,?!$%)!45W$;!'(4(.5.($%!$*!+:8-!

5!4).-$6!(+!.-)!(

)!8$49')V(.>!(%!A$.-!8$49:.5.($%!5%6!

4)4$;>?!5+!7)''!5+!.-)!%$%8$%@)V(.>!$*!.-)!$@);5''!9;$A')4!

*$;4:'5.($%<!J)!^&0_H!-)'9+!:+!.$!+$'@)!5!+(4('5;!5'A)6$M

%$;45'!)+.(45.($%!9;$A')4!7(.-!.-)!+54)!'$7!8$49')V(.>!$*!

.-)!0_H!5%6!M!4(,;5.($%<!g(@)%!.-)!+-5;)6!G$:;();!;$$.+!

58;$++!.-)!0_H?!M!5%6!9-5+$;!F)'6+?!(.!45>!A)!9$++(A')!.$!

599'>!$:;!6(;)8.($%5'!599;$58-!.$!M!4(,;5.($%!5%6!9-5+$;!

F)'6+!5+!7)''?!5'.-$:,-!7)!6$!%$.!5..)49.!.-(+!(%!$:;!7$;D<!

2.2. Inverse Rendering Approaches

! =%!8$%.;5+.!7(.-!.-)!(%@);+)!F'.);(%,!599;$58-)+?!(%@);+)!

;)%6);(%,!/5%5'>+(+&A>&+>%.-)+(+3!4).-$6+?!)<,<!L"SO?!+)5;8-!

*$;!@5':)+!$*!.-)!#012!+:;*58)!95;54).);+!/)<,<!ah^G+!5%6!

+:;*58)!%$;45'+3!.-5.!7$:'6!9;$6:8)!.-)!$A+);@)6!.;5%+()%.+!

(*!.-)!#012!+:;*58)!7);)!.$!A)!;)%6);)6<!2(%8)!5!*:''!+)5;8-!

(%!.-)!+:;*58)!95;54).);!+958)!7$:'6!A)!(%.;58.5A')?!(%@);+)!

;)%6);(%,!(+!.>9(85''>!9);*$;4)6!@(5!6(j);)%.(5A')!;)%6);(%,!

/(<)<?!5%!)%);,>!4(%(4([5.($%!(%!+:;*58)!95;54).);+3<!2:;*58)!

95;54).);+!;)8$@);5A')!(%!.-(+!75>!(%8':6)!+:;*58)!'$85.($%+!

5%6!%$;45'+!LNPO?!5%6!('':4(%5.($%!)j)8.+!+:8-!5+!+85..);(%,!

LNQMNZO!5%6!(%.);;)I)8.($%+!LNS?!PTO<!

2.3. Surface Fitting Methods

! 1AW)8.!+:;*58)!85%!A)!F.!$%!.-)!$A.5(%)6!+:;*58)!%$;45'+!

)(.-);!95;54).;(85''>!$;!%$%&95;54).;(85''><!#$%&95;54).;(8!

F..(%,!(+!8$44$%'>!:+)6!*$;!+.);)$&A5+)6!N^!;)8$%+.;:8.($%!

LP"MPNO!7-);)5+!95;54).;(8!599;$58-)+!5;)!4$;)!+:(.)6!*$;!

4)+-!;)F%)4)%.!LPPMPXO<!2:;*58)!F..(%,!45>!A)!+))%!5+!5%!

(%@);+)!9;$A')4! 7-);)! .-)! ,$5'! (+! .$! (%.);9$'5.)! 5! +4$$.-!

45%(*$'6!7(.-$:.!@($'5.(%,!.-)!,(@)%!%$;45'!8$%6(.($%+<!Y)!

85%!%5.:;5''>!*$;4:'5.)!+:;*58)!F..(%,!5+!6(j:+($%!9;$8)++)+!

$;!)%);,>!4(%(4([5.($%!4).-$6+?!A$.-!$*!7-(8-!+))D!.$!F%6!

.-)!;(,-.!.;56)&$j!A).7))%!;),:'5;(.>!/+4$$.-%)++3!5%6!65.5!

F6)'(.>!/+:;*58)!$;.-$,$%5'(.>!.$!.-)!,(@)%!%$;45'+3<!Y)!:+)!

.-)!)%);,>&4(%(4([5.($%!599;$58-!$*!LPZO!.$!F.!5!+:;*58)!$%!

$:;!;)8$@);)6!%$;45'+<!

3. Mathematical Framework

! B*.);!A;()I>!;)@()7(%,!.-)!@$':4).;(8!5'A)6$!4$6)'!5%6!

6(+8:++(%,!(.+!'(4(.5.($%+?!7)!6)@)'$9!$:;!6(;)8.($%5'!5'A)6$!

4$6)'?!5%6!9;$9$+)!)b8()%.!75>+!*$;!+$'@(%,!.-)!5++$8(5.)6!

(%@);+)!9;$A')4!$*!)+.(45.(%,!.-)!+:;*58)!%$;45'+<!!

3.1. =e Volumet ric Albedo Model

! =%!.;5%+()%.!(45,(%,!599;$58-)+?!5!.(4)&;)+$'@)6!6).)8.$;!

(+!:+)6!.$!4)5+:;)!.-)!(%8(6)%.!I:V!$*!9-$.$%+!5+!5!*:%8.($%!

$*!)4(..)6!'(,-.!(49:'+)+<!\58-!$*!.-)+)!.(4)!4)5+:;)4)%.+!

;)8$;6+!.-)!(49:'+)!;)+9$%+)!$*!.-)!#012!+8)%)!5.!9$+(.($%+!

$%!5!@(+(A')!+:;*58)!.$!9;$6:8)!5!@$':4)!$*!.;5%+()%.+<!

! 0).!:+!6)%$.)!.-)!.-;))&6(4)%+($%5'!+8)%)!8$$;6(%5.)+!A>!

(, , )?!5%6!5++:4)!.-)!@(+(A')!+:;*58)!(+!9$+(.($%)6!5'$%,!

 = 0<!Y)!6)%$.)!A>!(

′

, 

′

,  = 0)!9$+(.($%+!$%!.-(+!@(+(A')!

+:;*58)l!+))!G(,:;)!"<!B!8$44$%!.;5%+()%.!(45,(%,!4$6)'!(+!

.-)!8$%*$85'!@$':4).;(8!5'A)6$!4$6)'!

(

′

, 

′

, ) = ddd

(, , )



Ω

/"3!

2



(

′

−)

+ (

′

−)

+ 

−,!

(%!7-(8-!!6)%$.)+!5!.-;))&6(4)%+($%5'!5'A)6$!@$':4)!7(.-!

F%(.)!+:99$;.!Ω?!5%6!()!;)'5.)+!.-)!;$:%6&.;(9!.(4)!$*!I(,-.!

$*!'(,-.!7(.-!.7(8)!.-)!6(+.5%8)!!A).7))%!.-)!+8)%)!(, , )!

5%6!.-)!+)%+(%,!(

′

, 

′

,  = 0)!'$85.($%+<!U);)?!≈3 ×10

6)%$.)+!.-)!+9))6!$*!'(,-.!7-(')!1 

⁄

= (2 

⁄

)

!4$6)'+!.-)!

;56($4).;(8!*5''&$j!6:)!.$!6(+.5%8)<!J)!+85'(%,!1 

⁄

!85%!A)!

;)4$@)6!*;$4!/"3!(*!7)!9;)+85')!!A>!(2 

⁄

)

!(%!56@5%8)<!=%!

.-)!85+)!$*!;).;$&;)I)8.(@)!+:;*58)+?!5!*5''&$j!*58.$;!$*!1/

(+!4$;)!8$44$%'>!5++:4)6<

! H$!6(+8;).([)!4$6)'!/"3?!7)!+549')!Ω!:+(%,!?!!5%6!!

9$(%.+!$%!.-)!&?!&!5%6!&5V)+?!;)+9)8.(@)'><!B++:4(%,!.-5.!

.-)!.;5%+()%.!!-5+!A))%!9;)&+85')6!A>!(2 

⁄

)

?!7)!85%!7;(.)!

.-)!6(+8;).([)6!4$6)'!8$4958.'>!:+(%,!45.;(V!%$.5.($%!5+!

= ,!

/]3!

(%!7-(8-!, ∈

𝑁

𝑀

!5%6!!(+!5!A(%5;>!45.;(V!7(.-!@5':)+!

$A.5(%)6!A>!+549'(%,!()<!2(%8)!!(+!5!'$7&95++!$9);5.$;!$*!

-(,-!8$%6(.($%!%:4A);?!.-)!.5+D!$*!F%6(%,!!*;$4!,(@)%!!(+!

5%!(''&9$+)6!9;$A')4!LPSO<!h5.-);!.-5%!8$49:.)!.-)!+$':.($%!

6(;)8.'>!5+!

opt

= 

−1

?!7)!+-$:'6!F%6!(.!5+!.-)!+$':.($%!$*!

.-)!;),:'5;([)6!')5+.&+K:5;)+!9;$A')4!

minimize () =

‖

−

‖

+ 

‖



‖

/N3!

(%!7-(8-!!;)9;)+)%.+!.-)!.;56)&$j!A).7))%!65.5!F6)'(.>!5%6!

;),:'5;(.>!/+4$$.-%)++3!$*!.-)!+$':.($%<!

! 1`H$$')!et al.!L"]O!%$.)!.-5.!9;$A')4!/N3!85%!A)!+$'@)6!(%!

5%!)b8()%.!45%%);!7(.-!.-)(;!+$&85'')6!'(,-.&8$%)!.;5%+*$;4!

/0_H3<!=*!7)!6)%$.)!.-)(;!;)+549'(%,!$9);5.$;!A>!?!7)!85%!

)V9;)++!9;$A')4!/N3!)K:(@5')%.'>!(%!.);4+!$*!.-)!;)+549')6!

5'A)6$! = 

∗

!5%6!.;5%+()%.+! = 

∗

!5+!

minimize () =

‖

−

‖

+ 

‖



‖

/P3!

(%!7-(8-!= 

∗

!(+!5!.-;))&6(4)%+($%5'!F'.);!/.-5.!(+?!5!

'(%)5;!+958)&(%@5;(5%.!$9);5.$;3?!7-$+)!(49:'+)!;)+9$%+)!(+!

+-$7%!(%!G(,:;)!N!/53<!2)8.($%!"!$*!.-)!+:99')4)%.!9;$@(6)+!

.-)!6).5('+!$*!.-)!;)+549'(%,!$9);5.$;!

∗

! 2(%8)!!(+!5!.-;))&6(4)%+($%5'!F'.);?!7)!85%!8$49:.)!.-)!

+$':.($%!

opt

= (

∗

+ )

−1



∗

!$*!/P3!)b8()%.'>!(%!.-)!

G$:;();!6$45(%!:+(%,!Y()%);!6)8$%@$':.($%<!J)!;),:'5;(.>!

95;54).);!!85%!A)!(%.);9;).)6!5+!.-)!%$(+)&.$&+(,%5'!;5.($!(%!

.-(+!F'.);(%,!8$%.)V.<!J)!+$':.($%!$*!$:;!$;(,(%5'!9;$A')4!/N3!

(+!$A.5(%)6!A>!;)+549'(%,!.-)!6)8$%@$'@)6!+$':.($%!:+(%,!.-)!

56W$(%.!;)+549'(%,!$9);5.$;?!.-5.!(+?!

opt

= 

opt

3.2. =e Directional Albedo Model

! =%!.-)!85+)!$*!(+$.;$9(8!9$(%.!)4(..);+?!/"3!(+!5%!56)K:5.)!

4$6)'!*$;!-(,-);&$;6);!'(,-.!.;5%+9$;.<!U$7)@);?!*$;!.>9(85'!

6(j:+)!$;!054A);.(5%!$AW)8.!+:;*58)+?!+:8-!5!4$6)'!(,%$;)+!

.-)!;56($4).;(8!*5''&$j!6:)!.$!054A);.`+!8$+(%)!'57?!(<)<?!.-)!

*5''&$j!6:)!.$!.-)!5%,')!A).7))%!.-)!(%8(6)%.!'(,-.!;5>+!5%6!

.-)!+:;*58)!%$;45'+l!+))!LQTO<!=%8$;9$;5.(%,!8$+(%)!.);4+!(%!

/"3!%$.!$%'>!>()'6+!5!4$;)!588:;5.)!*$;75;6!4$6)'?!A:.!4$;)!

(49$;.5%.'>?!(.!)%5A')+!;)8$@);>!$*!+:;*58)!%$;45'+!*;$4!.-)!

.;5%+()%.+!@(5!.-)!(%@);+)!4$6)'<!

! ^)%$.(%,!.-)!.7$!+95.(5'!8$$;6(%5.)+!A>!= (, , )!5%6!



′

= (

′

, 

′

,  = 0)!*$;!A;)@(.>?!7)!:965.)!4$6)'!/"3!5+!!

(

′

, 

′

, ) = d



()()





′

−

‖

′

−‖



Ω

/Q3!







(



′

−

)

(



′

−

)

+ 

−



(%!7-(8-!() = (

𝑥

, 

𝑦

, 

𝑧

)() ∈

!(+!.-)!+:;*58)!%$;45'!

5.!<!1:;!4$6)'!85%!A)!+))%!5+!5!F;+.&$;6);!599;$V(45.($%!$*!

.-)!9->+(85''>!A5+)6!$%)!(%!L"SO?!5''$7(%,!:+!.$!;)*$;4:'5.)!

%$;45'!)+.(45.($%!5+!5!'(%)5;!')5+.&+K:5;)+!9;$A')4<!k$6)'!

/Q3!(+!5'+$!(6)%.(85'!.$!.-)!$%)!(%!L"NO!+5@)!*$;!.-)!5A+)%8)!$*!

$88':+($%!.);4+<!2)8.($%!N!$*!$:;!959);!+:99')4)%.!6);(@)+!

.-)!;)'5.($%+-(9!A).7))%!/Q3!5%6!.-)!9->+(85'!$%)<!B++:4(%,!

*:;.-);!.-5.!.-)!9;$W)8.($%+!



(),



′

−

‖

′

−‖



= cos = 1, ∀ 

′

, ,!

/R3!

4$6)'!/Q3!;)6:8)+!.$!.-)!@$':4).;(8!5'A)6$!4$6)'!/"3<!

! #$.)!(%!/Q3!.-5.!5'A)6$!() ∈!(+!5!+85'5;!K:5%.(.>?!5%6!

+:;*58)!%$;45'!() ∈

!(+!5!:%(.&%$;4!@)8.$;<!h5.-);!.-5%!

;)9;)+)%.!.-)!.7$!K:5%.(.()+!:+(%,!+)95;5.)!@5;(5A')+?!7)!85%!

8$4A(%)!.-)4!(%.$!5!+(%,')!6(;)8.($%5'&5'A)6$!@)8.$;!

() = (

𝑥

, 

𝑦

, 

𝑧

)() = ()() ∈

/X3!

+:8-!.-5.!.-)!6(;)8.($%!5%6!.-)!45,%(.:6)!$*!()!)%8$6)!(.+!

+:;*58)!%$;45'!5%6!5'A)6$?!;)+9)8.(@)'><!2:A+.(.:.(%,!/X3!(%!

4$6)'!/Q3!5%6!:+(%,! =

‖



′

−

‖

?!7)!$A.5(%!.-)!6(;)8.($%5'!

5'A)6$!4$6)'!

(

′

, 

′

, ) = d

〈

(), 

′

−

〉



Ω

/Z3!







(

′

−)

+ (

′

−)

+ 

−



;)'5.(%,!.-)!6(;)8.($%5'!5'A)6$!!.$!.-)!.;5%+()%.+!<!G(,:;)!P!

('':+.;5.)+!5%6!+:445;([)+!.-)!6(;)8.($%5'!5'A)6$!4$6)'<!

! /53!0_H!F'.);!D);%)'! ! /A3!^(;)8.($%5'!0_H!F'.);!D);%)'+!/&?!&!5%6!&6(;)8.($%+3!

125&'*!P7!Constructing the D-LCT ﬁlter kernels: A*!(25=3L0,-*!3'#-/4,'.!?',+&0*/!#!3='**L+2.*-/2,-#(E!/=243L2-@#'2#-3!Q*'-*(!"#$7!A*!BL

;CD!")$!0,-/2/3/!,4!3='**!/=243L2-@#'2#-3!Q*'-*(/!3=#3!'*(#3*!+2'*032,-#(!#()*+,!"#()*+,!R!-,'.#($!3,!3=*!3'#-/2*-3/7!A*!𝑧L+2'*032,-#(!BL;CD!

Q*'-*(!")E!4#'!'25=3$!2/!2+*-320#(!3,!3=*!;CD!Q*'-*(!"#$7!

!

!

!

!

!

!

!

!

!

125&'*!S7!Directional albedo model:!N3!(,0#32,-!𝐬 = (𝑥, 𝑦, 𝑧)!,-!

3=*!,)G*03E!+2'*032,-#(!#()*+,!𝛖(𝐬)!=#/!+2'*032,-!#-+!.#5-23&+*!,4!

3=*!-,'.#(!𝐧(𝐬)!#-+!3=*!#()*+,!𝜌(𝐬)E!'*/?*032@*(>7!C,-3'2)&32,-!,4!

#()*+,!𝜌(𝐬)!3,!3=*!/&'4#0*!#3!𝐬

′

= (𝑥

′

, 𝑦

′

, 𝑧 = 0)!+*0'*#/*/E!2-!3=*!

H'/3!,'+*'E!#/!3=*!0,/2-*!,4!3=*!#-5(*!𝜃!)*3F**-!𝛖(𝐬)!#-+!𝐬

′

− 𝐬7!

J2/2)(*!/&'4#0*





(



)

I2++*-!,)G*03!





(



)



(



)





′

−





′



′



′





(



)

! H$!6(+8;).([)!$:;! 6(;)8.($%5'!5'A)6$!4$6)'?!7)!+549')!Ω!

+(4('5;'>!.$!/]3<!J(+!9;$6:8)+!.-)!+>+.)4!$*!'(%)5;!)K:5.($%+!

(%!$:;!6(;)8.($%5'!5'A)6$+!= 

𝑥

∗

, 

𝑦

∗

, 

𝑧

∗



∗

= !

/S3!

(%!7-(8-!!(+!.-)!45.;(V!*;$4!/]3?!5%6!7)!$A.5(%!.-)!)%.;()+!

$*!= 

𝑥

, 

𝑦

, 

𝑧

!A>!+549'(%,!(

′

−)!$%!Ω<!h)8$@);(%,!

.-)!6(;)8.($%5'!5'A)6$!!,(@)%!.-)!.;5%+()%.+!!(+!(''&9$+)6!(%!

.-5.!(.!;)K:(;)+!:+!.$!F%6!.-)!@5':)+!$*!3

!@5;(5A')+!7(.-!

$%'>!

!)K:5.($%+<!2:8-!5!;5%D&6)F8()%.!9;$A')4!85%!A)!

+$'@)6!A>!*$;4:'5.(%,!/S3!5+!.-)!')5+.&+K:5;)+!9;$A')4!

minimize () =

‖

−

‖

+ 

‖



‖

/"T3!

+(4('5;'>!.$!.-)!;),:'5;([)6!599;$58-!(%!/N3<!Y-(')!7)!8$:'6!

5'.);%5.(@)'>!$9.(4([)!.-)!He&0"!@);+($%!$*!9;$A')4!/"T3!.$!

$A.5(%!5!A)..);!+$':.($%?!7)!*$8:+!$%!$:;!0]!@5;(5%.!*$;!%$7!

.$!+$'@)!/"T3!)b8()%.'>!5+!5!@)8.$;!6)8$%@$':.($%!9;$A')4<!

3.3. Directional Light-cone Transform

! Y-);)5+!$:;!')5+.&+K:5;)+!$9.(4([5.($%!9;$A')4!/"T3!-5+!

.-)!+(49')?!8'$+)6&*$;4!+$':.($%!



opt

= (

∗



∗

+ )

−1



∗



∗

,!

/""3!

.-(+!+$':.($%!(+!.$$!)V9)%+(@)!.$!8$49:.)!%5(@)'>!*$;!.>9(85'!

9;$A')4+! 7(.-!

= 

 ≈10

!@$V)'+<! _$49:.(%,!

opt

6(;)8.'>!7(.-!%:4);(85'!4).-$6+!+:8-!5+!_-$')+D>!5%6!0^0!

6)8$49$+(.($%+!7$:'6!(%8:;!5!(

)!8$+.!7-);)5+!(.);5.(@)!

$%)+!/)<,<!8$%W:,5.)!,;56()%.+3?!5!(

)!8$+.<!J)+)!,)%);5'!

4).-$6+!5;)!.-);)*$;)!:%+:(.)6!.$!9;58.(85'!9;$A')4!+([)+<!

!! H$!+$'@)!9;$A')4!/"T3!)b8()%.'>?!7)!,)%);5'([)!.-)!0_H!

.)8-%(K:)!:+)6!(%!/P3!.$!.-)!@)8.$;(5'!9;$A')4<!Y)!85%!7;(.)!

9;$A')4!/"T3!)K:(@5')%.'>!5+!

minimize (







) =





𝑥

, 

𝑦

, 







−



+ 

‖









‖

/"]3!

(%!7-(8-!















𝑥









𝑦









𝑧









𝛖















𝑥

∗



𝑦

∗



𝑧

∗

















𝐓

𝑑

∗









𝑥



𝑦



𝑧









𝛖

/"N3!

(+!.-)!;)+549')6!@5;(5A')?!5%6! = 

∗

!5+!A)*$;)<!G;$4!.-)!

+$':.($%!





opt

!$*!/"]3?!7)!;)8$@);!.-)!+$':.($%!$*!.-)!$;(,(%5'!

9;$A')4!/"T3!5+!

opt

= 

𝑑







opt

<!2)8.($%!]!$*!.-)!+:99')4)%.!

6);(@)+!.-)!;)+549');!

𝑑

∗

!5%6!(.+!;)'5.($%+-(9!.$!

∗

!(%!/P3<!

! #$.)!(%!/"]3!.-5.!

𝑥

?!

𝑦

!5%6!!5;)!+-(*.&(%@5;(5%.?!+$!.-)>!

85%!A)!8$49$+)6!7(.-!.-)!'(,-.&8$%)!F'.);!!.$!9;$6:8)!.-)!

6(;)8.($%5'!'(,-.&8$%)!F'.);+!

𝑥

?!

𝑦

!5%6!<!J)(;!F'.);!

D);%)'+!5;)!+-$7%!(%!G(,:;)!N!/A3<!2(%8)!5''!.-)!$9);5.$;+!(%!

9;$A')4!/"]3!5;)!F'.);+?!$%)!85%!(%.);9;).!/"]3!5+!5!@)8.$;(5'!

6)8$%@$':.($%!9;$A')4?!7(.-!5!FV)6!%$(+)&.$&+(,%5'!;5.($!!

58;$++!.-)!@)8.$;!*;)K:)%8()+<!1%)!85%!+$'@)!/"]3!)b8()%.'>!

:+(%,!$:;!@)8.$;!)V.)%+($%!$*!Y()%);!6)8$%@$':.($%<!

3.4. Cholesky–Wiener Deconvolution

! ^)%$.(%,!.-)!45.;(8)+!(%!/"]3!5+!

𝑥

= 

𝑥

, 

𝑦

= 

𝑦

5%6!

𝑧

= !*$;!+(49'(8(.>?!7)!7;(.)!.-)!%$;45'!)K:5.($%+!

29-);)+

a:%%>!

2);59(+!

/53!U(66)%!$AW)8.!

/A3!0_H!@$':4)!

/83!^&0_H!%$;45'!@$':4)+!/&?!&!5%6!&8$49$%)%.+3!

/63!^&0_H!+:;*58)!

125&'*!T7!Transient imaging using the D-LCT: !BL;CD!"0$!0#?3&'*/!H-*!+*3#2(/!,4!,)G*03!/&'4#0*/!"#$!-,3!0#?3&'*+!)>!3=*!;CD!")$7!J,(&.*/!

")$U"0$!#'*!'*-+*'*+!&/2-5!.#92.&.!2-3*-/23>!?',G*032,-7!BL;CD!/&'4#0*/!"+$!#'*!H3!+2'*03(>!,-3,!3=*!BL;CD!-,'.#(/!"0$7!1,'!"+$E!F*!&/*+!

Q-,F-!)#0Q5',&-+!.#/Q/!3,!H'/3!'*.,@*!3=*!)#0Q5',&-+!?,2-3/7!N((!=2++*-!,)G*03/!=#@*!+2V&/*!/&'4#0*/7!

5++$8(5.)6!7(.-!.-)!')5+.&+K:5;)+!9;$A')4!/"]3!5+!









𝑥

+  

𝑥

∗



𝑦



𝑥

∗



𝑧



𝑦

∗



𝑥



𝑦

+  

𝑦

∗



𝑧



𝑧

∗



𝑥



𝑧

∗



𝑦



𝑧

+ 

















𝐇

∗

𝐇+𝜆𝐈















𝑥









𝑦









𝑧









𝛖















𝑥

∗







𝑦

∗







𝑧

∗













𝐇

∗

𝛕







/"P3!

.-5.!(+?!5!3 ×3!A'$8D!+>+.)4!$*!)K:5.($%+?!7-);)!)58-!A'$8D!

)')4)%.!(+!5!F'.);!$;!5!F'.);!+(,%5'<!J)!+.;:8.:;)!$*!/"P3!.-:+!

+:,,)+.+!.-5.!7)!+$'@)!.-)!3 ×3!+>+.)4!:+(%,!.-)!_-$')+D>!

6)8$49$+(.($%?!9);*$;4(%,!.-)!.7$!5++$8(5.)6!*$;75;6&!5%6!

A58D&+:A+.(.:.($%+!:+(%,!F'.);(%,!$9);5.($%+<!J)!;(,-.&-5%6!

+(6)!@)8.$;!

∗

!85%!A)!8$49:.)6!(%!.-)!G$:;();!6$45(%?!5+!

)58-!

𝑥

∗

?!

𝑦

∗

!5%6!

𝑧

∗

!(+!5!F'.);<!

! i+(%,!.-)!0^0!@5;(5%.!$*!.-)!_-$')+D>!*58.$;([5.($%?!7)!

*58.$;!.-)!45.;(V!= 

∗

+ !5+!= 

∗

?!7-);)!

=











𝑦𝑥





𝑧𝑥



𝑧𝑦









, =









𝑥𝑥



𝑦𝑦



𝑧𝑧







/"Q3!

5%6!.-)!)')4)%.+!$*!!5%6!!5;)!,(@)%!A>!



𝑗𝑗

= 

𝑗

∗



𝑗

+ −

∑



𝑗𝑘



𝑘𝑘



𝑗𝑘

∗

𝑗−1

𝑘=1

/"R3!



𝑖𝑗

= 

𝑗𝑗

−1





𝑖

∗



𝑗

−

∑



𝑗𝑘



𝑘𝑘



𝑗𝑘

∗

𝑗−1

𝑘=1



:+(%,!.-)!8$%@)%.($%!1 = , 2 = , 3 = !(%!A$.-!+:4+<!Y)!

85%!;)56('>!@);(*>!.-)!6>%54(8!9;$,;544(%,!9;$8)6:;)!/"R3!

A>!599'>(%,!.-)!)'(4(%5.($%!+.)9+!$*!.-)!_-$')+D>!5',$;(.-4!

LQ"O!.$!.-)!A'$8D!)')4)%.+!$*!45.;(V!<!

! G(%5''>?!.-)!.;(5%,:'5;([)6!+>+.)4!

∗

= 

∗

!85%!A)!

+$'@)6!:+(%,!*$;75;6&!5%6!A58D&+:A+.(.:.($%+!

= 

−1



∗

, = 

−∗



−1

,!

/"X3!

A$.-!$*!7-(8-!85%!A)!9);*$;4)6!5+!5!+);()+!$*!F'.);(%,!+.)9+!

(%!.-)!G$:;();!6$45(%<!G$;!)V549')?!.-)!A'$8D&)')4)%.+!$*!!

85%!A)!$A.5(%)6!:+(%,!!

= 

𝑥

∗

?!

= 

𝑦

∗

 −

𝑦𝑥



!5%6!



= 

𝑦

∗

 −

𝑧𝑥



−

𝑧𝑦



<!1A+);@)!-);)!

𝑦𝑥



?!

𝑧𝑥



5%6!

𝑧𝑦



!45>!A)!8$49:.)6!(%!.-)!G$:;();!6$45(%?!5+!)58-!

45.;(V!

𝑦𝑥

?!

𝑧𝑥

!5%6!

𝑧𝑦

!;)9;)+)%.+!5!N^!F'.);<!Y)!8$49:.)!

.-)!)')4)%.+!$*!!85%!A)!8$49:.)6!(%!5!+(4('5;!45%%);<!

! G$;!'5;,)!9;$A')4+!7-);)!+.$;(%,!.-)!G$:;();!8$)b8()%.+!

$*!.-)!A'$8D!)')4)%.+!$*!!5%6!!(+!%$.!*)5+(A')?!7)!85%!:+)!

5%!(.);5.(@)!+$'@);!'(D)!8$%W:,5.)!,;56()%.+?!5%6!8$49:.)!.-)!

*$;75;6!4599(%,!(

∗

+ )!:+(%,?!5,5(%?!F'.);(%,!

$9);5.($%+!(%!.-)!G$:;();!6$45(%<!

3.5. Surface Reconstruction

! U5@(%,!$A.5(%)6!.-)!F)'6!!$*!6(;)8.($%5'!5'A)6$?!7)!:+)!

.-)!4).-$6!$*!LQ]O!.$!F.!5!%$%95;54).;(8!+:;*58)<!G(..(%,!.-)!

+:;*58)!54$:%.+!.$!;)8$@);(%,!5%!(%6(85.$;!*:%8.($%!!$*!.-)!

+8)%)!$AW)8.!+$!.-5.!.-)!,;56()%.!$*!!)K:5'+!<!\V9;)++)6!5+!

5%!$9.(4([5.($%!9;$A')4?!7)!-5@)!

minimize () =

‖



∗

−

∗



‖

+ 

‖



‖

/"Z3!

(%!7-(8-!!6)%$.)+!5!6(+8;).([5.($%!$*!.-)!.-;))&6(4)%+($%5'!

,;56()%.!$9);5.$;<!=%!9;58.(8)?!-$7)@);?!.-)![);$&')@)'&+).!$*!

.-)!+$':.($%!

opt

!$*!/"Z3!45>!6)@(5.)!*;$4!.-)!+:;*58)!$*!.-)!

.;:)!+8)%)!$AW)8.!6:)!.$!.-)!%$(+)!(%!!5%6!.-)!6(+8;).([5.($%!

125&'*!W7!Impact of regularity parameter on depth and surface normals:!A*!BL;CD!?',+&0*/!#00&'#3*!+*?3=!"(*43!?(,3/$!#-+!-,'.#(/!

"'25=3!?(,3/$!,@*'!#!F2+*!'#-5*!,4!@#(&*/!,4!𝜆E!+*.,-/3'#32-5!23/!&/*4&(-*//!4,'!0#/*/!F=*'*!3=*!<:X!2/!-,3!Q-,F-!*9#03(>7!A*!;CD!-,'.#(!

*'','/!#'*!4,'!3=*!-,'.#(!@*03,'/!,)3#2-*+!&/2-5!YTPZ!F23=!3=*!;CD!+*?3=!#/!3=*!2-?&37![,2-3/!,4!.2-2.&.!*'','!.#'Q*+!F23=!+,3/7

epth

error

Normal

error

125&'*!\7!Accuracy of D-LCT and LCT:!A*!;CD!#-+!BL;CD!+*?3=/!"(*43!?(,3/$!=#@*!3=*!X]<8!,4!T7^W!#-+!S7^\0.E

#-+!3=*!]N8!,4!67_W!

#-+!67T^0.E!'*/?*032@*(>7!A*!;CD!#-+!BL;CD!/&'4#0*!-,'.#(/!"'25=3!?(,3/$!=#@*!*-+L?,2-3!X]<8!`7^6!#-+!`7TM0.E!#-+!]N8!,4!`7\6!#-+!

`7P_0.E!'*/?*032@*(>7!A*!;CD!+,*/!-,3!-#32@*(>!?',+&0*!/&'4#0*!-,'.#(/E!/,!F*!,)3#2-!3=*.!&/2-5!YTSZ!F23=!3=*!;CD!+*?3=!#/!3=*!2-?&37

RMSE!

0_H!

^&0_H!

!

MAE!

0_H!

^&0_H!

Depth error

Normal error

RMSE!

0_H!

^&0_H!

MAE!

0_H!

^&0_H!

0_H!

^&0_H!

0_H!

^&0_H!

$*!.-)!9;$A')4<!H$!4(.(,5.)!+:8-!(++:)+?!7)!*$''$7!5,5(%!.-)!

599;$58-!$*!LQ]O!.$!)V.;58.!5%!(+$+:;*58)!$*!

opt

!(%+.)56<!

4. Experimental Results

! 2(%8)!$:;!6(;)8.($%5'!0_H!599;$58-!;)8$@);+!A$.-!5'A)6$!

5%6!+:;*58)!%$;45'+?!(.!85%!A)!:+)6!(%!.-)!.;56(.($%5'!8$%.)V.!

$*!.7$&6(4)%+($%5'!#012!(45,(%,!5+!7)''!5+!.$!;)8$%+.;:8.!

+:;*58)+!(%!.-;))!6(4)%+($%+<!Y)!6)4$%+.;5.)!$:;!599;$58-!

*$;!A$.-!:+)&85+)+?!5'+$!8$495;(%,!(.!.$!4).-$6+!+9)8(5'([)6!

.$!)58-!$%)<!Y)!:+)!m#012!LQNO!5%6!2.5%*$;6!L"PO!65.5+).+!

*$;!)V9);(4)%.5'!@5'(65.($%<!J)!m#012!65.5+).!8$%+(+.+!$*!

4:'.(9')&A$:%8)!.;5%+()%.+!$*!+>%.-).(8!$AW)8.+!0.50m!575>!

*;$4!5!1m ×1m!@(+(A')!+:;*58)<!J)!65.5+).!-5+!5!.)49$;5'!

;)+$':.($%!$*!512!9(V)'+!7(.-!A(%+!$*!7(6.-!10ps?!5%6!+95.(5'!

;)+$':.($%+!$*!256 ×256!9(V)'+<!J)!2.5%*$;6!+).!8$%+(+.+!$*!

.;5%+()%.+!4)5+:;)6!$%!5!2m ×2m!+:;*58)!$*!%5.:;5'!-(66)%!

$AW)8.+!1m!575>?!7(.-!54A()%.!'(,-.!5%6!%$(+)<!J(+!65.5+).!

-5+!5!+95.(5'!;)+$':.($%!$*!512 ×512!$;!64 ×64!9(V)'+?!5%6!

5!.)49$;5'!;)+$':.($%!$*!512!7(.-!A(%+!$*!7(6.-!32ps<!

4.1. NLOS Imaging Experiments

Directional Transient Imaging. G(,:;)!Q!+-$7+!.-)!%$;45'!

(45,)+!$A.5(%)6!:+(%,!.-)!^&0_H<!J)+)!(45,)+!8$%.5(%!.-)!

F%)!@5;(5.($%+!(%!$AW)8.!+:;*58)+!+:8-!.-)!+4$$.-!+:;*58)!$*!

.-)!+9-);)+!5%6!.-)!*:;!$*!.-)!A:%%><!J)+)!6).5('+!7$:'6!A)!

6(b8:'.!.$!;)8$@);!9$+.&-$8!*;$4!5'A)6$&$%'>!(45,)+!:+(%,!

6).5('&)%-5%8)4)%.!.)8-%(K:)+?!*$;!)V549')<!J)!;)+:'.+!*$;!

.-)!0_H!;)+)4A')!.-)!&8$49$%)%.!$*!.-)!^&0_H!$%)+!7(.-!

+:A.')!6(j);)%8)+!.-5.!85%!A)!)V9)8.)6!*;$4!.-)!+(49'(*>(%,!

5++:49.($%!/R3<!Y)!;)%6);!6(;)8.($%5'!5'A)6$!@$':4)+!:+(%,!

45V(4:4!(%.)%+(.>!9;$W)8.($%c!*$;!)58-!9$(%.!(

′

, 

′

,  = 0)!

$%!.-)!(45,)!9'5%)?!7)!F%6!5'$%,!.-)!&5V(+!.-)!6(;)8.($%5'&

5'A)6$!7(.-!45V(4:4!&8$49$%)%.!@5':)+<!Y)! ;)8$%+.;:8.!

.-)!+:;*58)+!A>!F;+.!45+D(%,!$:.!.-)!A58D,;$:%6!9(V)'+!7(.-!

,;$:%6&.;:.-!45+D+?!5%6!9);*$;4(%,!f$(++$%!;)8$%+.;:8.($%!

$%!.-)!*$;),;$:%6!9$(%.+!5%6!.-)!6(;)8.($%5'!5'A)6$<!Y)!:+)!

= 2

!*$;!5''!+8)%)+<!!

Accuracy of Depth and Surface Normals.!G(,:;)!R!+-$7+!

.-)!);;$;!459+!*$;!.-)!;)8$@);)6!6)9.-!5%6!+:;*58)!%$;45'+!

$*!.-)!Ca:%%>E<!J)!;)8$@);)6!0_H!5%6!^&0_H!6)9.-!459+!

-5@)!;$$.&4)5%&+K:5;)6!);;$;+!/hk2\3!Q<SX84!5%6!P<SR84!

!!!!!!!!!!!

!!!!!!!!!!!!!^(+8$A$':+

!!!!!!!!!!!!^;5,$%

/53!G);45.!*'$7!

/A3!H+5(!et al.!L"SO!

/83!^&0_H!%$;45'!@$':4)+!/&?!&!5%6! &8$49$%)%.+3!

/63!^&0_H!+:;*58)!

125&'*!_7!Surface reconstruction using captured data:!<a!=#/!#!/?#32#(!'*/,(&32,-!,4!64 × 64!?29*(/!"6.2-!*9?,/&'*$E!#-+!3=*!'*.#2-2-5!

/0*-*/E!512 × 512!"6_`.2-!*9?,/&'*$7!O*!&/*!𝜆 = 2

E!2

!#-+!2

!4,'!<aE!B2/0,),(&/!#-+!B'#5,-E!'*/?*032@*(>7!b-/*3/!2-!3=*!(*43L.,/3!0,(&.-!

/=,F!3=*!/0*-*!,)G*03/7!1*'.#3!c,F!Y6_Z!#-+!3=*!.*3=,+!,4!D/#2!et al.!Y6^Z!,-(>!?#'32#((>!'*0,-/3'&03!3=*!/&'4#0*/7!

]*3=,+/!

64 × 64

128 × 128!

256 × 256!

512 × 512!

N()*+,

1d[!

`7\/!

M7`/!

_7P/!

PM7S/!

[=#/,'!12*(+/!

`7^/!

P76/!

6M7\/!

S^7P/!

𝑓

𝑘

!.25'#32,-!

67T/!

\7\/!

M`7^/!

WM7\/!

;CD!

`7T/!

M7S/!

_7T/!

P`7S/!

:,'.#(/

1*'.#3!c,F!

67\/!

T7\/!

M67P/!

_\7T/!

I*2+*!et al.!

e6`=!

! :fN!

D/#2!et al.!

W=!

! :fN!

BL;CD!"%&'/$!

T7M/!

M67^/!

_^7T/!

PW`7`/!

D#)(*!67!Running times of various methods:!]*#/&'*+!&/2-5!#-!

_L0,'*E!M7W`KIg!C[a!4,'!3=*!d&-->E!#((!#3!3*.?,'#(!'*/,(&32,-/!,4!

512!?29*(/7!A*!.*3=,+/!,4!I*2+*!et al.!Y6PZ!#-+!D/#2!et al.!Y6^Z!+,!

-,3!/0#(*!3,!'*/,(&32,-/!=25=*'!3=#-!64 × 647!!

5%6!.-)!4)5%!5A+$':.)!);;$;+!/kB\3!$*!"<ZX84!5%6!"<QS84!

;)+9)8.(@)'>?!58;$++!*$;),;$:%6!9(V)'+<!J)!+:;*58)!%$;45'+!

)+.(45.)6!:+(%,!.-)!0_H!5%6!.-)!^&0_H!-5@)!.-)!)%6&9$(%.!

hk2\!$*!T<S"84!5%6!T<Q]84?!kB\!$*!T<R"84!5%6!T<NZ84!

;)+9)8.(@)'><!#$.)?!.-)!0_H!6$)+!%$.?!A>!(.+)'*?!9;$6:8)!5%>!

+:;*58)!%$;45'+?!+$!7)!$A.5(%!.-)!%$;45'+!:+(%,!.-)!4).-$6!

LQPO!7(.-!.-)!0_H!6)9.-!5+!.-)!(%9:.!/7)!:+)!R!%)(,-A$;(%,!

9$(%.+!.$!9;$6:8)!.-)!$9.(4:4!;)+:'.+3<!=%!G(,:;)!X?!7)!9'$.!

.-)!(%I:)%8)!$*!;),:'5;([5.($%!95;54).);!!$%!.-)!6)9.-!5%6!

+:;*58)!%$;45'!);;$;+?!('':+.;5.(%,!.-5.!.-)!^&0_H!9);*$;4+!

+.5A')!$@);!5!7(6)!;5%,)!$*!.!J(+!85%!A)!:+)*:'!(%!9;58.(85'!

(45,(%,!+8)%5;($+!(%!7-(8-!.-)!+(,%5'&.$&%$(+)!;5.($!!$*!.-)!

859.:;)6!.;5%+()%.!65.5!(+!%$.!D%$7%!)V58.'><!

Surface Reconstruction with Captured Data.!H$!+-$7!.-)!

;$A:+.%)++!$*!.-)!^&0_H!5,5(%+.!6(j);)%.!.>9)+!$*!%$(+)!.-5.!

5;)!9;)+)%.!(%!;)5'!859.:;)!)%@(;$%4)%.+?!7)!9);*$;4!+:;*58)!

;)8$%+.;:8.($%!7(.-!.-)!2.5%*$;6!65.5+).<!G(,:;)!Z!+-$7+!.-)!

6(;)8.($%5'!5'A)6$!5%6!+:;*58)+!$*!;)8$@);)6!2i?!^(+8$A$':+!

5%6!.-)!^;5,$%!$AW)8.+<!Y)!;)8$%+.;:8.!.-)!+:;*58)+!A>!F;+.!

.-;)+-$'6(%,!.-)!%$;4!$*!6(;)8.($%5'!5'A)6$!@)8.$;+!.$!45+D!

$:.!.-)!A58D,;$:%6!9$(%.+?!.-)%!9);*$;4(%,!f$(++$%!+:;*58)!

;)8$%+.;:8.($%+!$%!.-)!;)45(%(%,!*$;),;$:%6!9$(%.+<!Y)!:+)!

= 2

?!2

!5%6!2

!*$;!.-)!.-;))!+8)%)+<!=%!.-)!2i!+8)%)?!.-)!

%$;45'!@$':4)+!;)@)5'!.-)!$;()%.5.($%!$*!.-)!')..);+!2!5%6!i!

/2!9$(%.+!.$!.-)!:99);&')*.?!i!9$(%.+!.$!.-)!:99);&;(,-.3<!J)!

')*.!95;.!$*!i!(+!95;.(5''>!$88':6)6?!+$!.-)!6(j);)%.!4).-$6+!

9;$6:8)!6(j);)%.!45V(45'!(%.)%+(.>!9;$W)8.($%+!5'$%,!.-)!&

5V(+<!G$;!.-)+)!;)8$%+.;:8.($%!.5+D+!7(.-!%$(+>!.;5%+()%.+?!.-)!

4).-$6!$*!H+5(!et al.!L"SO!5%6!G);45.!I$7!L"ZO!;)8$%+.;:8.!

$%'>!.-)!;$:,-!+-59)+!$*!.-)!$AW)8.+<!Y)!(%(.(5'([)!L"SO!:+(%,!

.-)!0_H?!A:.!$.-);!(%(.(5'([5.($%+!5;)!5'+$!9$++(A')<!Y)!:+)6!

.-)!--density!I5,!(%!.-)!f$(++$%!;)8$%+.;:8.($%!+$*.75;)!

LPZO!.$!5@$(6!.-)!*:+($%!$*!%)5;A>!+:;*58)!+),4)%.+<!

Computational Eﬃciency.!Y-(')!.-)!^&0_H!-5+!.-)!+54)!

8$49:.5.($%5'!8$49')V(.>!5+!.-)!0_H?!7)!9);*$;4!9×!4$;)!

8$49:.5.($%+!9);!@$V)'!6:)!.$!.-)!$:.);!_-$')+D>!*58.$;(%,!

;)K:(;)6<!J)!^&0_H!(+!1000×!*5+.);!8$495;)6!7(.-!+(4('5;!

4).-$6+!.-5.!5;)!8595A')!$*!;)8$@);(%,!.-)!+:;*58)!%$;45'+!

$*!$AW)8.+!7(.-!5!8$49')V!,)$4).;><!Y-(')!G);45.!I$7!L"ZO!

(+!4×!*5+.);!.-5%!$:;!599;$58-?!(.!(+!599'(85A')!4$+.'>!.$!.-)!

;)8$%+.;:8.($%!$*!+:;*58)+!$*!$AW)8.+!7(.-!+(49');!,)$4).;>!

+:8-!5+!5!A$7'!$;!5!+9-);)!/+))!2)8.($%!N!$*!.-)!+:99')4)%.!

*$;!.-)!;)8$%+.;:8.($%+3<!H5A')!"!9;$@(6)+!.-)!;:%%(%,!.(4)+!

$*!6(j);)%.!4).-$6+!$%!5%!Z&8$;)?!]<XTgU[!_fi<!

5. Discussion

! 1:;!7$;D!9;$9$+)+!5%!)b8()%.!4).-$6!.$!W$(%.'>!)+.(45.)!

.-)!5'A)6$!5%6!.-)!+:;*58)!%$;45'+!$*!#012!$AW)8.+!:+(%,!5!

6)8$%@$':.($%!599;$58-<!1:;!^(;)8.($%5'!0_H!-5+!.-)!+54)!

'$7!8$49:.5.($%5'!8$49')V(.>!5+!5'A)6$&$%'>!4).-$6+?!)<,<!

M!4(,;5.($%!5%6!.-)!0_H?!A:.!(+!8595A')!$*!;)8$%+.;:8.(%,!

-(,-&K:5'(.>!+:;*58)+<!!

Limitations.!1:;!*$;75;6!4$6)'!/Z3!5++:4)+!.-)!+8)%)!-5+!

4$+.'>!%$%&+9)8:'5;!+:;*58)+<!G$;.:%5.)'>?!$:;!')5+.&+K:5;)+!

(%@);+)!4).-$6!9;$@(6)+!+$4)!6),;))!$*!;$A:+.%)++!5,5(%+.!

+9)8:'5;(.()+!A>!.;)5.(%,!.-)4!5+!$:.'();+!/+))!)<,<!.-)!^;5,$%!

;)8$%+.;:8.($%?!G(,:;)!Z3<! 2(4('5;'>?!7)!.;)5.! $88':+($%+!(%!

.-)!+8)%)!5+!$:.'();+!.$!$:;!')5+.&+K:5;)+!*$;4:'5.($%<!i+(%,!

5%!

&A5+)6!65.5!F6)'(.>!.);4!(%+.)56!$*!$:;!

&A5+)6!$%)!

/"T3!8$:'6!*:;.-);!(49;$@)!.-)!;$A:+.%)++!$*!$:;!4).-$6?!5.!

.-)!8$+.!$*!(%8;)5+)6!8$49:.5.($%!.(4)+<!J)!

&A5+)6!65.5&

.);4!5'+$!)%*$;8)+!+95;+(.>?!7-(8-!45>!;)4$@)!.-)!%))6!*$;!

45+D(%,!$:.!A58D,;$:%6!9(V)'+<!

! 1:;!*$;75;6!4$6)'!5'+$!'(%)5;([)+!.-)!8$+(%)!*5''&$j!6:)!

.$!.-)!(%.);58.($%!A).7))%!+:;*58)!%$;45'+!5%6!.-)!.7$!'(,-.!

;5>+!/(%8(6)%.!5%6!;)I)8.)63<!1:;!'(%)5;([)6!*5''&$j!4$6)'!(+!

5%!:%6);&)+.(45.$;!$*!.-)!.;:)!*5''&$j?!5%6!+:;*58)!'$85.($%+!

.-5.!45D)!'5;,);!5%,')+!$%!5@);5,)!7(.-!.-)!@(+(A')!75''!5;)!

)+.(45.)6!.$!A)!5.!9$+(.($%+!8'$+);!.$!.-)!@(+(A')!75''?!7-);)!

.-)!*5''&$j!(+!(%6))6!')++<!J(+!85:+)+!;$:%6);!+:;*58)+!.$!A)!

)+.(45.)6!+'(,-.'>!I5..);!.-5%!.-)>!+-$:'6!A)!/+))!.-)!5;4+!$*!

.-)!^(+8$A$':+?!G(,:;)!Z3?!A:.!%$.!5+!I5.!5+!.-)!)+.(45.)+!$*!

.-)!0_H?!7-(8-!5++:4)+![);$!8$+(%)!*5''&$j<!J(+!(++:)!85%!

A)!$@);8$4)!A>!(.);5.(@)'>!;)7)(,-.(%,!.-)!F;+.!.);4!$*!/"T3!

:+(%,!.-)!;5.($!$*!.-)!.;:)!*5''&$j!.$!.-)!'(%)5;!$%)?!A5+)6!$%!!

.-)!%$;45'+!'5+.!)+.(45.)6l!+))!2)8.($%!N!$*!.-)!+:99')4)%.<!

Future Work.!H$!(49;$@)!;)8$%+.;:8.($%!.(4)+?!7)!9'5%!.$!

(49')4)%.!.-)!^&0_H!9;$8)6:;)!$%!5!gfi<!2(4('5;'>!.$!.-)!

$;(,(%5'!0_H?!.-)!^&0_H!(+!-(,-'>!95;5'')'([5A')!5%6!85%!A)!

+(,%(F85%.'>!588)');5.)6!:+(%,!5!gfi!(49')4)%.5.($%<!0(D)!

.-)!gfi!(49')4)%.5.($%!$*!.-)!0_H?!7)!)V9)8.!gfi&A5+)6!

^&0_H!.$!;)K:(;)!4(''(+)8$%6+!$*!9;$8)++(%,!.(4)!*$;!'$7);!

+95.(5'!;)+$':.($%+?!)<,<!32 ×32!$;!64 ×64!9(V)'+<!Y)! 9'5%!

5'+$!.$!8$%+(6);!

!$;!TV!;),:'5;([);+!.$!A)..);!9;)+);@)!.-)!

6(+8$%.(%:(.()+!(%!.-)!;)8$%+.;:8.)6!+:;*58)+<!

6. Conclusion

! #012!(45,(%,!599;$58-)+!-5@)!.>9(85''>!A))%!8'5++(F)6!

5+!;)8$@);(%,!)(.-);!.-)!5'A)6$!$;!.-)!+:;*58)!%$;45'+!$*!.-)!

-(66)%!$AW)8.+<!=%!.-(+!7$;D?!7)!+-$7)6!.-5.!(.!(+!9$++(A')!.$!

;)8$@);!A$.-!K:5%.(.()+!W$(%.'><!=%!8'$+(%,?!;)8$%+.;:8.($%!$*!

+:;*58)+!$*!-(66)%!N^!$AW)8.+! 85%! A)! ;),5;6)6!5+!.-)!%)V.!

*;$%.();!*$;!#012!(45,(%,!A)85:+)!(.!5''$7+!:+!.$!;)9;)+)%.!

A)..);!.-)!N^!)%@(;$%4)%.!7)!:'.(45.)'>!'(@)!(%<!Y)!A)'()@)!

.-)!^&0_H!(+!5!A(,!+.)9!.$75;6+!9:+-(%,!A)>$%6!@$':4).;(8!

5'A)6$!599;$58-)+?!9;$@(6(%,!5!9;58.(85'!75>!.$!)+.(45.)!.-)!

-(66)%!+:;*58)!%$;45'+!%))6)6!*$;!+:;*58)!;)8$%+.;:8.($%<!

Acknowledgements.!Y)! .-5%D! k<! n<! g5'(%6$! *$;! -)'9!7(.-!

65.5+).!LQNO!5%6!=<!gD($:')D5+!*$;!.-)!8$6)!$*!L"Z?!"SO<!^<0<!

75+!+:99$;.)6!A>!5!2.5%*$;6!g;56:5.)!G)''$7+-(9<!g<Y<!75+!

+:99$;.)6! A>! 5%! #2G! _Bh\\h! B75;6! /==2! "QQNNNN3?! 5!

2'$5%!G)''$7+-(9?!A>!.-)!dBi2H!1b8)!$*!29$%+$;)6!h)&

+)5;8-!.-;$:,-!.-)!e(+:5'!_$49:.(%,!_)%.);!__G!,;5%.?!.-)!

^BhfB!h\e\B0!9;$,;54?!5%6!5!f\_B2\!A>!.-)!Bh0<

! !

References

! Y6Z! h#0,?,!d*'3,(,332E!8()*'3!K7!@#-![&33*-E!C='2/32#-!d(&.E!N+!

;#5*-+2GQE!O2((*.!;7!J,/E!#-+!N((#'+![7!],/Q7!:,-L2-@#/2@*!

2.#52-5! 3=',&5=! ,?#i&*! /0#33*'2-5! (#>*'/7! NatureE!

S^6"WSMP$jMPMUMPSE!M`6M7!

! YMZ! %'2!k#3gE!8'#-!<.#((E!#-+!l#',-!<2()*')*'57!;,,Q2-5!#',&-+!

0,'-*'/! #-+! 3=',&5=! 3=2-! 3&')2+! (#>*'/! 2-! '*#(! 32.*! F23=!

/0#33*'*+! 2-0,=*'*-3! (25=37! Nat. PhotonicsE! \"_$jTS^UTTPE!

M`6M7!

! YPZ! d'#-+,-! ]7! <.23=E! ]#33=*F! %mD,,(*E! #-+! ],=23! K&?3#7!

D'#0Q2-5! .&(32?(*! ,)G*03/! ,&3/2+*! 3=*! (2-*! ,4! /25=3! &/2-5!

/?*0Q(*!2.#52-57!b-!CVPRE!M`6_7!

! YSZ! N=.*+!k2'.#-2E!D>(*'!I&30=2/,-E!h#.*/!B#@2/E!#-+!X#.*/=!

X#/Q#'7!;,,Q2-5!#',&-+!3=*!0,'-*'!&/2-5!3'#-/2*-3!2.#52-57!

b-!ICCVE!M``^7!

! YTZ! C='2/3,/! A'#.?,&(2+2/! et al.! 89?(,232-5! ,00(&/2,-! 2-! -,-L

(2-*L,4L/25=3!#032@*!2.#52-57!IEEE Trans. Comput. ImagingE!

S"P$jS6^USP6E!M`6_7!

! Y\Z! k#3=*'2-*! ;7! d,&.#-! et al.! D&'-2-5! 0,'-*'/! 2-3,! 0#.*'#/j!

['2-02?(*/!#-+!.*3=,+/7!b-!ICCVE!M`6W7!

! YWZ! ]#-*(!d#'#+#+!et al.!b-4*''2-5!(25=3!H*(+/!4',.!/=#+,F/7!b-!

CVPRE!M`6_7!

! Y_Z! ]#=*+!d#3#'/*=E!<*'5*>!<&Q=,@E!n=*#-!<=*-E!I*#3=!K*.#'E!

X,9#-#! X*g@#-2E! #-+! N'2/32+*! B,5#'2&7! [#//2@*! /*-/2-5!

#',&-+! 3=*! 0,'-*'! &/2-5! /?#32#(! 0,=*'*-0*7! Nat. Commun.E!

^"6$j6U\E!M`6_7!

! Y^Z! C=#'(*/!<#&-+*'/E!h,=-!]&''#>Ld'&0*E!#-+!J2@*Q!k7!K,>#(7!

C,.?&3#32,-#(! ?*'2/0,?>! F23=! #-! ,'+2-#'>! +2523#(! 0#.*'#7!

NatureE!T\T"WWS`$jSWMUSWTE!M`6^7!

!Y6`Z! B#@2+! d7! ;2-+*((E! K,'+,-! O*3g/3*2-E! #-+! J(#+(*-! k,(3&-7!

N0,&/320!-,-L(2-*L,4L/25=3!2.#52-57!b-!CVPRE!M`6^7!

!Y66Z! N-+'*#/! J*(3*-E! A,.#/!O2((F#0=*'E! %3Q'2/3! K&?3#E!N/=,Q!

J* *' #' # 5= #@ # -E ! ] ,&- 52 ! K 7! d # F* - +2 E! # - +! X # .* / =! X # /Q# '7 !

X*0,@*'2-5! 3='**L+2.*-/2,-#(! /=#?*! #',&-+! #! 0,'-*'! &/2-5!

&(3'#4#/3!32.*L,4Lc25=3!2.#52-57!Nat. Commun.E!PjWSTE!M`6M7!

!Y6MZ! ]#33=*F!%mD,,(*E!B#@2+!d7!;2-+*((E!#-+!K,'+,-!O*3g/3*2-7!

C,-4,0#(! -,-L(2-*L,4L/25=3!2.#52-5!)#/*+!,-!3=*! (25=3L0,-*!

3'#-/4,'.7!NatureE!TTT"W\^\$jPP_UPS6E!M`6_7!

!Y6PZ! 1*(29!I*2+*E!]#33=*F!%mD,,(*E!k#2!n#-5E!B#@2+!d7!;2-+*((E!

<3*@*-! B2#.,-+E! #-+! K,'+,-! O*3g/3*2-7! :,-L(2-*L,4L/25=3!

2.#52-5! F23=! ?#'32#(! ,00(&+*'/! #-+! /&'4#0*! -,'.#(/7! ACM

Trans GraphE!P_"P$jMMj6UMMj6`E!M`6^7!

!Y6SZ! B#@2+!d7!;2-+*((E!K,'+,-!O*3g/3*2-E!#-+!]#33=*F!%mD,,(*7!

O#@*L)#/*+! -,-L(2-*L,4L/25=3! 2.#52-5! &/2-5! 4#/3! 4LQ!

.25'#32,-7!ACM Trans. Graph. TOGE!P_"S$j66\E!M`6^7!

!Y6TZ! C='2/3,?=*'! N7! ]*3g(*'E! B#@2+! d7! ;2-+*((E! #-+! K,'+,-!

O*3g/3*2-7! k*>=,(*! 2.#52-5j! -,-L(2-*L,4L/25=3!2.#52-5!#-+!

3'#0Q2-5!,4!.,@2-5!,)G*03/!#(,-5!#!/2-5(*!,?320#(!?#3=!#3!(,-5!

/3#-+,V!+2/3#-0*/7!ArXiv191206727 Cs EessE!M`6^7!

!Y6\Z! ]#'0,! ;#! ]#--#E! 12,-#! k2-*E! 8'20! d'*23)#0=E! h,-#3=#-!

h#0Q/,-E! D#(=#! <&(3#-E! #-+! N-+'*#/! J*(3*-7! 8'','!

)#0Q?',G*032,-! #(5,'23=./! 4,'! -,-L(2-*L,4L/25=3! 2.#52-57!

IEEE Trans. Pattern Anal. Mach. Intell.E! S6"W$j6\6TU6\M\E!

M`6^7!

!Y6WZ! C=2#Ll2-! D/#2E! k2'2#Q,/! :7! k&3&(#Q,/E! <'2-2@#/#! K7!

:#'#/2.=#-E! #-+! N/=F2-! C7! <#-Q#'#-#'#>#-#-7! A*!

5*,.*3'>! ,4! H'/3L'*3&'-2-5! ?=,3,-/! 4,'! -,-L(2-*L,4L/25=3!

2.#52-57!b-!CVPRE!M`6W7!

!Y6_Z! <=&.2#-!o2-E!<,32'2/!:,&/2#/E!k2'2#Q,/!:7!k&3&(#Q,/E!N/F2-!

C7!<#-Q#'#-#'#>#-#-E!<'2-2@#/#!K7!:#'#/2.=#-E!#-+!b,#--2/!

KQ2,&(*Q#/7!N! 3=*,'>! ,4! 1*'.#3! ?#3=/! 4,'! -,-L(2-*L,4L/25=3!

/=#?*!'*0,-/3'&032,-7!b-!CVPRE!M`6^7!

!Y6^Z! C=2#Ll2-! D/#2E! N/F2-! C7! <#-Q#'#-#'#>#-#-E! #-+! b,#--2/!

KQ2,&(*Q#/7! d*>,-+! @,(&.*3'20! #()*+,p#! /&'4#0*!

,?32.2g#32,-! 4'#.*F,'Q! 4,'! -,-L(2-*L,4L/25=3! 2.#52-57! b-!

CVPRE!M`6^7!

!YM`Z! 1*(29!I*2+*E!;*2!o2#,E!O,(45#-5!I*2+'20=E!#-+!]#33=2#/!d7!

I&((2-7! B2V&/*! .2'','/j! PB! '*0,-/3'&032,-! 4',.! +2V&/*!

2-+2'*03! 2((&.2-#32,-! &/2-5! 2-*9?*-/2@*! 32.*L,4Lc25=3!

/*-/,'/7!b-!CVPRE!M`6S7!

!YM6Z! N0=&3#! k#+#.)2E! I#-5! n=#,E! d,92-! <=2E! #-+! X#.*/=!

X#/Q#'7!%00(&+*+!2.#52-5!F23=!32.*L,4Lc25=3!/*-/,'/7!ACM

Trans GraphE!PT"M$j6Tj6U6Tj6ME!M`6\7!

!YMMZ! h,-#3=#-! k(*2-E! C='2/3,?=! [*3*'/E! h#2.*! ]#'3q-E! ]#'32-!

;#&'*-g2/E!#-+!]#33=2#/!d7!I&((2-7!D'#0Q2-5!,)G*03/!,&3/2+*!

3=*! (2-*! ,4! /25=3! &/2-5! MB! 2-3*-/23>! 2.#5*/7! Sci. Rep.E!

\jPMS^6E!M`6\7!

!YMPZ! ]#&',! d&33#4#@#E! h*//20#! n*.#-E! N()*'3,! D,/2E! k*@2-!

8(20*2'2E!#-+!N-+'*#/!J*(3*-7!:,-L(2-*L,4L/25=3!2.#52-5!&/2-5!

#! 32.*L5#3*+! /2-5(*! ?=,3,-! #@#(#-0=*! +2,+*7! Opt. ExpressE!

MP"6\$jM`^^WUM6`66E!M`6T7!

!YMSZ! K*-*@2*@*! K#'2*?>! et al.! <2-5(*L?=,3,-! /*-/232@*! (25=3L2-L

H5=3!2.#52-57!Nat. Commun.E!\j\`M6E!M`6T7!

!YMTZ! 1*2=&!o&!et al.!X*@*#(2-5!=2++*-!/0*-*/!)>!?=,3,-L*r02*-3!

,00(&/2,-L)#/*+!,??,'3&-2/320!#032@*!2.#52-57!Opt. ExpressE!

M\"_$j^^STU^^\ME!M`6_7!

!YM\Z! J203,'! N'*((#-,E! B2*5,! K&32*''*gE! #-+! N+'2#-! h#'#),7! 1#/3!

)#0QL?',G*032,-! 4,'! -,-L(2-*! ,4! /25=3! '*0,-/3'&032,-7! Opt.

ExpressE!MT"6`$j66TWSU66T_PE!M`6W7!

!YMWZ! ]#33=*F!%mD,,(*E!1*(29!I*2+*E!B#@2+!d7!;2-+*((E!k#2!n#-5E!

<3*@*-! B2#.,-+E! #-+! K,'+,-! O*3g/3*2-7! X*0,-/3'&032-5!

D'#-/2*-3! b.#5*/! 1',.! <2-5(*L[=,3,-! <*-/,'/7! b-! CVPRE!

M`6W7!

!YM_Z! ]#33=*F!%mD,,(*E!B#@2+!d7!;2-+*((E!#-+!K,'+,-!O*3g/3*2-7!

X*#(L32.*! :,-L(2-*L,4L/25=3! b.#52-57! b-! ACM SIGGRAPH

2018 Emerging TechnologiesE!M`6_7!

!YM^Z! <&/#-! C=#-E! X>#-! 87! O#')&'3,-E! K*-*@2*@*! K#'2*?>E!

h,-#3=#-! ;*#0=E! #-+! B#-2*(*! 1#002,7! :,-L(2-*L,4L/25=3!

3'#0Q2-5!,4!?*,?(*!#3!(,-5!'#-5*7!Opt. ExpressE!MT"^$j6`6`^U

6`66WE!M`6W7!

!YP`Z! l,#--! N (3 .# -- E! <3 *? =*-! ] 0; #&5=(2-E! ] 2( */ ! h 7! [# +5 *3 3E !

J2@*Q! k7! K,>#(E! N(4'*+! %7! I*',E! #-+! B#-2*(*! 1#002,7!

s&#-3&.L2-/?2'*+! 0,.?&3#32,-#(! 2.#52-57! ScienceE!

P\6"\S`P$j*##3MM^_E!M`6_7!

!YP6Z! %3Q'2/3! K&?3#E! A,.#/! O2((F#0=*'E!N-+'*#/! J*(3*-E!N/=,Q!

J* *' #' # 5= #@ # -E ! # - +! X #.* /= ! X #/ Q#'7 ! X * 0, - /3 '&0 32 ,- ! , 4!

=2++*-! PB! /=#?*/! &/2-5! +2V&/*! '*c*032,-/7! Opt. ExpressE!

M`"6W$j6^`^\U6^6`_E!M`6M7!

!YPMZ! d>*,-5G,,!N=-E!NQ/=#3!B#@*E!N/=,Q!J**'#'#5=#@#-E!b,#--2/!

KQ2,&(*Q#/E!#-+!N/F2-!C7!<#-Q#'#-#'#>#-#-7!C,-@,(&32,-#(!

N??',92.#32,-/! 3,! 3=*! K*-*'#(! :,-L;2-*L,4L<25=3! b.#52-5!

%?*'#3,'7!b-!ICCVE!M`6^7!

!YPPZ! o2#,0=&-!;2&!et al.!:,-L(2-*L,4L/25=3!2.#52-5!&/2-5!?=#/,'L

H*(+!@2'3&#(!F#@*!,?320/7!NatureE!j6USE!M`6^7!

!YPSZ! ]#33=*F! ]7! ;,?*'! #-+! ]20=#*(! h7! d(#0Q7! %?*-BXj! N-!

"T!

N??',92.#3*!B2V*'*-32#)(*!X*-+*'*'7!b-!ECCVE!M`6S7!

!YPTZ! ['#.,,Q!k=&-5&'-E!B#-2*(!<0=',*+*'E!<=&#-5!n=#,E!k#@23#!

d#(#E!#-+!<3*@*!]#'/0=-*'7!]#30=2-5!'*#(!4#)'20/!F23=!.20',L

#??*#'#-0*! .,+*(/7! ACM Trans GraphE! PT"6$j6j6U6jM\E!

M`6T7!

!YP\Z! b,#--2/! KQ2,&(*Q#/E! N-#3! ;*@2-E! #-+! D,++! n20Q(*'7! N-!

*@#(&#32,-! ,4! 0,.?&3#32,-#(! 2.#52-5! 3*0=-2i&*/! 4,'!

=*3*',5*-*,&/!2-@*'/*!/0#33*'2-57!b-!ECCVE!M`6\7!

!YPWZ! <=&#-5! n=#,E! ;24#-! O&E! 1't+,! B&'#-+E! #-+! X#@2!

X#.#.,,'3=27! B,F-/#.?(2-5! /0#33*'2-5! ?#'#.*3*'/! 4,'!

'*-+*'2-5! #-2/,3',?20! .*+2#7! ACM Trans GraphE!

PT"\$j6\\j6U6\\j66E!M`6\7!

!YP_Z! N+#.! K*@#E! l,#@! l7! <0=*0=-*'E! l,-#3#-! C=*'->#QE! #-+!

X#G2@!K&?3#7!oL'#>!0,.?&3*+!3,.,5'#?=>!3=',&5=!/0#33*'7!b-!

ECCVE!M`6_7!

!YP^Z! <3*?=*-!X,)*'3!]#'/0=-*'7!b-@*'/*!X*-+*'2-5!4,'!C,.?&3*'!

K'#?=20/E![=B!A*/2/E!C,'-*((!a-2@*'/23>E!6^^_7!

!YS`Z! K&/3#@,! [#3,F! #-+! o#@2*'! [&*>,7! N! /&'@*>! ,4! 2-@*'/*!

/&'4#0*! +*/25-! 4',.! (25=3! 3'#-/?,'3! )*=#@2,'! /?*02H0#32,-7!

Comput. Graph. ForumE!MMj\\PU\_WE!M``P7!

!YS6Z! d*-!N??(*3,-!#-+!I&5&*/!D#(),37!K(,)#((>!.2-2.#(!/&'4#0*/!

)>! 0,-32-&,&/! .#92.#(! c,F/7! IEEE Trans. Pattern Anal.

Mach. Intell.E!M_"6$j6`\U66_E!M``\7!

!YSMZ! %(2@2*'! 1#&5*'#/! #-+! X*-#&+! k*'2@*-7! C,.?(*3*! +*-/*!

/3*'*,@2/2,-!&/2-5!(*@*(!/*3!.*3=,+/7!b-!ECCVE!6^^_7!

!YSPZ! N.#u(!B*(#&-,>!#-+!8..#-&*(!['#+,/7!K'#+2*-3!c,F/!4,'!

,?32.2g2-5!3'2#-5&(#'!.*/=L)#/*+!/&'4#0*/j!#??(20#32,-/!3,!P+!

'*0,-/3'&032,-! ?',)(*./! +*#(2-5! F23=!@2/2)2(23>7! Int. J.

Comput. Vis.E!^T"M$j6``U6MPE!M`667!

!YSSZ! ]#3=2*&!B*/)'&-E!]#'Q!]*>*'E![*3*'!<0='v+*'E!#-+!N(#-!I7!

d#''7!b.?(2023!4#2'2-5!,4!2''*5&(#'!.*/=*/!&/2-5!+2V&/2,-!#-+!

0&'@#3&'*!c,F7!b-!SIGGRAPHE!6^^^7!

!YSTZ! ]#'Q!]*>*'E!]#3=2*&!B*/)'&-E![*3*'!<0='v+*'E!#-+!N(#-!I7!

d#''7! B2/0'*3*! B2V*'*-32#(LK*,.*3'>! %?*'#3,'/! 4,'!

D'2#-5&(#3*+!ML]#-24,(+/7!b-!Visualization and Mathematics

IIIE!M``P7!

!YS\Z! %7!<,'Q2-*E!B7!C,=*-L%'E!l7!;2?.#-E!]7!N(*9#E!C7!Xv//(E!#-+!

I7L[7 !< *2 +* ( 7!; # ?( # 02# -!< & '4 # 0* !8 +2 3 2-5 7!b - !Proceedings of the

2004 Eurographics/ACM SIGGRAPH Symposium on

Geometry ProcessingE!M``S7!

!YSWZ! b(>#! 80Q/3*2-E! h*#-L[=2(2??*! [,-/E! l2>2-5! D,-5E! C7LC7! h#>!

k&,E! #-+! ]#3=2*&! B*/)'&-7! K*-*'#(2g*+! /&'4#0*! c,F/! 4,'!

.*/=! ?',0*//2-57! b-! Proceedings of the Fifth Eurographics

Symposium on Geometry ProcessingE!M``W7!

!YS_Z! ]20=#*(! k#g=+#-E! ]#33=*F! d,(23=,E! #-+! I&5&*/! I,??*7!

[,2//,-!/&'4#0*!'*0,-/3'&032,-7!b-!Proceedings of the Fourth

Eurographics Symposium on Geometry ProcessingE!M``\7!

!YS^Z! [*'!C='2/32#-!I#-/*-7!B2/0'*3*!b-@*'/*![',)(*./j!b-/25=3!#-+!

N(5,'23=./7!<bN]E!M`6`7!

!YT`Z! B2!O&!et al.!1'*i&*-0>!N-#(>/2/!,4!D'#-/2*-3!;25=3!D'#-/?,'3!

F23=! N??(20#32,-/! 2-! d#'*! <*-/,'! b.#52-57! b-! Computer

Vision – ECCV 2012E!M`6M7!

!YT6Z! ;(,>+! :7! D'*4*3=*-! #-+! B#@2+! d#&E! bbb7! :&.*'20#(! ;2-*#'!

N(5*)'#7!<bN]E!6^^W7!

!YTMZ! ]20=#*(! k#g=+#-! #-+! I&5&*/! I,??*7! <0'**-*+! [,2//,-!

<&'4#0*! X*0,-/3'&032,-7! ACM Trans GraphE! PM"P$jM^j6U

M^j6PE!M`6P7!

!YTPZ! K'#?=20/!#-+!b.#52-5!;#)7!nL:;%<!B#3#/*3E!K'#?=20/!#-+!

b.#52-5! ;#)7! Y%-(2-*Z7! N@#2(#)(*j!

=33?jff5'#?=20/7&-2g#'7*/f-(,/w+#3#/*37! YN00*//*+j! 6PL<*?L

M`6^Z7!

!YTSZ! I&5&*/! I,??*E! D,->! B*X,/*E! D,.! B&0=#.?E! h,=-!

]0B,-#(+E! #-+! O*'-*'! <3&*3g(*7! <&'4#0*! X*0,-/3'&032,-!

4',.!a-,'5#-2g*+![,2-3/7!b-!SIGGRAPHE!6^^M7!