Chinese
Adv Search
Home | Accepted | Article In Press | Current Issue | Archive | Special Issues | Collections | Featured Articles | Statistics

2020, 2(3): 213-221 Published Date:2020-6-20

DOI: 10.1016/j.vrih.2020.03.001

Summary study of data-driven photometric stereo methods

Full Text: PDF (5) HTML (96)

Export: EndNote | Reference Manager | ProCite | BibTex | RefWorks

Abstract:

Background
A photometric stereo method aims to recover the surface normal of a 3D object observed under varying light directions. It is an ill-defined problem because the general reflectance properties of the surface are unknown.
Methods
This paper reviews existing data-driven methods, with a focus on their technical insights into the photometric stereo problem. We divide these methods into two categories, per-pixel and all-pixel, according to how they process an image. We discuss the differences and relationships between these methods from the perspective of inputs, networks, and data, which are key factors in designing a deep learning approach.
Results
We demonstrate the performance of the models using a popular benchmark dataset.
Conclusions
Data-driven photometric stereo methods have shown that they possess a superior performance advantage over traditional methods. However, these methods suffer from various limitations, such as limited generalization capability. Finally, this study suggests directions for future research.
Keywords: Photometric stereo ; Data-driven methods ; Non-Lambertian reflectance

Cite this article:

Qian ZHENG, Boxin SHI, Gang PAN. Summary study of data-driven photometric stereo methods. Virtual Reality & Intelligent Hardware, 2020, 2(3): 213-221 DOI:10.1016/j.vrih.2020.03.001

1. Kendall A, Martirosyan H, Dasgupta S, Henry P, Kennedy R, Bachrach A, Bry A. End-to-end learning of geometry and context for deep stereo regression. In: Proceedings of the IEEE International Conference on Computer Vision. 2017, 66-75 DOI:10.1109/ICCV.2017.17

2. Furukawa Y, J.Accurate Ponce, dense, and robust multiview stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(8): 1362–1376 DOI:10.1109/tpami.2009.161

3. Woodham R J. Photometric method for determining surface orientation from multiple images. Optical Engineering, 1980, 19(1): 191139 DOI:10.1117/12.7972479

4. Park J, Sinha S N, Matsushita Y, Tai Y W, Kweon I S. Robust multiview photometric stereo using planar mesh parameterization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(8): 1591–1604 DOI:10.1109/tpami.2016.2608944

5. Shi B X, Wu Z, Mo Z P, Duan D L, Yeung S K, Tan P. A benchmark dataset and evaluation for non-lambertian and uncalibrated photometric stereo. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA, IEEE, 2016 DOI:10.1109/cvpr.2016.403

6. Chen L X, Zheng Y Q, Shi B X, Subpa-Asa A, Sato I. A microfacet-based model for photometric stereo with general isotropic reflectance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019 DOI:10.1109/tpami.2019.2927909

7. Ikehata S, Aizawa K. Photometric stereo using constrained bivariate regression for general isotropic surfaces. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA, IEEE, 2014 DOI:10.1109/cvpr.2014.280

8. Ikehata S, Wipf D, Matsushita Y, Aizawa K. Robust photometric stereo using sparse regression. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI. IEEE, 2012 DOI:10.1109/cvpr.2012.6247691

9. Santo H, Samejima M, Sugano Y, Shi B X, Matsushita Y. Deep photometric stereo network. In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW). Venice, IEEE, 2017 DOI:10.1109/iccvw.2017.66

10. Garon M, Sunkavalli K, Hadap S, Carr N, Lalonde J F. Fast spatially-varying indoor lighting estimation. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA, IEEE, 2019 DOI:10.1109/cvpr.2019.00707

11. Ikehata S. CNN-PS: CNN-based photometric stereo for general non-convex surfaces//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018, 3–19 DOI:10.1007/978-3-030-01267-0_1

12. Chen G Y, Han K, Wong K Y K. PS-FCN: A flexible learning framework for photometric stereo// Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018, 3–19 DOI:10.1007/978-3-030-01240-3_1

13. Taniai T, Maehara T. Neural inverse rendering for general reflectance photometric stereo. In: International Conference on Machine Learning. 2018

14. Li J X, Robles-Kelly A, You S D, Matsushita Y. Learning to minify photometric stereo. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA, IEEE, 2019 DOI:10.1109/cvpr.2019.00775

15. Zheng Q, Jia Y M, Shi B X, Jiang X D, Duan L Y, Kot A. SPLINE-net: sparse photometric stereo through lighting interpolation and normal estimation networks. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South), IEEE, 2019 DOI:10.1109/iccv.2019.00864

16. Chen G Y, Han K, Shi B X, Matsushita Y, Wong K Y K K. Self-calibrating deep photometric stereo networks. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA, IEEE, 2019 DOI:10.1109/cvpr.2019.00894

17. Hold-Geoffroy Y, Gotardo P, Lalonde J F. Single day outdoor photometric stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019 DOI:10.1109/TPAMI.2019.2962693

18. Ackermann J, Goesele M. A survey of photometric stereo techniques. Foundations and Trends® in Computer Graphics and Vision, 2015, 9(3/4): 149–254 DOI:10.1561/0600000065

19. Herbort S, Wöhler C. An introduction to image-based 3D surface reconstruction and a survey of photometric stereo methods. 3D Research, 2011, 2(3): 4 DOI:10.1007/3dres.03(2011)4

20. Wu L, Ganesh A, Shi B X, Matsushita Y, Wang Y T, Ma Y. Robust photometric stereo via low-rank matrix completion and recovery//Computer Vision-ACCV 2010. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011, 703–717 DOI:10.1007/978-3-642-19318-7_55

21. Zheng Q, Kumar A, Shi B X, Pan G. Numerical reflectance compensation for non-lambertian photometric stereo. IEEE Transactions on Image Processing, 2019, 28(7): 3177–3191 DOI:10.1109/tip.2019.2894963

22. Shi B X, Tan P, Matsushita Y, Ikeuchi K. Bi-polynomial modeling of low-frequency reflectances. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(6): 1078–1091 DOI:10.1109/tpami.2013.196

23. Antensteiner D, Stolc S, Soukup D. Single image multi-spectral photometric stereo using a split u-shaped CNN. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2019

24. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2015, 234–241 DOI:10.1007/978-3-319-24574-4_28

25. Ju Y K, Dong X H, Wang Y Y, Qi L, Dong J Y. A dual-cue network for multispectral photometric stereo. Pattern Recognition, 2020, 100: 107162 DOI:10.1016/j.patcog.2019.107162

26. Huang G, Liu Z, van der Maaten L, Weinberger K Q. Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, IEEE, 2017 DOI:10.1109/cvpr.2017.243

27. Johnson M K, Adelson E H. Shape estimation in natural illumination. In: CVPR 2011. ColoradoSprings, CO, USA, IEEE, 2011 DOI:10.1109/cvpr.2011.5995510

28. Wiles O, Zisserman A. SilNet: single- and multi-view reconstruction by learning from silhouettes. In: Proceedings of the British Machine Vision Conference 2017. London, UK, British Machine Vision Association, 2017 DOI:10.5244/c.31.99

29. Matusik W, Pfister H, Brand M, McMillan L. A data-driven reflectance model. In: ACM SIGGRAPH 2003 Papers on- SIGGRAPH. San Diego, California, New York, USA, ACM Press, 2003 DOI:10.1145/1201775.882343

30. Burley B, Studios W D. Physically-based shading at Disney, part of practical physically based shading in film and game production. In: Proceedings of ACM SIGGRAPH Courses. 2012

31. Jakob, W.Mitsubarenderer, 2010

32. Cycles. https://www.cycles-renderer.org/

33. Alldrin N, Zickler T, Kriegman D. Photometric stereo with non-parametric and spatially-varying reflectance. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK, USA, IEEE, 2008 DOI:10.1109/cvpr.2008.4587656

34. Einarsson P, Chabert C F, Jones A, Ma W C, Lamond B, Hawkins T, Bolas M, Sylwan S, Debevec P. Relighting human locomotion with flowed reflectance fields. In: Proceedings of the Eurographics Conference on Rendering Techniques, 2006, 183-194

email E-mail this page

Articles by authors

VRIH

BAIDU SCHOLAR