Chinese
Adv Search
Home | Accepted | Article In Press | Current Issue | Archive | Special Issues | Collections | Featured Articles | Statistics

2019, 1(1): 39-54 Published Date:2019-2-1

DOI: 10.3724/SP.J.2096-5796.2018.0004

A review on image-based rendering

Full Text: PDF (157) HTML (883)

Export: EndNote | Reference Manager | ProCite | BibTex | RefWorks

Abstract:

Image-based rendering is important both in the field of computer graphics and computer vision, and it is also widely used in virtual reality technology. For more than two decades, people have done a lot of work on the research of image-based rendering, and these methods can be divided into two categories according to whether the geometric information of the scene is utilized. According to this classification, we introduce some classical methods and representative methods proposed in recent years. We also compare and analyze the basic principles, advantages and disadvantages of different methods. Finally, some suggestions are given for research directions on image-based rendering techniques in the future.
Keywords: Image-based rendering ; Virtual reality ; Image interpolation ; Panorama

Cite this article:

Yuan CHANG, Guo-Ping WANG. A review on image-based rendering. Virtual Reality & Intelligent Hardware, 2019, 1(1): 39-54 DOI:10.3724/SP.J.2096-5796.2018.0004

1. Snavely N, Seitz S M, Szeliski R. Photo Tourism: Exploring image collections in 3D. ACM Transactions on Graphics, 2006, 25: 835–846 DOI:10.1145/1141911.1141964

2. Furukawa Y, J. Accurate Ponce, dense, and robust multi-view stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2010, 32(8): 1362–1376 DOI:10.1109/TPAMI.2009.161

3. Google Street View. https: //www. google. com/streetview/

4. Shum H Y, Kang S B. A review of image-based rendering techniques. In: NganK N, SikoraT, SunM T, eds. Proceedings of IEEE/SPIE Visual Communications and Image Processing (VCIP), Perth. Australia, 2000, 2–13

5. Adelson E H, Bergen J R. The plenoptic function and the elements of early vision. In: Landy M S, Movshon J A, eds. Computational Models of Visual Processing. Cambridge: MIT Press, 1991

6. Wong T T, Heng P A, Or S H, Ng W Y. Image-based rendering with controllable illumination. In: Dorsey J, Slusallek P, eds. Proceedings of the 8-th Eurographics Workshop on Rendering. Berlin: Springer-Verlag, 1997, 13–22

7. McMillan L, Bishop G. Plenoptic modeling: An image-based rendering system. In: Mair SG, Cook R, eds. Proceedings of the 22nd annual conference on Computer graphics and interactive techniques. New York: ACM, 1995, 39–46

8. Levoy M, Hanrahan P. Light field rendering. In: Fujii J, eds. Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. New York: ACM, 1996, 31–42 DOI:10.1145/237170.237199

9. Shum H Y, He L W. Rendering with concentric mosaics. In: Waggenspack W, eds. Proceedings of the 26th annual conference on Computer graphics and interactive techniques. New York: ACM Press/Addison-Wesley Publishing Co., 1999, 299–306 DOI:10.1145/311535.311573

10. Szeliski R, Shum H Y. Creating full view panoramic image mosaics and texture-mapped models. In: OwenG S, WhittedT, Mones-HattalB, eds. Proceedings of the 24th annual conference on Computer graphics and interactive techniques. New York: ACM Press/Addison-Wesley Publishing Co., New York, 1997, 251–258 DOI:10.1145/258734.258861

11. Chen S E. Quick Time VR–an image-based approach to virtual environment navigation. In: Mair SG, Cook R, eds. Proceedings of the 22nd annual conference on Computer graphics and interactive techniques. New York: ACM, 1995, 29–38 DOI:10.1145/218380.218395

12. Roundshot. http: //www. roundshot. com

13. Shum H Y, Szeliski R. Construction and refinement of panoramic mosaics with global and local alignment. In: Sixth International Conference on Computer Vision (ICCV’98). Bombay, 1998, 953–958 DOI:10.1109/ICCV.1998.710831

14. Gortler S J, Grzeszczuk R, Szeliski R, Cohen M F. The lumigraph. In: Fujii J, eds. Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. New York: ACM, 1996, 43–54 DOI:10.1145/237170.237200

15. Debevec P, Downing G, Bolas M, Peng H-Y, Urbach J. Spherical light field environment capture for virtual reality using a motorized pan/tilt head and offset camera. In: ACM Siggraph 2015 Posters. Los Angeles, California, 2015, 1–1 DOI:10.1145/2787626.2787648

16. Buehler C, Bosse M, McMillan L, Gortler S, Cohen M. Unstructured lumigraph rendering. In: Pocock L, eds. Proceedings of the 28th annual conference on Computer graphics and interactive techniques. New York: ACM, 2001, 425–432 DOI:10.1145/383259.383309

17. Rademacher P. View-dependent geometry. In: Waggenspack W, eds. Proceedings of the 26th annual conference on Computer graphics and interactive techniques. New York: ACM Press/Addison-Wesley Publishing Co., 1999, 439–446 DOI:10.1145/311535.311612

18. Vedula S, Baker S, Kanade T. Spatio-temporal view interpolation. In: Gibson S, Debevec P, eds. Proceedings of the 13th Eurographics Workshop on Rendering. Aire-la-Ville: Eurographics Association, 2002

19. Chaurasia G, Duchene S, Sorkine-Hornung O, Drettakis G. Depth synthesis and local warps for plausible image-based navigation. ACM Transactions on Graphics, 2013, 32(3): 1–12 DOI:10.1145/2487228.2487238

20. Lipski C, Klose F, Magnor M. Correspondence and depth-image based rendering a hybrid approach for free-viewpoint video. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(6): 942–951 DOI:10.1109/TCSVT.2014.2302379

21. Shade J, Gortler S, He L W, Szeliski R. Layered depth images. In: CunninghamS, BransfordW, CohenM F, eds. Proceedings of the 25th annual conference on Computer graphics and interactive techniques. New York: ACM, 1998, 231–242. DOI:10.1145/280814.280882

22. Chang C, Bishop G, Lastra A. LDI tree: A hierarchical representation for image-based rendering. In: Waggenspack W, eds. Proceedings of the 26th annual conference on Computer graphics and interactive techniques. New York: ACM Press/Addison-Wesley Publishing Co., 1999: 291–298 DOI:10.1145/311535.311571

23. Penner E, Zhang L. Soft 3D reconstruction for view synthesis. ACM Transactions on Graphics, 2017, 36(6): 1–11 DOI:10.1145/3130800.3130855

24. Chen S, Williams L. View interpolation for image synthesis. In: Whitton MC, eds. Proceedings of the 20th annual conference on Computer graphics and interactive techniques. New York: ACM, 1993, 279–288 DOI:10.1145/166117.166153

25. Seitz S M, Dyer C M. View morphing. In: Fujii J, eds. Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. New York: ACM, 1996, 21–30 DOI:10.1145/237170.237196

26. Nie Y W, Zhang Z S, Sun H Q, Su T, Li G. Homography propagation and optimization for wide-baseline street image interpolation. IEEE Transactions on Visualization and Computer Graphics, 2017, 23(10): 2328–2341 DOI:10.1109/TVCG.2016.2618878

27. Kopf J, Langguth F, Scharstein D, Szeliski R, Goesele M. Image-based rendering in the gradient domain. ACM Transactions on Graphics, 2013, 32(6): 1–9 DOI:10.1145/2508363.2508369

28. Sinha S N, Kopf J, Goesele M, Scharstein D. Image-based rendering for scenes with reflections. ACM Transactions on Graphics, 2012, 31(4): 1–10

29. Thonat T, Djelouah A, Durand F, Drettakis G. Thin structures in image based rendering. Computer Graphics Forum, 2018, 37: 107–118

30. Yang H S, Lin W Y, Lu J B. DAISY Filter Flow: A generalized discrete approach to dense correspondences. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, 2014, 3406–3413

31. Bao L C, Yang Q X, Jin H L. Fast edge-preserving PatchMatch for large displacement optical flow. IEEE Transactions on Image Processing, 2014, 23: 4996–5006 DOI:10.1109/CVPR.2014.452

32. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2274–2282 DOI:10.1109/TPAMI.2012.120

33. Fischler M A, Bolles R C. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM, 1981, 24(6): 381–395 DOI:10.1145/358669.358692

34. Hawe S, Kleinsteuber M, Diepold K. Dense disparity maps from sparse disparity measurements. In: 2011 International Conference on Computer Vision. Barcelona, 2011, 2126–2133 DOI:10.1109/ICCV.2011.6126488

35. Sinha S N, Steedly D, Szeliski R. Piecewise planar stereo for image-based rendering. In: 2009 IEEE 12th International Conference on Computer Vision. Kyoto, 2009, 1881–1888

36. Gallup D, Frahm J, Pollefeys M. Piecewise planar and non-planar stereo for urban scene reconstruction. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA, 2010, 1418–1425

37. Furukawa Y, Curless B, Seitz S M, Szeliski R. Manhattan-world stereo. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL, 2009, 1422–1429 DOI:10.1109/CVPR.2009.5206867

38. Lipski C, Linz C, Neumann T, Wacker M, Magnor M. High resolution image correspondences for video post-produc-tion. In: 2010 Conference on Visual Media Production. London, 2010, 33–39 DOI:33-39.10.1109/CVMP.2010.12

39. Liu C, Yuen J, Torralba A. SIFT flow: Dense correspondence across different scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(5): 978–994 DOI:10.1109/TPAMI.2010.147

40. Hosni A, Bleyer M, Rhemann C, Gelautz M, Rother C. Real-time local stereo matching using guided image filtering. In: 2011 IEEE International Conference on Multimedia and Expo. Barcelona, 2011, 1–6 DOI:10.1109/ICME.2011.6012131

41. Ma Z Y, He K M, Wei Y C. Constant time weighted median filtering for stereo matching and beyond. In: 2013 IEEE International Conference on Computer Vision. Sydney, NSW, 2013, 49–56

42. He K M, Sun J, Tang X O. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2013, 35: 1397–1409 DOI:10.1109/TPAMI.2012.213

43. Hedman P, Alsisan S, Szeliski R, Kopf J. Casual 3D photography. ACM Transactions on Graphics, 2017, 36: 234: 1–15

44. Schönberger J L, Zheng E L, Frahm J M. Pixelwise view selection for unstructured multi-view stereo. In: European Conference on Computer Vision. Amsterdam, the Netherlands, 2016

45. Hedman P, Kopf J. Instant 3D photography. ACM Transactions on Graphics, 2017, 36(6): 1–15 DOI:10.1145/3130800.3130828

46. Hedman P, Ritschel T, Drettakis G, Brostow G. Scalable inside-out image-based rendering. ACM Transactions on Graphics, 2016, 35(6): 1–11. DOI:10.1145/2980179.2982420

47. Pérez P, Gangnet M, Blake A. Poisson image editing. ACM Transactions on Graphics, 2003, 22: 313–318 DOI:10.1145/1201775.882269

48. Burt P J, Adelson E H. A multiresolution spline with application to image mosaics. ACM Transactions on Graphics, 1983, 2: 217–236 DOI:10.1145/245.247

49. Kalantari N K, Wang T C, Ramamoorthi R. Learning based view synthesis for light field cameras. ACM Transactions on Graphics, 2016, 35(6): 1–10 DOI:10.1145/2980179.2980251

50. Flynn J, Neulander I, Philbin J, Snavely N. DeepStereo: Learning to predict new views from the world’s imagery. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, 2016, 35(6): 1–10 DOI:10.1145/2980179.2980251

email E-mail this page

Articles by authors

VRIH

BAIDU SCHOLAR

WANFANG DATA