Academia.eduAcademia.edu

Outline

Quality of Experience in a Stereoscopic Multiview Environment

2017, IEEE Transactions on Multimedia

https://doi.org/10.1109/TMM.2017.2714425

Abstract

In this paper we investigate how visualization factors, such as disparity, mobility, angular resolution and viewpoint interpolation, influence the Quality of Experience (QoE) in a stereoscopic multiview environment. In order to do so, we set up a dedicated testing room and conducted subjective experiments. We also developed a framework that emulates a super-multiview environment. This framework can be used to investigate and assess the effects of angular resolution and viewpoint interpolation on the quality of experience produced by multiview systems, and provide relevant cues as to how the baselines of cameras and interpolation strategies in such systems affect user experience. Aspects such as visual comfort, model fluidity, sense of immersion, and the 3D experience as a whole have been assessed for several test cases. Obtained results suggest that user experience in an motion parallax environment is not as critically influenced by configuration parameters such as disparity as initially thought. In addition, extensive subjective tests have indicated that while users are very sensitive to angular resolution in multiview 3D systems, this sensitivity tends not to be as critical when a user is performing a task that involves a great amount of interaction with the multiview content. These tests have also indicated that interpolating intermediate viewpoints can be effective in reducing the required view density without degrading the perceived QoE.

References (51)

  1. L. Zhang and W. J. Tam, "Stereoscopic image generation based on depth images for 3D TV," IEEE Trans. on Broadcasting, vol. 51, no. 2, pp. 191-199, June 2005.
  2. V. De Silva, A. Fernando, S. Worrall, H. K. Arachchi, and A. Kondoz, "Sensitivity analysis of the human visual system for depth cues in stereoscopic 3-D displays," IEEE Trans. on Multimedia, vol. 13, no. 3, pp. 498-506, 2011.
  3. J. Gutiérrez, F. Jaureguizar, and N. García, "Subjective comparison of consumer television technologies for 3d visualization," Journal of Display Technology, vol. 11, no. 11, pp. 967-974, 2015.
  4. S. Reichelt, R. Häussler, G. Fütterer, and N. Leister, "Depth cues in human visual perception and their realization in 3d displays," in Proceedings of SPIE, 2010, pp. 76900B-76900B-12.
  5. ISO/IEC SC29WG11, "Summary of Call for Evidence on Free- Viewpoint Television: Super-Multiview and Free Navigation," Output doc., 116th MPEG Meeting, Oct 2016.
  6. F. Pereira, E. A. B. da Silva, and G. Lafruit, "Plenoptic imaging: Repre- sentation and processing," Academic Press, To appear in R. Chellappa and S. Theodoridis, Academic Press Library in Signal Processing.
  7. T. Ebrahimi, S. Foessel, F. Pereira, and P. Schelkens, "JPEG Pleno: Toward an Efficient Representation of Visual Reality," IEEE MultiMedia, vol. 23, no. 4, pp. 14-20, 2016.
  8. L. Janowski and M. Pinson, "The Accuracy of Subjects in a Quality Experiment: A Theoretical Subject Model," IEEE Trans. on Multimedia, vol. 17, no. 12, pp. 2210-2224, 2015.
  9. E. Bosc, R. Pepion, P. Le Callet, M. Koppel, P. Ndjiki-Nya, M. Pres- sigout, and L. Morin, "Towards a new quality metric for 3-d synthesized view assessment," IEEE Journal of Selected Topics in Signal Processing, vol. 5, no. 7, pp. 1332-1343, 2011.
  10. M. S. Farid, M. Lucenteforte, and M. Grangetto, "Objective quality metric for 3d virtual views," in Proceedings of ICIP, 2015, pp. 3720- 3724.
  11. Junyong You, Liyuan Xing, Andrew Perkis, and Xu Wang, "Perceptual quality assessment for stereoscopic images based on 2d image quality metrics and disparity analysis," in Proceeding of the VPQM, January 2010.
  12. T. Kim, J. Kang, S. Lee, and A. C. Bovik, "Multimodal interactive continuous scoring of subjective 3D video quality of experience," IEEE Trans. on Multimedia, vol. 16, no. 2, pp. 387-402, 2014.
  13. Atanas Boev, Danilo Hollosi, Atanas Gotchev, and Karen Egiazarian, "Classification and simulation of stereoscopic artifacts in mobile 3DTV content," in Proceedings of SPIE, 2009, vol. 7237, pp. 72371F-72371F- 12.
  14. Jiwoo Kang, Taewan Kim, and Sanghoon Lee, "Implementation of Mul- timodal Interactive Continuous Scoring for 3D Quality of Experience," Wireless Personal Communications, vol. 84, no. 2, pp. 1133-1149, 2016.
  15. ITU-R, "Subjective assessment of stereoscopic television pictures," Tech. Rep., Rec. BT.1438, 2000.
  16. Jiheng Wang, Perceptual Quality-of-Experience of Stereoscopic 3D Images and Videos, Ph.D. thesis, University of Waterloo, 2016.
  17. F. Qi, D. Zhao, and W. Gao, "Reduced reference stereoscopic image quality assessment based on binocular perceptual information," IEEE Trans. on Multimedia, vol. 17, no. 12, pp. 2338-2344, 2015.
  18. W. Zhou and L. Yu, "Binocular responses for no-reference 3D image quality assessment," IEEE Trans. on Multimedia, vol. 18, no. 6, pp. 1077-1084, 2016.
  19. S. Tourancheau, M. Sjöström, R. Olsson, A. Persson, and T. Ericson, "Evaluation of quality of experience in interactive 3D visualization: methodology and results," in Proceedings of SPIE, January 2012, vol. 8288, pp. 82880O+.
  20. C. Caudek and D. R. Proffitt, "Depth perception in motion parallax and stereokinesis," Journal of Experimental Psichology, vol. 19, no. 1, pp. 32-47, 1993.
  21. C. Zhang, Z. Yin, and D. Florêncio, "Improving depth perception with motion parallax and its application in teleconferencing," in Proceedings of the IEEE International Workshop on Multimedia Signal Processing (MMSP'09), October 2009, pp. 1-6.
  22. & WG11, "Technical report of the joint ad hoc group for digital representations of light/sound fields for immersive media applications," Output doc., MPEG 115th Meeting, June 2016.
  23. ISO/IEC JTC 1/SC 29/WG 1, "JPEG Pleno Call for Proposals on Light Field Coding," Output doc., 73rd Meeting, Oct 2016.
  24. A. G. Ciancio, J. F. L. de Oliveira, F. M. L. Ribeiro, and E. A. B. da Silva, "Quality perception in 3D interactive environments," in Proceedings of ISCAS, 2013, pp. 9-12.
  25. F. M. L. Ribeiro, J. F. L. de Oliveira, A. G. Ciancio, E. A. B. da Silva, L. G. C. Tavares, J. N. Gois, C. D. Estrada, A. Said, and B. Lee, "Impact of angular view density on the user experience in a multiview interactive 3D environment," in Proceedings of VPQM, 2014.
  26. A. K. Moorthy and A. C. Bovik, "A survey on 3D quality of experience and 3D quality assessment," in Proceedings of SPIE, March 2013, vol. 8651, pp. 86510M+.
  27. S. J. Daly, R. T. Held, and D. M. Hoffman, "Perceptual issues in stereoscopic signal processing," IEEE Trans. on Broadcasting, vol. 57, no. 2, pp. 347-361, June 2011.
  28. L. Xing, J. You, T. Ebrahimi, and A. Perkis, "Assessment of stereoscopic crosstalk perception," IEEE Trans. on Multimedia, vol. 14, no. 3, pp. 326-337, 2012.
  29. W. Chen, J. Fournier, M. Barkowsky, and P. Le Callet, "Exploration of quality of experience of stereoscopic images: binocular depth," in Proceeding of the VPQM, January 2012, pp. 1-6.
  30. S. Yano, S. Ide, T. Mitsuhashi, and H. Thwaites, "A study of visual fatigue and visual comfort for 3D HDTV/HDTV images," Displays, vol. 23, no. 4, pp. 191-201, April 2002.
  31. Y. Takaki, Y. Urano, and H. Nishio, "Motion-parallax smoothness of short-, medium-, and long-distance 3D image presentation using multi- view displays," Opt. Express, vol. 20, no. 24, pp. 27180-27197, 11 2012.
  32. S. Pastoor and K. Schenke, "Subjective assessments of the resolution of viewing directions in a multi-viewpoint 3D TV system," Proceedings of the SID, vol. 30, no. 3, pp. 217-223, 1989.
  33. D. Runde, "How to realize a natural image reproduction using stereo- scopic displays with motion parallax," IEEE Trans. on Circuits and Systems for Video Technology, vol. 10, no. 3, pp. 376-386, April 2000.
  34. F. Speranza, W. J. Tam, T. Martin, L. Stelmach, and C. Ahn, "Perceived smoothness of viewpoint transition in multi-viewpoint stereoscopic displays," in Proceedings of SPIE, June 2005, vol. 5664, pp. 72-82.
  35. M. Tanimoto, Free-Viewpoint Television, pp. 53-76, Springer Berlin Heidelberg, 2010.
  36. Alioscopy, "Alioscopy autostereoscopic 3D screens," http://www.alioscopy.com/en/home.php, July 2013.
  37. OpenSceneGraph, "Open source high performance 3D graphics toolkit," http://www.openscenegraph.org, July 2013.
  38. Y. Takaki, "Development of Super Multi-View Displays," ITE Trans. on Media Technology and Applications, vol. 2, no. 1, pp. 8-14, 2014.
  39. L. H. Hardy, G. Rand, and M. C. Rittler, "Tests for the detection and analysis of color-blindness. I. The Ishihara test: An evaluation," J. Opt. Soc. Am., vol. 35, no. 4, pp. 268-275, Apr 1945.
  40. ITU-R, "Subjective methods for assessment of stereoscopic 3DTV systems," Tech. Rep., Rec. BT.2021, 2012.
  41. S. Winkler and D. Min, "Stereo/multiview picture quality: Overview and recent advances," Signal Processing: Image Communication, vol. 28, no. 10, pp. 1358-1373, 2013.
  42. B. S. Everitt and A. Skrondal, "The Cambridge dictionary of statistics," Cambridge: Cambridge, 2002.
  43. D. C. Montgomery, Design and analysis of experiments, John Wiley & Sons, 1991.
  44. URL, "3D: Multi-view interactive 3D environments," Available at http://www.smt.ufrj.br/~tvdigital/3d., May 2016.
  45. M. Zwicker, W. Matusik, F. Durand, and H. Pfister, "Antialiasing for automultiscopic 3D displays," in Eurographics Symposium on Rendering, 2006, pp. 73-82.
  46. M. Pharr and G. Humphreys, Physically based rendering: From theory to implementation, Morgan Kaufmann, 2004.
  47. ISO/IEC SC29WG11, "Experimental Framework for FTV," Output doc., 110th MPEG Meeting, Oct 2014.
  48. Christoph Fehn, "Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV," in Proceedings of SPIE, 2004, vol. 5291, pp. 93-104.
  49. Felipe M. L. Ribeiro was born in Rio de Janeiro, Brazil, in 1987. He received his B.Sc. degree in Electronics Engineering from Universidade Federal do Rio de Janeiro (UFRJ) in 2012; he received his M.Sc. from COPPE/UFRJ in 2014 and is now pursuing his D.Sc. degree from COPPE/UFRJ, both in Electrical Engineering. His research interests in- clude image processing, video/image quality evalua- tion, computer vision, machine learning, and pattern recognition.
  50. José F. L. de Oliveira has graduated in Electrical Engineering (1994) from the Universidade Federal do Rio de Janeiro (UFRJ) and received M.Sc. (1997) and D.Sc. (2003) in Electrical Engineering from the Universidade Federal do Rio de Janeiro (UFRJ). His research interests include signal processing, image compression, and pattern recognition-tracking.
  51. Alexandre G. Ciancio received the B.Sc. and M.Sc. degrees in Electrical Engineering from Universidade Federal do Rio de Janeiro (COPPE/UFRJ), Brazil, in 1999 and 2001, respectively, and the Ph.D. degree in Electrical Engineering from the University of Southern California, Los Angeles, CA, in 2006. He was a Post Doc researcher at COPPE/UFRJ from 2006 to 2007 and took part in research projects at COPPE/UFRJ from 2007 to 2015, including a collaboration with HP on multimedia quality assess- ment. His main areas of interest are distributed compression algorithms and quality assessment of digital video. He is currently at the Brazilian Patent Office (INPI), where he was a patent examiner at the Telecommunications Division from 2009 to 2015. He is now Technical Assistant of the Director of Patents of INPI.