Academia.eduAcademia.edu

Outline

Title
Introduction
Background
Scene Point Motion Analysis
Experimental Results and Discussion
Conclusion
References
All Topics
Computer Science
Artificial Intelligence

Ray-Space Motion Compensation for Lenslet Plenoptic Video Coding

Profile image of Byeungwoo JeonByeungwoo Jeon

2022, arXiv (Cornell University)

https://doi.org/10.48550/ARXIV.2207.00522
visibility

…

description

15 pages

link

1 file

Related papers

Lossless coding of light field images based on minimum-rate predictors
Luis A. Silva Cruz

Journal of Visual Communication and Image Representation, 2018

Recent developments in light field acquisition and computational photography are driving new research efforts on light field encoding methods, capable of exploiting the specific features of this type of visual data. This paper presents a research study of lossless light field image compression, using Minimum-Rate Predictors (MRP) and mainstream image and video encoders. The research is focused on three light field representation formats: lenslet images, stack of sub-aperture images and epipolar images. The main contributions of this work are the 'Spiral-blackend' serialization method and the use of MRP for the lossless compression of light fields with joint encoding of RGB data. The results show that the lenslet format yields lower compression efficiencies than other formats. Furthermore, it is demonstrated that the MRP algorithm consistently outperforms HEVC-RExt, JPEG2000, JPEG-LS and CALIC when light fields are represented by either a stack of sub-aperture or epipolar images.

View PDFchevron_right
A Novel Randomize Hierarchical Extension of MV-HEVC for Improved light Field Compression
Mansi Sharma

9th International Conference on 3D Immersion (IC3D 2019), Brussels, Belgium, 2019

This paper presents a novel scheme for light field compression based on a randomize hierarchical multi-view extension of high efficiency video coding (dubbed as RH-MVHEVC). Specifically, a light field data are arranged as a multiple pseudotemporal video sequences which are efficiently compressed withMV-HEVC encoder, following an integrated random coding technique and hierarchical prediction scheme. The critical advantage of proposed RH-MVHEVC scheme is that it utilizes not just a temporal and inter-view prediction, but efficiently exploits the strong intrinsic similarities within each sub-aperture image and among neighboring sub-aperture images in both horizontal and vertical directions. Experimental results consistently outperform the state-of-the-art compression methods on benchmark ICME 2016 and ICIP 2017 grand challenge data sets. It achieves an average up to 33.803% BDrate reduction and 1.7978 dB BD-PSNR improvement compared with an advanced JEM video encoder, and an average 20.4156% BD-rate reduction and 2.0644 dB BD-PSNR improvement compared with a latest image-based JEM-anchor coding scheme.

View PDFchevron_right
Performance comparison of video encoders in light field image compression
Manuela Pereira

Electronic Imaging, 2021

Efficient compression plays a significant role in Light Field imaging technology because of the huge amount of data needed for their representation. Video encoders using different strategies are commonly used for Light Field image compression. In this paper, different video encoder implementations including HM, VTM, x265, xvc, VP9, and AV1 are analysed and compared in terms of coding efficiency, and encoder/decoder time-complexity. Light field images are compressed as pseudo-videos.

View PDFchevron_right
Scalable Light Field Coding with Support for Region of Interest Enhancement
Paulo Nunes

2018 26th European Signal Processing Conference (EUSIPCO), 2018

Light field imaging based on microlens arraysa.k.a. holoscopic, plenoptic, and integral imaging-has currently risen up as a feasible and prospective technology for future image and video applications. However, deploying actual light field applications will require identifying more powerful representation and coding solutions that support emerging manipulation and interaction functionalities. In this context, this paper proposes a novel scalable coding approach that supports a new type of scalability, referred to as Field of View (FOV) scalability, in which enhancement layers can correspond to regions of interest (ROI). The proposed scalable coding approach comprises a base layer compliant with the High Efficiency Video Coding (HEVC) standard, complemented by one or more enhancement layers that progressively allow richer versions of the same light field content in terms of content manipulation and interaction possibilities, for the whole scene or just for a given ROI. Experimental results show the advantages of the proposed scalable coding approach with ROI support to cater for users with different preferences/requirements in terms of interaction functionalities. Keywords-light field, field of view scalability, region of interest, image compression, HEVC This work was supported by FCT (Fundação para a Ciência e a Tecnologia, Portugal) under the UID/EEA/50008/2013 project.

View PDFchevron_right
Data compression for light-field rendering
M. Magnor

IEEE Transactions on Circuits and Systems for Video Technology, 2000

Two light-field compression schemes are presented. The codecs are compared with regard to compression efficiency and rendering performance. The first proposed coder is based on video-compression techniques that have been modified to code the four-dimensional light-field data structure efficiently. The second coder relies entirely on disparity-compensated image prediction, establishing a hierarchical structure among the light-field images. Coding performance of both schemes is evaluated using publicly available light fields of synthetic, as well as real-world, scenes. Compression ratios vary between 100 : 1 and 2000 : 1, depending on reconstruction quality and light-field scene characteristics.

View PDFchevron_right
Light field compression using disparity-compensated lifting and shape adaptation
Xiaoqing Zhu

IEEE Transactions on Image Processing, 2000

We propose a novel approach for light field compression that incorporates disparity compensation into 4-D wavelet coding using disparity-compensated lifting. With this approach, we obtain the benefits of wavelet coding, including compression efficiency and scalability in all dimensions. Additionally, our proposed approach solves the irreversibility limitations of previous wavelet coding approaches. Experimental results show that the compression efficiency of the proposed technique outperforms current state-of-theart wavelet coding techniques by a wide margin.

View PDFchevron_right
Lossless Coding of Light Fields Based on 4D Minimum Rate Predictors
Sergio Faria

IEEE Transactions on Image Processing, 2022

Common representations of light fields use fourdimensional data structures, where a given pixel is closely related not only to its spatial neighbours within the same view, but also to its angular neighbours, co-located in adjacent views. Such structure presents increased redundancy between pixels, when compared with regular single-view images. Then, these redundancies are exploited to obtain compressed representations of the light field, using prediction algorithms specifically tailored to estimate pixel values based on both spatial and angular references. This paper proposes new encoding schemes which take advantage of the four-dimensional light field data structures to improve the coding performance of Minimum Rate Predictors. The proposed methods expand previous research on lossless coding beyond the current state-of-the-art. The experimental results, obtained using both traditional datasets and others more challenging, show bit-rate savings no smaller than 10%, when compared with existing methods for lossless light field compression.

View PDFchevron_right
Evaluation of 4D Light Field Compression Methods
David Barina

2019

Light field data records the amount of light at multiple points in space, captured e.g. by an array of cameras or by a light-field camera that uses microlenses. Since the storage and transmission requirements for such data are tremendous, compression techniques for light fields are gaining momentum in recent years. Although plenty of efficient compression formats do exist for still and moving images, only a little research on the impact of these methods on light field imagery is performed. In this paper, we evaluate the impact of state-of-the-art image and video compression methods on quality of images rendered from light field data. The methods include recent video compression standards, especially AV1 and XVC finalised in 2018. To fully exploit the potential of common image compression methods on four-dimensional light field imagery, we have extended these methods into three and four dimensions. In this paper, we show that the four-dimensional light field data can be compressed much more than independent still images while maintaining the same visual quality of a perceived picture. We gradually compare the compression performance of all image and video compression methods, and eventually answer the question, "What is the best compression method for light field data?".

View PDFchevron_right
Steered Mixture-of-Experts for Light Field Images and Video: Representation and Coding
Ruben Verhack

IEEE Transactions on Multimedia

Research in light field (LF) processing has heavily increased over the last decade. This is largely driven by the desire to achieve the same level of immersion and navigational freedom for camera-captured scenes as it is currently available for CGI content. Standardization organizations such as MPEG and JPEG continue to follow conventional coding paradigms in which viewpoints are discretely represented on 2-D regular grids. These grids are then further decorrelated through hybrid DPCM/transform techniques. However, these 2-D regular grids are less suited for high-dimensional data, such as LFs. We propose a novel coding framework for higher-dimensional image modalities, called Steered Mixture-of-Experts (SMoE). Coherent areas in the higher-dimensional space are represented by single higher-dimensional entities, called kernels. These kernels hold spatially localized information about light rays at any angle arriving at a certain region. The global model consists thus of a set of kernels which define a continuous approximation of the underlying plenoptic function. We introduce the theory of SMoE and illustrate its application for 2-D images, 4-D LF images, and 5-D LF video. We also propose an efficient coding strategy to convert the model parameters into a bitstream. Even without provisions for high-frequency information, the proposed method performs comparable to the state of the art for low-tomid range bitrates with respect to subjective visual quality of 4-D LF images. In case of 5-D LF video, we observe superior decorrelation and coding performance with coding gains of a factor of 4x in bitrate for the same quality. At least equally important is the fact that our method inherently has desired functionality for LF rendering which is lacking in other state-ofthe-art techniques: (1) full zero-delay random access, (2) lightweight pixel-parallel view reconstruction, and (3) intrinsic view interpolation and super-resolution.

View PDFchevron_right
Fast and Efficient Lenslet Image Compression
Manuela Pereira

ArXiv, 2019

Light field imaging is characterized by capturing brightness, color, and directional information of light rays in a scene. This leads to image representations with huge amount of data that require efficient coding schemes. In this paper, lenslet images are rendered into sub-aperture images. These images are organized as a pseudo-sequence input for the HEVC video codec. To better exploit redundancy among the neighboring sub-aperture images and consequently decrease the distances between a sub-aperture image and its references used for prediction, sub-aperture images are divided into four smaller groups that are scanned in a serpentine order. The most central sub-aperture image, which has the highest similarity to all the other images, is used as the initial reference image for each of the four regions. Furthermore, a structure is defined that selects spatially adjacent sub-aperture images as prediction references with the highest similarity to the current image. In this way, encoding...

View PDFchevron_right

References (78)

  1. G. Lippmann, ''Épreuves rØversibles donnant la sensation du relief,'' J. dePhys. ThØ orique et AppliquØ e, vol. 7, no. 1, pp. 821-825, Jan. 1908.
  2. R. Ng, ''Digital light field photography,'' Ph.D. dissertation, Dept. Comput. Sci., Stanford Univ., Stanford, CA, USA, 2006
  3. H. Jeon et al., "Accurate depth map estimation from a lenslet light field camera," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
  4. E. Adelson and J. Bergen, The plenoptic function and the elements of early vision. Cambridge, Mass.: Vision and Modeling Group, Media Laboratory, Massachusetts Institute of Technology, 1991.
  5. M. Levoy and P. Hanrahan, "Light field rendering," Proceedings of the 23rd annual conference on Computer graphics and interactive techniques -SIGGRAPH '96, 1996.
  6. R. Ng, "Light field photography with a hand-held plenoptic camera," Technical report, Stanford University, 2005.
  7. A. Lumsdaine and T. Georgiev, "The focused plenoptic camera," 2009 IEEE International Conference on Computational Photography (ICCP), 2009.
  8. Looking Glass display, https://lookingglassfactory.com/.
  9. Sony Spatial Reality display, https://electronics.sony.com/spatial-reality- display/p/elfsr1.
  10. A. Mieee, "A 3D Dct Compression Algorithm for Omnidirectional Integral Images," 2006 IEEE International Conference on Acoustics Speed and Signal Processing Proceedings.
  11. H.-H. Kang, D.-H. Shin, and E.-S. Kim, ''Compression scheme of subimages using karhunen-loeve transform in three-dimensional integral imaging,'' Opt. Commun., vol. 281, no. 14, pp. 3640-3647, Jul. 2008.
  12. A. Aggoun, ''Compression of 3D integral images using 3D wavelet transform,'' J. Display Technol., vol. 7, no. 11, pp. 586-592, Nov. 2011.
  13. M. B. de Carvalho et al., "A 4D DCT-Based Lenslet Light Field Codec," 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, 2018, pp. 435-439.
  14. M. Budagavi and D. Kwon, "Intra motion compensation and entropy coding improvements for HEVC screen content coding," 2013 Picture Coding Symposium (PCS), 2013.
  15. Y. Li, R. Olsson and M. Sjostrom, "Compression of unfocused plenoptic images using a displacement intra prediction," 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2016.
  16. C. Conti, L. Soares and P. Nunes, "HEVC-based 3D holoscopic video coding using self-similarity compensated prediction," Signal Processing: Image Communication, vol. 42, pp. 59-78, 2016.
  17. L. F. R. Lucas et al., "Locally linear embedding-based prediction for 3D holoscopic image coding using HEVC," 2014 22nd European Signal Processing Conference (EUSIPCO), pp. 11-15, 2014.
  18. X. Jin, H. Han and Q. Dai, "Image Reshaping for Efficient Compression of Plenoptic Content," IEEE Journal of Selected Topics in Signal Processing, vol. 11, no. 7, pp. 1173-1186, 2017.
  19. X. Jin, H. Han and Q. Dai, "Plenoptic Image Coding Using Macropixel- Based Intra Prediction," IEEE Transactions on Image Processing, vol. 27, no. 8, pp. 3954-3968, 2018.
  20. F. Dai, J. Zhang, Y. Ma and Y. Zhang, "Lenselet image compression scheme based on subaperture images streaming," 2015 IEEE International Conference on Image Processing (ICIP), 2015.
  21. L. Li, Z. Li, B. Li, D. Liu and H. Li, "Pseudo-Sequence-Based 2-D Hierarchical Coding Structure for Light-Field Image Compression," IEEE Journal of Selected Topics in Signal Processing, vol. 11, no. 7, pp. 1107- 1119, 2017.
  22. J. Wei, S. Wang, Y. Zhao and F. Jin, "Hierarchical prediction structure for subimage coding and multithreaded parallel implementation in integral imaging," Applied Optics, vol. 50, no. 12, p. 1707, 2011.
  23. W. Ahmad, R. Olsson and M. Sjostrom, "Interpreting plenoptic images as multi-view sequences for improved compression," 2017 IEEE International Conference on Image Processing (ICIP), 2017.
  24. X. Jiang, M. Le Pendu and C. Guillemot, "Light field compression using depth image based view synthesis," 2017 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2017.
  25. X. Huang, P. An, L. Shen and R. Ma, "Efficient Light Field Images Compression Method Based on Depth Estimation and Optimization," IEEE Access, vol. 6, pp. 48984-48993, 2018.
  26. M. Le Pendu, C. Guillemot and A. Smolin, "A Fourier Disparity Layer Representation for Light Fields," IEEE Transactions on Image Processing, vol. 28, no. 11, pp. 5740-5753, 2019.
  27. L. Li, X. Jin and T. Zhong, "Imaging-Correlated Intra Prediction for Plenoptic 2.0 Video Coding," 2020 IEEE International Conference on Multimedia and Expo (ICME), 2020.
  28. G. Wang, W. Xiang, M. Pickering and C. Chen, "Light Field Multi-View Video Coding with Two-Directional Parallel Inter-View Prediction," IEEE Transactions on Image Processing, vol. 25, no. 11, pp. 5104-5117, 2016.
  29. J. Khoury, M. Pourazad and P. Nasiopoulos, "A New Prediction Structure for Efficient MV-HEVC based Light Field Video Compression," 2019 International Conference on Computing, Networking and Communications (ICNC), 2019.
  30. N. Mehajabin, S. Luo, H. Wei Yu, J. Khoury, J. Kaur and M. Pourazad, "An Efficient Random Access Light Field Video Compression Utilizing Diagonal Inter-View Prediction," 2019 IEEE International Conference on Image Processing (ICIP), 2019.
  31. W. Tu, X. Jin, L. Li, C. Yan, Y. Sun, M. Xiao, W. Han and J. Zhang, "Efficient Content Adaptive Plenoptic Video Coding," IEEE Access, vol. 8, pp. 5797-5804, 2020.
  32. T. Nguyen Huu, V. Van Duong and B. Jeon, "Random-access-aware Light Field Video Coding using Tree Pruning Method," 2020 IEEE International Conference on Visual Communications and Image Processing (VCIP), 2020.
  33. R. Sjoberg, Y. Chen, A. Fujibayashi, M. Hannuksela, J. Samuelsson, T. Tan, Y. Wang and S. Wenger, "Overview of HEVC High-Level Syntax and Reference Picture Management," IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1858-1870, 2012.
  34. G. Tech, Y. Chen, K. Müller, J. Ohm, A. Vetro and Y. Wang, "Overview of the Multiview and 3D Extensions of High Efficiency Video Coding," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 1, pp. 35-49, Jan. 2016.
  35. L. Li and X. Jin, "Macropixel-constrained Collocated Position Search for Plenoptic Video Coding," 2019 IEEE Visual Communications and Image Processing (VCIP), 2019.
  36. N. Purnachand, L. N. Alves and A. Navarro, "Improvements to TZ search motion estimation algorithm for multiview video coding," 2012 19th International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 388-391, 2012.
  37. R. Verhack, T. Sikora, G. Van Wallendael and P. Lambert, "Steered Mixture-of-Experts for Light Field Images and Video: Representation and Coding," IEEE Transactions on Multimedia, vol. 22, no. 3, pp. 579-593, 2020.
  38. R. Gray, "Vector quantization," IEEE ASSP Magazine, vol. 1, no. 2, pp. 4-29, 1984.
  39. T. Ebrahimi, S. Foessel, F. Pereira and P. Schelkens, "JPEG Pleno: Toward an Efficient Representation of Visual Reality," IEEE MultiMedia, vol. 23, no. 4, pp. 14-20, 2016.
  40. M. ŘeřAEbek, T. Bruylants, T. Ebrahimi, F. Pereira, and P. Schelkens, ''ICME 2016 grand challenge: Light field image compression-Call for proposals and evaluation procedure,'' Seattle, WA, USA, Tech. Rep., 2016.
  41. I. Viola and T. Ebrahimi, "Quality Assessment of Compression Solutions for Icip 2017 Grand Challenge on Light Field Image Coding," 2018 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2018.
  42. G. Lafruit, D. Bonatto, C. Tulvan, M. Preda and L. Yu, "Understanding MPEG-I Coding Standardization in Immersive VR/AR Applications," SMPTE Motion Imaging Journal, vol. 128, no. 10, pp. 33-39, 2019.
  43. MPEG-I, ''MPEG-I Visual activities on 6DoF and light fields,'' ISO/IEC JTC1/SC29/WG11, Macao, China, N17285, 2017.
  44. M. Teratani, X. Jin, L. Li, G. Lafruit, and L. Yu, "Exploration experiments and common test conditions for dense light fields," ISO/IEC JTC1/SC29/WG11, Marrakesh, Morocco, N18173, 2019.
  45. O. Johannsen, A. Sulc and B. Goldluecke, "On Linear Structure from Motion for Light Field Cameras", 2015 IEEE International Conference on Computer Vision (ICCV), 2015.
  46. Q. Zhang, Q. Wang, H. Li and J. Yu, "Ray-Space Epipolar Geometry for Light Field Cameras", IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2020.
  47. P. David, M. Le Pendu and C. Guillemot, "White lenslet image guided demosaicing for plenoptic cameras," 2017 IEEE 19th International Workshop on Multimedia Signal Processing (MMSP), 2017.
  48. A. Vieira, H. Duarte, C. Perra, L. Tavora and P. Assuncao, "Data formats for high efficiency coding of Lytro-Illum light fields," 2015 International Conference on Image Processing Theory, Tools and Applications (IPTA), 2015.
  49. I. Viola, M. Řeřábek and T. Ebrahimi, "Comparison and Evaluation of Light Field Image Coding Approaches," 2017 IEEE Journal of Selected Topics in Signal Processing, vol. 11, no. 7, pp. 1092-1106, 2017.
  50. C. Conti, L. D. Soares and P. Nunes, "Dense Light Field Coding: A Survey," IEEE Access, vol. 8, pp. 49244-49284, 2020.
  51. X. Jin, K. Tong, T. Fujii and M. Teratani, "[DLF] Applications of Lenslet Format for Dense Light Field," ISO/IEC JTC 1/SC 29/WG 4 M56789, Online, 2021.
  52. M. Martínez-Corral and B. Javidi, "Fundamentals of 3D imaging and displays: a tutorial on integral imaging, light-field, and plenoptic systems," Advances in Optics and Photonics, vol. 10, no. 3, p. 512, 2018.
  53. Xin Jin et al., "[DLF] Summary of Evidence in Lenslet Coding," ISO/IEC JTC 1/SC 29/WG 4 M56790, Online, 2021.
  54. R. C. Jain, R. Kasturi, and B. G. Schunck, "Chap. 9. Shading," Machine vision, New York: McGraw-Hill, 1995.
  55. Y. Bok, H. Jeon and I. S. Kweon, "Geometric Calibration of Micro-Lens- Based Light Field Cameras Using Line Features," 2017 IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 2, pp. 287-300, 2017.
  56. S. Vedula, P. Rander, R. Collins and T. Kanade, "Three-dimensional scene flow", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp. 475-480, 2005.
  57. T. Basha, Y. Moses and N. Kiryati, "Multi-view scene flow estimation: A view centered variational approach," 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1506-1513, 2010.
  58. P. David, M. Le Pendu and C. Guillemot, "Scene Flow Estimation from Sparse Light Fields Using a Local 4D Affine Model," in IEEE Transactions on Computational Imaging, vol. 6, pp. 791-805, 2020.
  59. S. Ma, B. Smith and M. Gupta, "Differential Scene Flow from Light Field Gradients", International Journal of Computer Vision, vol. 128, no. 3, pp. 679-697, 2019.
  60. M. Herzberger, "Gaussian Optics and Gaussian Brackets* †," Journal of the Optical Society of America, vol. 33, no. 12, p. 651, 1943.
  61. B. Girod, "Motion-compensating prediction with fractional-pel accuracy," in IEEE Transactions on Communications, vol. 41, no. 4, pp. 604-612, 1993.
  62. W. Lu and A. Antoniou, Two-dimensional digital filters, 1st ed. CRC Press, 1992.
  63. K. Ugur et al., "Motion Compensated Prediction and Interpolation Filter Design in H.265/HEVC," IEEE Journal of Selected Topics in Signal Processing, vol. 7, no. 6, pp. 946-956, 2013.
  64. G. J. Sullivan, J. Ohm, W. Han and T. Wiegand, "Overview of the High Efficiency Video Coding (HEVC) Standard," IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649- 1668, 2012.
  65. J. Xu, R. Joshi and R. A. Cohen, "Overview of the Emerging HEVC Screen Content Coding Extension," IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 1, pp. 50-62, 2016.
  66. D. Dansereau, O. Pizarro and S. Williams, "Decoding, Calibration and Rectification for Lenselet-Based Plenoptic Cameras," 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013.
  67. T. Georgiev, Z. Yu, A. Lumsdaine and S. Goma, "Lytro camera technology: theory, algorithms, performance analysis," Multimedia Content and Mobile Devices, 2013.
  68. B. Horn and B. Schunck, "Determining optical flow", Artificial Intelligence, vol. 17, no. 1-3, pp. 185-203, 1981.
  69. Lucas, Bruce D., and Takeo Kanade. "An iterative image registration technique with an application to stereo vision." 1981.
  70. H. Jozawa, K. Kamikura, A. Sagata, H. Kotera and H. Watanabe, "Two- stage motion compensation using adaptive global MC and local affine MC," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 7, no. 1, pp. 75-85, Feb. 1997.
  71. B. Bross et al., "Overview of the Versatile Video Coding (VVC) Standard and its Applications," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 10, pp. 3736-3764, Oct. 2021.
  72. B. Girod, "Motion-compensating prediction with fractional-pel accuracy," in IEEE Transactions on Communications, vol. 41, no. 4, pp. 604-612, April 1993.
  73. HEVC reference software, https://vcgit.hhi.fraunhofer.de/jvet/HM.
  74. F. Bossen, "Common Test Conditions and Software Reference Configurations," JCTVC-H1100, San Jose CA, 2012.
  75. T. Wang, J. Zhu, N. Kalantari, A. Efros and R. Ramamoorthi, "Light field video capture using a learning-based hybrid imaging system," ACM Transactions on Graphics, vol. 36, no. 4, pp. 1-13, 2017.
  76. G. Bjøntegaard, "Calculation of average PSNR differences between RD-curves," ITU-T Q.6/SG16 VCEG 13th Meeting VCEG- M33, 2001.
  77. L. Condat, B. Forster-Heinlein and D. Van De Ville, "H2O: Reversible Hexagonal-Orthogonal Grid Conversion by 1-D Filtering," 2007 IEEE International Conference on Image Processing, 2007.
  78. A. Pohl, B. Cosenza, M. A. Mesa, C. C. Chi, and B. Juurlink, "An evaluation of current SIMD programming models for C++," Proceedings of the 3rd Workshop on Programming Models for SIMD/Vector Processing -WPMVP '16, 2016.

Related papers

Computationally Efficient Light Field Image Compression Using a Multiview HEVC Framework
Waqas Ahmad

IEEE Access, 2019

The acquisition of the spatial and angular information of a scene using light field (LF) technologies supplement a wide range of post-processing applications, such as scene reconstruction, refocusing, virtual view synthesis, and so forth. The additional angular information possessed by LF data increases the size of the overall data captured while offering the same spatial resolution. The main contributor to the size of captured data (i.e., angular information) contains a high correlation that is exploited by state-of-the-art video encoders by treating the LF as a pseudo video sequence (PVS). The interpretation of LF as a single PVS restricts the encoding scheme to only utilize a single-dimensional angular correlation present in the LF data. In this paper, we present an LF compression framework that efficiently exploits the spatial and angular correlation using a multiview extension of high-efficiency video coding (MV-HEVC). The input LF views are converted into multiple PVSs and are organized hierarchically. The rate-allocation scheme takes into account the assigned organization of frames and distributes quality/bits among them accordingly. Subsequently, the reference picture selection scheme prioritizes the reference frames based on the assigned quality. The proposed compression scheme is evaluated by following the common test conditions set by JPEG Pleno. The proposed scheme performs 0.75 dB better compared to state-of-the-art compression schemes and 2.5 dB better compared to the x265-based JPEG Pleno anchor scheme. Moreover, an optimized motionsearch scheme is proposed in the framework that reduces the computational complexity (in terms of the sum of absolute difference [SAD] computations) of motion estimation by up to 87% with a negligible loss in visual quality (approximately 0.05 dB). INDEX TERMS Compression, light field, MV-HEVC, plenoptic.

View PDFchevron_right
Raw Plenoptic Video Coding Under Hexagonal Lattice Resolution of Motion Vectors
Vinh Đinh văn

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2022

In raw plenoptic video, the optimal motion searching points mostly follow the hexagonal structure of micro-images. Based on this understanding, we propose a new motion vector resolution, namely the hexagonal lattice (HL) resolution which reflects micro-image structure. The HL resolution can be efficiently represented by HL basis. A study in this paper shows that motion vectors are highly concentrated at hexagonal lattice points, leading to use of the proposed resolution in the context of video compression. In this regard, we demonstrate the compression benefit brought by estimating motion vectors at HL resolution in the VVC codec.

View PDFchevron_right
Light-field video coding using geometry-based disparity compensation
Paulo Nunes

2014 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), 2014

One of the main challenges in 3D light-field imaging approaches lies in the massive amount of visual information involved in providing 3D content with sufficient resolution. Consequently, adequate coding tools are essential for efficient transmission and storage of this type of content. In this context, this paper presents and evaluates two coding solutions based on the High Efficiency Video Coding (HEVC) scheme and for efficient compression of the 3D light-field content. These two coding schemes aim to exploit the 3D geometry-based disparity information in the 3D light field content and replace the block-based disparity estimation. In the first scheme, the disparity map of each view is used to directly derive the vectors for compensation, and in the second scheme these disparity vectors (for all views) are calculated (for non-occluded areas) from the disparity map of the base view. A comparative study of these proposed coding schemes is performed and future research directions are also discussed.

View PDFchevron_right
Light Field Image Coding Using High-Order Intrablock Prediction
Paulo Nunes

IEEE Journal of Selected Topics in Signal Processing, 2017

This paper proposes a two-stage high order intra block prediction method for light field image coding. This method exploits the spatial redundancy in lenslet light field images by predicting each image block, through a geometric transformation applied to a region of the causal encoded area. Light field images comprise an array of micro-images that are related by complex geometric transformations that cannot be efficiently compensated by state-of-the-art image coding techniques, which are usually based on low order translational prediction models. The two-stage nature of the proposed method allows to choose the order of the prediction model most suitable for each block, ranging from pure translations to projective or bilinear transformations, optimized according to an appropriate rate-distortion criterion. The proposed higher order intra block prediction approach was integrated into an HEVC codec and evaluated for both unfocused and focused light field camera models, using different resolutions and microlens arrays. Experimental results show consistent bitrate savings, which can go up to 12.62%, when compared to a lower order intra block prediction solution and 49.82% when compared to HEVC still picture coding.

View PDFchevron_right
Light field HEVC-based image coding using locally linear embedding and self-similarity compensated prediction
Sergio Faria

2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), 2016

Light field imaging is a promising new technology that allows the user not only to change the focus and perspective after taking a picture, as well as to generate 3D content, among other applications. However, light field images are characterized by large amounts of data and there is a lack of coding tools to efficiently encode this type of content. Therefore, this paper proposes the addition of two new prediction tools to the HEVC framework, to improve its coding efficiency. The first tool is based on the local linear embedding-based prediction and the second one is based on the self-similarity compensated prediction. Experimental results show improvements over JPEG and HEVC in terms of average bitrate savings of-71.44% and-31.87%, and average PSNR gains of 4.73dB and 0.89dB, respectively.

View PDFchevron_right

Related topics

  • Computer Science
  • Artificial Intelligence
  • Computer Vision
  • Motion compensation
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved