Holography and its applications in Augmented Reality
2025, Handbook of Visual Display Technology
https://doi.org/10.1007/978-3-642-35947-7_245-1Abstract
Augmented Reality (AR) is a method of combining computer-generated virtual content with the real world. This augmentation is often achieved with the aid of a head-mounted device that the user looks through. Computer-Generated Holography (CGH) is a way of shaping light waves to form / replay three-dimensional content. Unlike currently widespread stereoscopic 3D displays, holography is capable of reconstructing objects showcasing real depth information, leading to a superior viewing experience. This work explores various aspects of CGH within the context of optical seethrough near-eye displays suited for AR applications. Although Holographic AR displays only appeared in the 2010s, they have benefited from decades of research into Computer-Generated Holography for projection and display applications. The particular focus of this work is the hardware of Holographic AR devices. Listed and examined are all the physical and optical components of an AR system, namely: the light source and its coherence requirement, light source conditioning, spatial light modulator (SLM), SLM illumination, relay / magnification optics, and combiner optics. In each section, trade-offs which are necessary to navigate while building holographic AR systems are identified and analyzed. Suggestions on how to optimize the overall projector layout are offered throughout the text.
References (83)
- Abeeluck, A. K., A. Iverson, H. Goetz and E. Passon (2018). 58-2: Invited Paper: High-Performance Displays for Wearable and HUD Applications. SID Symposium Digest of Technical Papers, Wiley Online Library.
- Adams, C. S. and I. G. Hughes (2018). Optics f2f: from Fourier to Fresnel. Oxford, United Kingdom, Oxford University Press.
- Apple (2023). Introducing Apple Vision Pro: Apple's first spatial computer. Introducing the era of spatial computing with Apple Vision Pro. Apple Newsroom.
- Apter, B., U. Efron and E. Bahat-Treidel (2004). "On the fringing-field effect in liquid-crystal beam-steering devices." Applied optics 43(1): 11-19.
- Banks, L., M. Birch, D. Krueerke, E. Buckley, A. Cable, N. Lawrence and P. Mash (2006). 73.4: Real-time Diffractive Video Projector Employing Ferroelectric LCOS SLM. SID Symposium Digest of Technical Papers, Wiley Online Library.
- Bartlett, T., W. McDonald and J. Hall (2019). "Adapting Texas Instruments DLP technology to demonstrate a phase spatial light modulator." SPIE OPTO 10932.
- Bartlett, T., W. McDonald, J. Hall, P. Oden, D. Doane, R. Ketchum and T. Byrum (2021). "Recent advances in the development of the Texas Instruments phase-only microelectromechanical systems (MEMS) spatial light modulator." SPIE OPTO 11698. Benton, S. A. and V. M. Bove Jr (2008). Holographic imaging. Hoboken, NJ, USA, John Wiley & Sons.
- Buckley, E. (2011). "Computer-generated phase-only holograms for real-time image display." Advanced Holography-Metrology and Imaging: 277-304.
- Cartamil-Bueno, S. J., A. Centeno and A. Zurutuza (2020). "Graphene Interferometric Modulator Displays-GIMOD Project." Public deliverable for the ATTRACT Final Conference.
- Chen, J.-S. and D. Chu (2015). "Improved layer-based method for rapid hologram generation and real-time interactive holographic display applications." Optics express 23(14): 18143-18155.
- Chen, J., S. M. Morris, T. D. Wilkinson, J. P. Freeman and H. J. Coles (2009). "High speed liquid crystal over silicon display based on the flexoelectro-optic effect." Optics Express 17(9): 7130-7137.
- Choi, M.-H., Y.-G. Ju and J.-H. Park (2020). "Holographic near-eye display with continuously expanded eyebox using two-dimensional replication and angular spectrum wrapping." Optics Express 28(1): 533-547.
- Choi, S., M. Gopakumar, Y. Peng, J. Kim, M. O'Toole and G. Wetzstein (2022). Time-multiplexed neural holography: a flexible framework for holographic near- eye displays with fast heavily-quantized spatial light modulators. ACM SIGGRAPH 2022 Conference Proceedings.
- Choi, S., M. Gopakumar, Y. Peng, J. Kim and G. Wetzstein (2021). "Neural 3D Holography: Learning Accurate Wave Propagation Models for 3D Holographic Virtual and Augmented Reality Displays." ACM Trans. Graph. (SIGGRAPH Asia).
- Christopher, P. J., R. Mouthaan, J. P. Freeman and T. D. Wilkinson (2019). "Improving pixel differentiation in holographic images." arXiv preprint arXiv:1912.12196.
- Coomber, S., C. Cameron, J. Hughes, D. Sheerin, C. Slinger, M. Smith and M. Stanley (2001). Optically addressed spatial light modulators for replaying computer-generated holograms, SPIE.
- Dainty, J. C. (2013). Laser speckle and related phenomena, Springer science & business Media.
- Demolder, A. (2022). "Toward the Standardization of High-Quality Computer- Generated Holography Media Production Workflow." SMPTE Motion Imaging Journal 131(1): 48-58.
- Deng, Y. and D. Chu (2017). "Coherence properties of different light sources and their effect on the image sharpness and speckle of holographic displays." Scientific Reports 7(1): 5893.
- Freeman, J. (2010). Visor projected helmet mounted display for fast jet aviators using a Fourier video projector, University of Cambridge.
- Freeman, J. P., T. D. Wilkinson and P. Wisely (2010). Visor projected HMD for fast jets using a holographic projector. Three-Dimensional Imaging, Visualization, and Display 2010 and Display Technologies and Applications for Defense, Security, and Avionics IV, SPIE.
- Gabor, D. (1948). "A new microscopic principle." Nature 161: 777-778.
- Gabor, D. (1949). "Microscopy by reconstructed wave-fronts." Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences 197(1051): 454-487.
- Gabor, D. (1971). "Nobel Lecture, December 11, 1971." Gao, H., X. Fan, W. Xiong and M. Hong (2021). "Recent advances in optical dynamic meta-holography." Opto-Electronic Advances 4(11): 210030.
- Goode, L. (2019) "Microsoft's HoloLens 2 Puts a Full-Fledged Computer on Your Face." WIRED.
- Goodman, J. W. (2005). Introduction to Fourier Optics, W. H. Freeman.
- Goodman, J. W. (2017). Introduction to Fourier Optics, W. H. Freeman.
- Greenwald, W. (2023) "Meta Quest 3 Review." PCMag.
- Hariharan, P. (2002). Basics of Holography, Cambridge University Press.
- Hayden, S. (2011). "Lenovo Reveals ThinkReality A3 Lightweight AR Glasses for Enterprise." Road to VR.
- Higashida, R., N. Funabashi, K.-i. Aoshima, M. Miura and K. Machida (2020). "Diffraction of light using high-density magneto-optical light modulator array." Optical Engineering 59(6): 064104.
- Hornbeck, L. J. (2001). "The DMDTM projection display chip: a MEMS-based technology." Mrs Bulletin 26(4): 325-327.
- Jang, C., K. Bang, M. Chae, B. Lee and D. Lanman (2022). "Waveguide holography: Towards true 3d holographic glasses." arXiv preprint arXiv:2211.02784.
- Jang, C., K. Bang, M. Chae, B. Lee and D. Lanman (2024). "Waveguide holography for 3D augmented reality glasses." Nature Communications 15(1): 66.
- Jang, C., K. Bang, G. Li and B. Lee (2018). "Holographic near-eye display with expanded eye-box." ACM Trans. Graph. 37(6): Article 195.
- Johnson, P. V., J. Kim and M. S. Banks (2014). "The visibility of color breakup and a means to reduce it." Journal of Vision 14(14): 10-10.
- Jolly, S. S. K. (2019). Holographic augmented reality: towards near-to-eye electroholography via guided wave acousto-optics, Massachusetts Institute of Technology.
- Kaczorowski, A., G. S. Gordon, A. Palani, S. Czerniawski and T. D. Wilkinson (2015). "Optimization-based adaptive optical correction for holographic projectors." Journal of Display Technology 11(7): 596-603.
- Kaczorowski, A., G. S. D. Gordon and T. D. Wilkinson (2016). "Adaptive, spatially-varying aberration correction for real-time holographic projectors." Optics Express 24(14): 15742-15756.
- Kaczorowski, A., A. Newman, A. Spiess and D. Milne (2019). An intrinsically eye safe approach to high apparent brightness augmented reality displays using digital holography. Eurodisplay 2019 : Book of abstracts of International Conference, Minsk, Society for Information Display, Belarusian State University of Informatics and Radioelectronics.
- Kelly, S. M. (2024). Apple's Vision Pro headset hits US stores today. Here's what you need to know. CNN Business, CNN.
- Ketchum, R., T. Zhang and P.-A. Blanche (2022). Camera feedback optimization of computer-generated holograms displayed by the Texas Instruments phase light modulator for AR/HUD applications, SPIE.
- Kim, H.-E., N. Kim, H. Song, H.-S. Lee and J.-H. Park (2011). Three-dimensional holographic display using active shutter for head mounted display application. Stereoscopic Displays and Applications XXII, SPIE.
- Kozacki, T. and M. Chlipala (2016). "Color holographic display with white light LED source and single phase only SLM." Optics Express 24(3): 2189-2199.
- Kress, B. C. (2020). Optical architectures for augmented-, virtual-, and mixed- reality headsets, SPIE.
- Lazarev, G., P.-J. Chen, J. Strauss, N. Fontaine and A. Forbes (2019). "Beyond the display: phase-only liquid crystal on silicon devices and their applications in photonics." Optics express 27(11): 16206-16249.
- Lazarev, G., A. Hermerschmidt, S. Krüger and S. Osten (2012). LCOS spatial light modulators: trends and applications. Optical Imaging and Metrology: Advanced Technologies, Wiley-VCH Verlag & Co. KGaA: 1-29.
- Leith, E. N. and J. Upatnieks (1962). "Reconstructed Wavefronts and Communication Theory*." Journal of the Optical Society of America 52(10): 1123-1130.
- Li, S.-Q., X. Xu, R. Maruthiyodan Veetil, V. Valuckas, R. Paniagua-Domínguez and A. I. Kuznetsov (2019). "Phase-only transmissive spatial light modulator based on tunable dielectric metasurface." Science 364(6445): 1087-1090.
- Lizana, A., L. Lobato, A. Márquez, C. Iemmi, I. Moreno, J. Campos and M. J. Yzuel (2011). "Study of liquid crystal on silicon displays for their application in digital holography." Advanced Holography-Metrology and Imaging.
- Maimone, A., A. Georgiou and J. S. Kollin (2017). "Holographic near-eye displays for virtual and augmented reality." ACM Trans. Graph. 36(4): Article 85.
- Makowski, M., I. Ducin, K. Kakarenko, J. Suszek, M. Sypek and A. Kolodziejczyk (2012). "Simple holographic projection in color." Optics express 20(22): 25130-25136.
- Mansha, S., P. Moitra, X. Xu, T. W. W. Mass, R. M. Veetil, X. Liang, S.-Q. Li, R. Paniagua-Domínguez and A. I. Kuznetsov (2022). "High resolution multispectral spatial light modulators based on tunable Fabry-Perot nanocavities." Light: Science & Applications 11(1): 141.
- Markley, E., N. Matsuda, F. Schiffers, O. Cossairt and G. Kuo (2023). Simultaneous Color Computer Generated Holography. SIGGRAPH Asia 2023 Conference Papers. Sydney, NSW, Australia, Association for Computing Machinery: Article 22.
- Moon, E., M. Kim, J. Roh, H. Kim and J. Hahn (2014). "Holographic head- mounted display with RGB light emitting diode light source." Optics Express 22(6): 6526-6534.
- Newman, A., A. Spiess, D. Milne and A. Kaczorowski (2020). Methods for laser speckle reduction in computer-generated holography (Conference Presentation), SPIE.
- Newman, A. J., T. Widjanarko, A. O. Spiess, D. F. Milne, T. D. Wilkinson and A. Kaczorowski (2021). "An intrinsically eye safe approach to high apparent brightness augmented reality displays using computer-generated holography." Journal of the Society for Information Display 29(11): 840-851.
- Newswire (2023). Swave Photonics Secures €10M Seed Funding Round to Develop Holographic Augmented Reality. PR Newswire.
- Osten, S. and A. Klauss (2021). HOLOEYE Photonics AG: HOLOEYE Spatial Light Modulators for color sequential holographic reconstruction in visual systems, SPIE.
- Ouyang, B., S. Gong, T. Lawrence and J. Hall (2022). Evaluating Texas instruments Phase Light Modulator (PLM), SPIE.
- Peng, Y., S. Choi, J. Kim and G. Wetzstein (2021). "Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration." Science Advances 7(46): eabg5040.
- Pi, D., J. Liu and Y. Wang (2022). "Review of computer-generated hologram algorithms for color dynamic holographic three-dimensional display." Light: Science & Applications 11(1): 231.
- Pivnenko, M., K. Li and D. Chu (2021). "Sub-millisecond switching of multi-level liquid crystal on silicon spatial light modulators for increased information bandwidth." Optics Express 29(16): 24614-24628.
- Ramanujan, S. (2023). "Trends in Emerging OPI Technologies: Dr Sujatha Ramanujan, Managing Director of the Luminate accelerator and investment fund in Rochester, USA, shares how new photonics tech is impacting industries including agtech, AR & VR, communications, healthcare, material science, unmanned vehicles and quantum." Electro Optics(334): 20-23.
- Robertson, A. (2022) "The Magic Leap 2 isn't a revolution, but it's a visible improvement." The Verge.
- Shi, L., F.-C. Huang, W. Lopes, W. Matusik and D. Luebke (2017). "Near-eye light field holographic rendering with spherical waves for wide field of view interactive 3D computer graphics." ACM Trans. Graph. 36(6): Article 236.
- Shiraki, A., N. Takada, M. Niwa, Y. Ichihashi, T. Shimobaba, N. Masuda and T. Ito (2009). "Simplified electroholographic color reconstruction system using graphics processing unit and liquid crystal display projector." Optics Express 17(18): 16038-16045.
- Sich, M. (2009). "Interactive Holography: Pursuit of a Dream." Computing in Science & Engineering 11(1): 62-65.
- Slinger, C., C. Cameron and M. Stanley (2005). "Computer-generated holography as a generic display technology." Computer 38(8): 46-53.
- Smalley, D. E. (2013). Holovideo on a stick: integrated optics for holographic video displays, Massachusetts Institute of Technology.
- Smalley, D. E., S. Jolly, G. E. Favalora and M. G. Moebius (2021). Status of leaky mode holography. Photonics, MDPI.
- Smalley, D. E., Q. Y. J. Smithwick, V. M. Bove, J. Barabas and S. Jolly (2013). "Anisotropic leaky-mode modulator for holographic video displays." Nature 498(7454): 313-317.
- Smith, W. J. (2000). Modern optical engineering : the design of optical systems. New York, McGraw Hill.
- St-Hilaire, P., S. A. Benton, M. E. Lucente, J. S. Underkoffler and H. Yoshikawa (1991). Real-time holographic display: Improvements using a multichannel acousto-optic modulator and holographic optical elements. Practical Holography V, SPIE.
- St-Hilaire, P., M. Lucente, J. Sutter, R. Pappu, C. Sparrell and S. Benton (1995). Scaling up the MIT holographic video system, SPIE.
- Stanley, M., M. A. Smith, A. P. Smith, P. J. Watson, S. D. Coomber, C. D. Cameron, C. W. Slinger and A. Wood (2004). 3D electronic holography display system using a 100 mega-pixel spatial light modulator. Optical Design and Engineering, SPIE.
- Sutherland, I. E. (1968). A head-mounted three dimensional display. Proceedings of the December 9-11, 1968, Fall Joint Computer Conference, Part I, Association for Computing Machinery.
- Takahashi, D. (2023) "VividQ and Dispelix create a 3D holographic tech for wearable AR." GamesBeat.
- Werner, J. (2024). "Catchup With Ivan Sutherland -Inventor Of The First AR Headset." Forbes.
- Widjanarko, T., M. El Guendy, A. Spiess, D. Sullivan, T. Durrant, O. Tastemur, A. Newman, D. Milne and A. Kaczorowski (2020). Clearing key barriers to mass adoption of augmented reality with computer-generated holography, SPIE.
- Yang, Y., A. Forbes and L. Cao (2023). "A review of liquid crystal spatial light modulators: devices and applications." Opto-Electronic Science 2(8): 230026- 230021-230026-230029.
- Yeom, H.-J., H.-J. Kim, S.-B. Kim, H. Zhang, B. Li, Y.-M. Ji, S.-H. Kim and J.- H. Park (2015). "3D holographic head mounted display using holographic optical elements with astigmatism aberration compensation." Optics express 23(25): 32025-32034.
- Zhang, Z., Z. You and D. Chu (2014). "Fundamentals of phase-only liquid crystal on silicon (LCOS) devices." Light: Science & Applications 3(10): e213-e213.