Revolutionizing Telepresence: Seamless Life-Sized Holographic Imaging in Real-Time

Abstract

Telepresence has emerged as a compelling research area driven by its potential to reduce travel costs, expedite communication, and enhance collaborative efforts. Among the latest developments in this field is mixed-reality telepresence, which holds significant promise. However, it has the formidable challenge of achieving strong, lifelike, and rapid 3D object reconstruction. Many current approaches emphasize real-time applications in the realm of cutting-edge technologies, particularly in the realm of immersive reality. However, these approaches often prioritize the synthesis of intermediary perspectives for specific viewing angles, rather than the comprehensive generation of complete 3D models. Introducing our ground-breaking system, aptly named "Holographic," we present a revolutionary end-to-end solution tailored for augmented and virtual reality telepresence. Our system seamlessly demonstrates the instantaneous creation of top-tier 3D models encompassing entire environments, including individuals and objects, using a sophisticated array of state-of-the-art depth cameras. What sets our innovation apart is the ability to transmit these highly detailed 3D models to remote users in real time. This enables individuals to wear virtual or augmented reality headsets to perceive, hear, and interact with remote participants in a three-dimensional space, creating an experience akin to a physical presence in the same location. This paradigm shift in audio-visual communication brings us closer to the immersive and authentic nature of face-to-face interaction. In this study, we provide a comprehensive and detailed overview of the holographic technical system, shedding light on its key interactive capabilities, the diverse range of potential application scenarios it unlocks, and initial findings from a qualitative study that delves into the unique communication medium it offers. Our primary focus is on delivering real-time 3D reconstruction that ensures the creation of accurate and realistic representations for users, paving the way for real-time holographic telepresence.