Nokia's volumetric video coding team is researching ways to improve the rendering quality of MPEG Immersive Video (3DOF+ and 6DOF) applications. Correct illumination of the virtual/augmented content depending on the scene surrounding the consumer provides potential for improvement.
In this master thesis, the student creates an application that can render augmented objects on a handheld device, using the "digital twin" of the surrounding room to improve illumination of the virtual object. The digital twin, recorded with a 3D mapping device, is a 3D point-cloud of the surrounding room. This point-cloud allows precise illumination estimation from 3D structure, windows, and more. Exploiting this data, the student develops algorithms to estimate the illumination of the room and metrics to objectively measure quality of illumination estimation and the illumination quality of a virtual object.
Dr. Christoph Bachhuber (Nokia – DE/Munich)