Deep learning simplifies generation of 3D holograms

Holograms that offer a three-dimensional (3D) view of objects provide a level of detail that is unavailable in regular two-dimensional (2D) images and as a result, they hold potential for use in various fields, including medical imaging, manufacturing and virtual reality. Holograms are traditionally constructed by recording the three-dimensional data of an object and the interactions of light with the object. However, this technique is computationally intensive as it requires the use of a special camera to capture the 3D images. In recent times, many deep-learning methods have been proposed for generating holograms, by using the 3D data captured using RGB-D cameras that capture the colour and depth information of an object.

Now, a team of researchers led by Professor Tomoyoshi Shimobaba from Chiba University has proposed a novel approach based on deep learning that streamlines hologram generation by producing 3D images directly from regular 2D colour images captured using ordinary cameras. Explaining the rationale behind this study, Shimobaba said there are several problems in realising holographic displays, such as the acquisition of 3D data, the computational cost of holograms, and the transformation of hologram images to match the characteristics of a holographic display device. “We undertook this study because we believe that deep learning has developed rapidly in recent years and has the potential to solve these problems,” Shimobaba said.

The new approach uses three deep neural networks (DNNs) to transform a regular 2D colour image into data that can be used to display a 3D scene or object as a hologram. The first DNN makes use of a colour image captured using a regular camera as the input and then predicts the associated depth map, providing information about the 3D structure of the image. Both the original RGB image and the depth map created by the first DNN are then used by the second DNN to generate a hologram. Then the third DNN refines the hologram, making it suitable for display on different devices.

The researchers found that the time taken by the proposed approach to process data and generate a hologram was reportedly better than that of a graphics processing unit. “Another noteworthy benefit of our approach is that the reproduced image of the final hologram can represent a natural 3D reproduced image. Moreover, since depth information is not used during hologram generation, this approach is inexpensive and does not require 3D imaging devices such as RGB-D cameras after training,” Shimobaba said.

This approach could find potential applications in heads-up and head-mounted displays for generating high-fidelity 3D displays. The research findings were published in Optics and Lasers in Engineering.

Image caption: A three-dimensional hologram. Researchers propose a computationally inexpensive deep learning approach that utilises three neural networks to transform two-dimensional images captured using regular cameras into three-dimensional holograms. Image credit: Kunal Mukherjee.