Chongfan Technology

News

All Categories >

Home page

2025

Light | Real-time holographic camera for high-fidelity hologram acquisition

Author:

Introduction

Holographic 3D display technology has been a research hotspot in recent years, attracting the attention of many researchers due to its features of no crosstalk and no visual fatigue. Currently, most reports on holographic 3D display technology focus on virtual 3D scenes, because the acquisition of real 3D scenes is difficult and slow. In addition, holographic 3D display relies on coherent light illumination, inevitably introducing speckle noise during the reconstruction process. Currently, real-time generation and high-fidelity reconstruction of holograms of real 3D scenes are still difficult to achieve.

Recently, Professor Wang Qionghua's team at Beihang University and Academician Zhuang Songlin and Professor Zhang Dawei's team at Shanghai University of Technology collaborated to propose a novel holographic camera for real-time acquisition of high-fidelity holograms of real 3D scenes. The acquisition end of this holographic camera uses a liquid camera equipped with an elastic membrane liquid lens, and the computation end uses a focus stacking network (FS-Net). This work achieves real-time incoherent acquisition and real-time high-fidelity hologram computation of real 3D scenes. The developed holographic camera breaks through the bottlenecks of existing holographic 3D display technologies, including the difficulties in acquiring real 3D scenes, low quality of holographic reconstructed images, and incorrect defocus blur. Real-time holographic camera for obtaining real 3D scene hologram was published in Light: Science & Applications 。

Holographic 3D display is considered an ideal 3D display method. However, since shooting real 3D scenes and encoding holograms require a lot of time, real-time display of real 3D scenes is still not achievable. Although researchers have used electrowetting liquid lenses to achieve fast shooting of real 3D scenes, the zoom response time of electrowetting liquid lenses is about 100 milliseconds, which is difficult to meet the needs of real-time shooting of real 3D scenes. In addition, the computation time of holograms increases significantly with the increase in data volume. Although in recent years, researchers have proposed deep learning-based hologram computation methods, which effectively accelerate the computation speed of holograms and improve the quality of holographic 3D display. However, the defocus blur of these methods' holographic reconstructed images often does not match that of the real 3D scene. How to achieve real-time shooting of real 3D scenes and high-fidelity holographic reconstruction is a problem to be solved in the field of holographic display.

To address the above problems, the researchers proposed a holographic camera for real-time acquisition of high-fidelity holograms of real 3D scenes. As Figure 1 shows, at the acquisition end of the holographic camera, the researchers designed a liquid camera based on an elastic membrane liquid lens. Driven by a voice coil motor, the liquid camera can perform fast zoom shooting within 15 milliseconds. At the computation end, the researchers proposed a real-time hologram generation method based on FS-Net. To solve the problem of mismatch between the blur of the holographic reconstructed image and the blur of the real 3D scene, an aberration compensation method based on Zernike polynomials was used in FS-Net. After the focus stack captured by the liquid camera is input into FS-Net, the time for FS-Net to compute the hologram is only 13 milliseconds. Holographic reconstruction experimental results show that the peak signal-to-noise ratio of the holographic reconstructed image reaches 40dB, and the time from shooting the real 3D scene to holographic 3D display is only 28 milliseconds. The proposed holographic camera achieves real-time acquisition and high-fidelity holographic reconstruction of real 3D scenes.

Figure 1: Conceptual diagram of the holographic camera

The core component in the holographic camera is the elastic membrane liquid lens, as Figure 2a shows. The elastic membrane liquid lens mainly consists of a housing, two glass windows, a pressure plate, a liquid cavity, a magnet, a coil, an elastic membrane, a filling liquid, and several vents. In order to make the response speed of the elastic membrane liquid lens fast enough, the researchers used high-refractive-index filling liquid materials and special elastic membrane control components to shorten the zoom response time of the elastic membrane liquid lens to 15 milliseconds. When different driving voltages are applied, the shape of the elastic membrane liquid lens will change under the action of Ampere force. Figure 2b and Figure 2c are schematic diagrams of the elastic membrane liquid lens in the convergent and divergent states, respectively.

Figure 2: Schematic diagram of the structure and effect of the elastic membrane liquid lens: a Structure of the elastic membrane liquid lens; b Effect of the elastic membrane liquid lens in the convergent state; c Effect of the elastic membrane liquid lens in the divergent state

The flowchart of FS-Net training and hologram generation is shown in Figure 3. Unlike traditional deep learning-based hologram computation methods, FS-Net takes the focus stack of a real 3D scene as input and uses a focus stack renderer and learnable Zernike polynomials during hologram training. As Figure 3a shows, first, the focus stack renderer is used to process the all-focus image and depth map of the real 3D scene into a focus stack. Then, the untrained FS-Net outputs the complex amplitude distribution of the focus stack. Then, the complex amplitude distribution is back-propagated using the physical diffraction model to obtain a multi-layer holographic reconstructed image. Subsequently, multiple binary masks are calculated according to the number of layers and the depth map, and the inner product of the binary masks and the focus stack and the multi-layer holographic reconstructed images are calculated respectively to obtain the target focused and defocused images and the reconstructed focused and defocused images. Finally, the loss function between the target focused and defocused images and the reconstructed focused and defocused images is calculated, and the learnable Zernike polynomials and the parameters in FS-Net are optimized. The above steps are repeated, and when the network iterates to a fixed number of times, the training of FS-Net stops. Figure 3b is the flowchart of the trained FS-Net generating holograms. The focus stack captured by the liquid camera is input into the trained FS-Net, and the complex amplitude distribution of the focus stack is output. After using Zernike phase compensation for the complex amplitude distribution, the high-fidelity hologram of the focus stack can be obtained using the double-phase encoding method.