01

2024

-

08

Deep Learning Optical Design and Computational Optical Imaging

Author:


In exploring the infinite possibilities of optical imaging technology, deep learning computational optical imaging is gradually becoming a hot spot in scientific research and application. Although the traditional optical imaging technology is mature, its imaging quality is limited by physical principles, such as diffraction limit and aberration, and it is difficult to further break through. The rise of computational imaging technology, combined with the knowledge of mathematics and signal processing, has opened up a new way for optical imaging. As a rapidly developing technology in recent years, deep learning has injected new vitality into computational optical imaging with its powerful data processing and feature extraction capabilities.

Deep learning computational optical imaging has a deep research background, which aims to solve the problems in traditional optical imaging through algorithm optimization and improve imaging quality. This field combines the knowledge of optics, computer science, mathematics and other disciplines, and uses deep learning models to obtain, encode and process light field information in multiple dimensions, thus breaking through the limitations of traditional imaging.

Looking to the future, deep learning computational optical imaging has a bright future. It can not only further improve the imaging resolution, reduce noise, and achieve super-resolution imaging, but also simplify the hardware equipment of the imaging system through algorithm optimization and reduce the cost. At the same time, its strong environmental adaptability will enable the imaging system to maintain stable performance in various complex environments, and provide strong support for medical, unmanned, remote sensing monitoring and other fields. With the deepening of interdisciplinary integration and the continuous advancement of technology, we have reason to believe that deep learning computational optical imaging will play a more important role in the future and lead a new round of revolution in imaging technology.

01 Deep Learning Optical Design (Optical Network and Reverse Design)

02 Deep Learning Computational Optical Imaging Topics

Introduction to Optical Design Lecturer in Deep Learning

The lecturer team comes from the national key university and the national "985 Project" physics and information interdisciplinary major, with many years of machine learning and research experience! His research direction involves the intersection of optical design and physics, deep learning, machine learning and so on. Has a wealth of knowledge and practical experience. Participated in a number of projects of the National Natural Science Foundation, including the publication of more than ten SCI papers and a national invention patent! He has served as a reviewer for several journals such as MDPI.

Lecturer Introduction to Deep Learning Computational Optical Imaging

The lecturer is from a top foreign university in optical imaging, and is good at computer vision and deep learning imaging research. In recent years, 15 SCI papers have been published and three invention patents have been granted. His research interests include image processing and computer vision, deep learning methods, physics-driven optical imaging, and cross-modal imaging.

Deep Learning Optical Design (Optical Networks and Reverse Design) Goals

1. The training goal of optical network based on deep learning is mainly to cultivate senior professionals with modern optical theoretical foundation and deep learning technology knowledge. They not only need to be familiar with the principles of modern optics, but also need to master the principles and applications of deep learning algorithms, and be able to combine deep learning and modern optical principles to design devices with optical acceleration functions.

2. Grasp the tools needed to build a deep learning model and learn to build a deep learning development environment. Let beginners use the deep learning framework to build common neural network models, understand the problems in the model training process and master common solutions.

3. Be familiar with the development status of metamaterials, basically master the multi-physics simulation software, and be able to use the software to calculate the energy band of photonic crystals and post-process the simulation results. Understand the application of super surface in optical and quantum fields, learn to use simulation software to simulate the structure of super surface and analyze the results.

4. Know how MATLAB interacts with COMSOL and Python, learn how to use Python to process COMSOL exported data, and how to use MATLAB to export COMSOL data and process it into data that Python can read.

5. Understand the development status of silicon-based optical networks, know the principles of matrix decomposition, learn to use the deep learning framework to build an MZI-based model framework and apply it to deep learning examples.

6. In the future, optical chips with acceleration function can be designed based on on-chip optical networks using the acceleration function of light. Based on the diffraction network, a fast imaging system can be designed in free space to accelerate the image recognition of automatic driving.

7. Using the deep learning model, the disadvantages of the traditional design method based on full-wave simulation can be overcome, the target response of a given structure can be quickly given, and the process of optical design can be accelerated.

Deep learning to compute optical imaging targets

1. Master the typical optical imaging mechanism, understand its corresponding mathematical model and the problem of demand solution.

2. Master the typical optimization theory and method, and be able to solve the typical computational imaging inverse problem by designing the objective function.

3. Master the principles and applications of deep learning algorithms, be able to deploy and modify typical deep network models through python programming, and be used to solve typical computational optical imaging problems.

Optical Design with Deep Learning

First day

Chapter 1 Introduction

Section 1 Overview of Deep Learning and Optical Networks

1.1 diffraction neural network

1.2 on-chip integrated optical neural network

The second section of deep learning and super surface reverse design overview.

Section 3 Challenges of Optical Networks and Inverse Design of Hypersurfaces

Section 4 Future Development Trend of Optical Network and Hypersurface Reverse Design

Chapter 2 Software Basics (Practical Operation)

The first section of Python environment construction

1.1 Anaconda 、Numpy、Matplotlib 和 Pandas 安装 

1.2 Virtual environment construction and Pytorch installation

Installation of 1.3 Pytorch GPU versions

Section 2 Basic Python Tutorial

2.1 common data structures and data types in Python

2.2 Numpy Basic Tutorial

2.3 Pandas Basic Tutorial

2.4 Matplotlib Basic Tutorial

Section III Basic Pytorch Tutorial

3.1 data operations

3.2 data preprocessing

3.3 Linear Algebra

Picture

The next day

Chapter 3 Deep Learning

Section 1 Machine Learning

Section II. Key components in machine learning

2.1 data

2.2 model

2.3 loss function

2.4 optimization algorithm

Section 3 Classification of Machine Learning

3.1 supervised learning

3.2 unsupervised learning

3.3 semi-supervised learning

3.4 reinforcement learning

3.5 Transfer Learning

Section 4 Deep Learning

The Development of 4.1 Deep Learning

Advances in 4.2 Deep Learning

4.3 artificial neural network

Chapter 4 Deep Learning Model (Practical)

The first section of the linear neural network example.

1.1 linear regression

1.2 softmax return

Section II. Examples of multilayer perceptrons

2.1 multilayer perceptron

2.2 model selection, under-fitting, and over-fitting

2.3 weight attenuation

2.4 Dropout 

Section 3 Examples of Convolutional Neural Networks

3.1 from fully connected layers to convolution

3.2 Channel and Convergence Layer

3.3 Convolutional Neural Network (LeNet)

3.4 batch normalization

3.5 residual connection

Section 4 Examples of recurrent neural networks.

4.1 series model

4.2 language models and datasets

4.3 recurrent neural network

Section 5 Generating Examples of Adversarial Networks

5.1 probability generation model

5.2 variational autoencoder

5.3 Generation Adversarial Network

Picture Picture

Third day

Chapter 5 Metamaterials

Section I Overview of Metamaterials

Section 2 Photonic Crystals (COMSOL Practical Operation)

Fundamentals and Applications of 2.1 Photonic Crystals

2.2 transfer matrix method for solving one-dimensional photonic crystal energy bands

2.3 plane wave expansion method for solving one-dimensional photonic crystal energy band

2.4 Finite Element Method for Solving Photonic Crystal Energy Band

2.4.1 Two-dimensional square lattice energy band

2.4.2 Two-dimensional square lattice photonic crystal plate energy band

2.4.3 Two-dimensional triangular lattice photonic crystal plate energy band

2.4.4 Two-dimensional hexagonal lattice photonic crystal plate energy band

Calculation of Continuum Bound States (BICs) and Their Topological Charge in 2.5 Photonic Crystal Plates

Section 3 The Role of Hypersurfaces in the Regulation of Light Field

3.1 phase regulation

3.2 light intensity regulation

3.3 polarization regulation

3.4 frequency regulation

3.5 joint regulation

Section 4 Hypersurface Simulation Example (COMSOL in Practice)

3.1 Frequency Selective Surface Periodic Complementary Open Resonant Ring

3.2 metasurface beam deflectors

Section 5 Research and Application of Supersurface in Quantum Optics

5.1 quantum plasmon

5.2 quantum light source

Measurement and manipulation of 5.3 quantum states

Applications of 5.4 Quantum Optics

The fourth day

Chapter 6 Optical Calculation Based on Mach-Zehnder Interferometer

Section 1 Introduction to Optical Computing and Optical Neural Networks

Introduction to the Background of 1.1 Light Computing

Development and Classification of 1.2 Optical Neural Networks

Research Status of 1.3 Optical Neural Network

The second section is based on the principle of MZI's optical neural network.

2.1 fully connected neural network principle explanation

Calculation Principle of Coherent Optical Matrix 2.2 on MZI Cascade

2.3 N-order unitary matrix factorization

2.4 Universal Architecture of Unitary Matrix Based on MZI Topology Cascade

Section III Acquisition and Processing of Training Data Sets (Python Practical Operation)

3.1 Python Program Environment Installation

Introduction to the main functions of the 3.2 Pycharm

The method of obtaining the 3.3 data set.

Pre-processing of 3.4 training data sets

Section 4 Construction of Universal Architecture of Unitary Matrix (Python Practical Operation)

Construction of 4.1 Second Order Unitary Matrix

Construction of 4.2 clement architecture

Section 5 Model Running of Optical Neural Networks (Python Practice)

Day Five

Chapter 7 All-Optical Diffraction Neural Networks

Section 1 Fundamentals of Scalar Diffraction Theory

1.1 Huygens-Fresnel principle

1.2 Rayleigh-Somofy diffraction formula

Angular spectrum theory of 1.3 diffraction

1.4 discrete Fourier transform

Section 6 Optical Diffraction Neural Networks (Python Practical)

2.1 artificial neural network structure

2.2 optical diffraction neural network structure

Handwritten Digit Recognition 2.3 Optical Diffraction Neural Network

Application of 2.4 Optical Diffraction Neural Network

Chapter 8 Examples of Metamaterial Reverse Design

The first section is based on neural network methods to implement the design of all-media supersurfaces (COMSOL practice)

Simulation of 1.1 Hypersurface Element

Parameter Extraction of 1.2 Hypersurface Element

Construction of 1.3 training data set

Training of 1.4 prediction models

Section 2 The combination of CNN and RNN to find the optical properties of the plasma structure (COMSOL practice)

Section III The DELAY intensive learning algorithm implements the automatic mode-locking control of the laser.

Deep Learning Topics in Computational Optical Imaging

The first day:

Chapter 1: Fundamentals of Optical Imaging

Section 1: Introduction

1. What is optical imaging?

2. Advances in optical imaging

Section 2: Important Properties of Optical Imaging

1. Product of object distance, focal length and spatial bandwidth

2. Resolution, field of view, depth of field

3. Spherical aberration, coma, field curvature, distortion, chromatic aberration, aberration

4. Point spread function, modulation transfer function.

Section III: Imaging quality evaluation index

1. Full reference evaluation

2. Semi-reference evaluation

3. No reference evaluation

Section 4: Development Trend of Optical Imaging

1. Function expansion (phase, three-dimensional, non-line-of-sight, through cloud and fog, remote sensing)

2. Performance improvement (field size, resolution, imaging speed)

3. System optimization (miniaturization, cheap, efficient manufacturing)

Chapter 2: Typical Computational Imaging

Section 1: Computed Tomography (CT) Imaging

1. Fundamentals (X-ray projection and detection)

2. Computational reconstruction based on deep learning

3. Technological progress and application

Section 2: Compressed Sensing Imaging

1. Sparse Representation and Measurement

2. Typical reconstruction algorithms and optimization

3. Compressed sensing imaging calculation reconstruction.

4. Application and development trend

Section III: Coded Aperture Imaging

1. Coded aperture design principle

2. Imaging system characteristics and advantages

3. Image Reconstruction Based on Deep Learning

4. Typical Application Overview

Chongqing Chongfan Technology Co., Ltd. Optical platform film thickness meter OCT optical coherence tomography LabVIEW control