Endowed with the superior computing speed and energy efficiency, optical neural networks (ONNs) have attracted ever-growing attention in recent years. Existing optical computing architectures are mainly single-channel due to the lack of advanced optical connection and interaction operators, solving simple tasks such as hand-written digit classification, saliency detection, etc. The limited computing capacity and scalability of single-channel ONNs restrict the optical implementation of advanced machine vision. Herein, we develop Monet: a multichannel optical neural network architecture for a universal multiple-input multiple-channel optical computing based on a novel projection-interference-prediction framework where the inter- and intra- channel connections are mapped to optical interference and diffraction. In our Monet, optical interference patterns are generated by projecting and interfering the multichannel inputs in a shared domain. These patterns encoding the correspondences together with feature embeddings are iteratively produced through the projection-interference process to predict the final output optically. For the first time, Monet validates that multichannel processing properties can be optically implemented with high-efficiency, enabling real-world intelligent multichannel-processing tasks solved via optical computing, including 3D/motion detections. Extensive experiments on different scenarios demonstrate the effectiveness of Monet in handling advanced machine vision tasks with comparative accuracy as the electronic counterparts yet achieving a ten-fold improvement in computing efficiency. For intelligent computing, the trends of dealing with real-world advanced tasks are irreversible. Breaking the capacity and scalability limitations of single-channel ONN and further exploring the multichannel processing potential of wave optics, we anticipate that the proposed technique will accelerate the development of more powerful optical AI as critical support for modern advanced machine vision.

@article{xu2022multichannel,
  title={A multichannel optical computing architecture for advanced machine vision},
  author={Xu, Zhihao and Yuan, Xiaoyun and Zhou, Tiankuang and Fang, Lu},
  journal={Light: Science \& Applications},
  volume={11},
  number={1},
  pages={1--13},
  year={2022},
  publisher={Nature Publishing Group}
}

@article{zhou2021large,

  title={Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit},

  author={Zhou, Tiankuang and Lin, Xing and Wu, Jiamin and Chen, Yitong and Xie, Hao and Li, Yipeng and Fan, Jingtao and Wu, Huaqiang and Fang, Lu and Dai, Qionghai},

  journal={Nature Photonics},

  pages={1--7},

  year={2021},

  publisher={Nature Publishing Group}

}

In this Letter we propose the Fourier-space diffractive deep neural network ($F\text{−}D2\mathrm{NN}$) for all-optical image processing that performs advanced computer vision tasks at the speed of light. The $F\text{−}D2\mathrm{NN}$ is achieved by placing the extremely compact diffractive modulation layers at the Fourier plane or both Fourier and imaging planes of an optical system, where the optical nonlinearity is introduced from ferroelectric thin films. We demonstrated that $F\text{−}D2\mathrm{NN}$ can be trained with deep learning algorithms for all-optical saliency detection and high-accuracy object classification.

@article{article,

author = {Yan, Tao and Wu, Jiamin and Tiankuang, Zhou and Xie, Hao and Xu, Feng and Fan, Jingtao and Fang, Lu and Lin, Xing and Dai, Qionghai},

year = {2019},

month = {07},

pages = {},

title = {Fourier-space Diffractive Deep Neural Network},

volume = {123},

journal = {Physical Review Letters},

doi = {10.1103/PhysRevLett.123.023901}

}

Training an artificial neural network with backpropagation algorithms to perform advanced machine learning tasks requires an extensive computational process. This paper proposes to implement the backpropagation algorithm optically for in situ training of both linear and nonlinear diffractive optical neural networks, which enables the acceleration of training speed and improvement in energy efficiency on core computing modules. We demonstrate that the gradient of a loss function with respect to the weights of diffractive layers can be accurately calculated by measuring the forward and backward propagated optical fields based on light reciprocity and phase conjunction principles. The diffractive modulation weights are updated by programming a high-speed spatial light modulator to minimize the error between prediction and target output and perform inference tasks at the speed of light. We numerically validate the effectiveness of our approach on simulated networks for various applications. The proposed in situ optical learning architecture achieves accuracy comparable to in silico training with an electronic computer on the tasks of object classification and matrix-vector multiplication, which further allows the diffractive optical neural network to adapt to system imperfections. Also, the self-adaptive property of our approach facilitates the novel application of the network for all-optical imaging through scattering media. The proposed approach paves the way for robust implementation of large-scale diffractive neural networks to perform distinctive tasks all-optically.

@article{Zhou:20,

author = {Tiankuang Zhou and Lu Fang and Tao Yan and Jiamin Wu and Yipeng Li and Jingtao Fan and Huaqiang Wu and Xing Lin and Qionghai Dai},

journal = {Photon. Res.},

keywords = {Diffractive optical elements; Light fields; Optical fields; Optical networks; Optical neural systems; Phase shifting digital holography},

number = {6},

pages = {940--953},

publisher = {OSA},

title = {In situ optical backpropagation training of diffractive optical neural networks},

volume = {8},

month = {Jun},

year = {2020},

url = {http://www.osapublishing.org/prj/abstract.cfm?URI=prj-8-6-940},

doi = {10.1364/PRJ.389553}

}