Xiaoyun Yuan


Xiaoyun Yuan

Postdoctoral researcher, Tsinghua University

Email: yuanxy92@gmail.com

Web: http://xiaoyunyuan.net

Research Interests

• Computational Photography

• Gigapixel Imaging

• Optical Computing

• Photoacoustic Imaging


Ph.D. in Electronic & Computer Engineering Aug 2015 – Jul 2020

Hong Kong University of Science and Technology, Hong Kong

B.Sc. in Electronics Science and Technology Aug 2010 – Jun 2014

University of Science and Technology of China, China

Academic Experience

Tsinghua University, Beijing, China Aug 2020 – Now

• Research topic: Computational photography, Optical computing, Human-centric analysis.

California Institute of Technology, Pasadena, California, USA Jun 2019 – Jun 2020

• Advisor: Dr. Lihong V. Wang (National Academy of Engineering member)

• Research topic: Photoacoustic computed tomography (PACT) for human brain imaging.

Tsinghua-Berkeley Shenzhen Institute, Tsinghua University, Shenzhen, China Jul 2017 – Present

• Advisor: Dr. Lu Fang and Dr. Qionghai Dai (Chinese Academy of Engineering member)

• Research topic: Realtime gigapixel imaging using unstructured camera arrays.

Tsinghua University, Beijing, China Aug 2016 – Jul 2017

• Advisor: Dr. Yebin Liu and Dr. Qionghai Dai (Chinese Academy of Engineering member)

• Research topic: Multiscale gigapixel videography using hybrid camera arrays.

Hong Kong University of Science and Technology, Hong Kong Aug 2015 – Present

• Advisor: Dr. Lu Fang

• Research topic: Gigapixel videography, augmented reality virtual try-on glasses.

University of Science and Technology of China, Hefei, China Jun 2014 – Jun 2015

• Advisor: Dr. Lu Fang

• Research topic: Image deblurring using hybrid prior.

Industry Experience

Research intern, Kunshan Duke Camputer Laboratory, Kunshan, China Mar 2017 – Jun 2017

● Advisor: Dr. David J. Brady (IEEE, OSA, SPIE fellow)

● Worked on “Mantis70” and “Mantis150” gigapixel camera array.

● Responsible for system function testing (auto white balance, autofocus, cooling), developed a new video stitching algorithm, and developed the prototype of “Pathfinder”.

Research intern, DJI, Shenzhen, China Jun 2015 – Aug 2015

● Worked on “Guidance” (a visual sensing system for drones).

● Responsible for the robustness test of depth perception and the accuracy measurement of visual odometry.

Software Development Engineer intern, iFlytek, Hefei, China Jul 2013 – Nov 2013

● Worked on “iflybook” (an interactive multimedia instruction product used by middle schools).

● Responsible for database interfaces (SQL, PHP), front-end development (JavaScript, CSS, HTML).



Scientific programming:

C/C++, CUDA, MATLAB, Python

Vision and graphics:

OpenCV (C++), OpenGL

Applications development:

PHP, CSS, JavaScript, HTML, Java, Unity


Linear algebra, Convex optimization, Machine learning, Statistical analysis, Probability theory


Photoshop, Premiere Pro, Arduino, Digital photography, Industrial camera development (FLIR, Ximea)

Project Experience

Photoacoustic Computed Tomography (PACT) for Human Brain Imaging Jun 2019 – Present

● Designed a 3D-PACT system with massively parallel transducer arrays and hemispherical scanning approach.

● Designed algorithms for analyzing and mapping brain functions.

● Proposed a multi-layer universal back-projection (UBP) algorithm for PACT reconstruction with heterogeneous media.

● Tested on human subjects and compare it to the state-of-the-art 7T MRI scanner.

Hierarchical Camera Array for Realtime Gigapixel Imaging Jun 2016 – Present

● Designed novel hierarchical camera array: wide-FoV camera for panoramic view capturing and unstructured telephoto camera for local detail capturing.

● Proposed an algorithm for fast online calibration to achieve realtime interactive gigapixel videography.

● Developed raw Bayer data capturing, pre-processing (demosaicing, auto white balance, color enhancement), compression and streaming systems.

● Proposed a cross-resolution depth perception algorithm to achieve multiscale 3D panoramic videography capturing for virtual reality rendering.

PANDA: A Gigapixel-level Human-centric Video Dataset Aug 2018 – Present

● Developed a gigapixel videography capturing system based on the hierarchical camera arrays.

● Benchmarked the state-of-the-art algorithms: object detection, tracking, and crowd counting.

● Designed a new task enabled by our dataset: interactive aware group detection using both wide-FoV and local detail information.

Magic Glasses: Augmented Reality Virtual Try-on System Jul 2015 – Jun 2016

● Developed a virtual try-on system for glasses using Kinect.

● Proposed an algorithm to automatically generate 3D glasses models from 2D images.

Image Deblurring using Combined Prior Sept 2014 – Jun 2015

● Proposed a deconvolution algorithm with combined patch recurrence image prior and separable kernel prior.

3D Virtual Campus Oct 2012 – Jun 2013

● Reconstructed the 3D building models using a laser scanner.

● Captured and map textures of the scanned models using DSLR cameras and 3ds max.

● Developed an interactive virtual campus tour program using Unity.


● Xiaoyun Yuan*, Mengqi Ji*, Jiamin Wu, David Brady, Qionghai Dai, Lu Fang. A Modular Hierarchical Array Camera. Light: Science & Applications. (Accepted)

● Shuai Na*, Jonathan J. Russin*, Li Lin*, Xiaoyun Yuan*, Peng Hu, Kay B. Jann, Lirong Yan, Konstantin Maslov, Junhui Shi, Danny J. Wang, Charles Y. Liu, and Lihong V. Wang. Massively Parallel Photoacoustic Computed Tomography of Human Brain Function. Nature Biomedical Engineering. (Accepted)

● Li Lin*, Peng Hu*, Xin Tong*, Shuai Na*, Rui Cao, Xiaoyun Yuan, David C. Garrett, Junhui Shi, Konstantin Maslov, and Lihong V. Wang. High-Speed Three-Dimensional Photoacoustic Computed Tomography for Preclinical Research and Clinical Translation. Nature Communications. (Accepted)

● Yang Tan*, Haitian Zheng*, Yinheng Zhu, Xiaoyun Yuan, Xing Lin, David Brady and Lu Fang. CrossNet++: Cross-scale Large-parallax Warping for Reference-based Super-resolution. IEEE Transactions on Pattern Analysis and Machine Intelligence. (Accepted)

● Jianing Zhang*, Tianyi Zhu*, Anke Zhang*, Xiaoyun Yuan*, Zihan Wang, Sebastian Beetschen, Lan Xu, Xing Lin, Qionghai Dai, Lu Fang. Multiscale-VR: Multiscale Gigapixel 3D Panoramic Videography for Virtual Reality. IEEE International Conference on Computational Photography (ICCP), Apr 2020.

● Xueyang Wang, Xiya Zhang, Yinheng Zhu, Yuchen Guo, Xiaoyun Yuan, Liuyu Xiang, Zerun Wang, Guiguang Ding, David Brady, Qionghai Dai, Lu Fang. PANDA: A Gigapixel-level Human-centric Video Dataset. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2020.

● Xiaoyun Yuan, Lu Fang, Qionghai Dai, David J Brady, Yebin Liu. Multiscale Gigapixel Video: A Cross Resolution Image Matching and Warping Approach. IEEE International Conference on Computational Photography (ICCP), May 2017.

● Xiaoyun Yuan, Ziwei Xu, Haoqian Wang, Yebin Liu, Lu Fang. Cascaded Image Deblurring with Combined Image Prior. IEEE Global Conference on Signal and Information Processing (GlobalSIP), November 2017

● Xiaoyun Yuan*, Difei Tang*, Yebin Liu, Qing Ling, Lu Fang. Magic Glasses: From 2D to 3D. IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), April 2016.

Academic Services

• Journal reviewer: IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), IEEE Transactions on Multimedia (TMM)

• Organized CVPR2019 workshop: “GigaVision: When Gigapixel Videography meets Computer Vision”.