Fourier ptychographic microscopy image stack reconstruction using implicit neural representations
Abstract
Image stacks provide invaluable 3D information in various biological and pathological imaging applications. Fourier ptychographic microscopy (FPM) enables reconstructing high-resolution, wide field-of-view image stacks without z-stack scanning, thus significantly accelerating image acquisition. However, existing FPM methods take tens of minutes to reconstruct and gigabytes of memory to store a high-resolution volumetric scene, impeding fast gigapixel-scale remote digital pathology. While deep learning approaches have been explored to address this challenge, existing methods poorly generalize to novel datasets and can produce unreliable hallucinations. This work presents FPM-INR, a compact and efficient framework that integrates physics-based optical models with implicit neural representations (INRs) to represent and reconstruct FPM image stacks. FPM-INR is agnostic to system design or sample types and does not require external training data. In our experiments, FPM-INR substantially outperforms traditional FPM algorithms with up to a 25-fold increase in speed and an 80-fold reduction in memory usage for continuous image stack representations.
Copyright and License
© 2023 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement.
Acknowledgement
H.Z., S.L., M.L., and C.Y. would like to thank the Heritage Research Institute for the Advancement of Medicine and Science at Caltech. B.Y.F. and C.A.M. were supported in part by the AFOSR Young Investigator Program and ONR. Partial funding for open access provided by the UMD Libraries' Open Access Publishing Fund.
Funding
Office of Naval Research (N000142312752); Air Force Office of Scientific Research Young Investigator Program (FA9550-22-1-0208); Heritage Research Institute for the Advancement of Medicine and Science at Caltech (HMRI-15-09-01); UMD Libraries' Open Access Publishing Fund.
Data Availability
The code is available at [55]. Data are available at [56].
See Supplement 1 for supporting content.
Conflict of Interest
The authors declare no conflicts of interest.
Files
Name | Size | Download all |
---|---|---|
md5:baf5250d8f652fe18c9e0713c565c56e
|
15.5 MB | Preview Download |
md5:5e5c01d19c5a13ca6ef3007bf1e22858
|
1.4 MB | Preview Download |
Additional details
- Office of Naval Research
- N000142312752
- United States Air Force Office of Scientific Research
- FA9550-22-1-0208
- Heritage Research Institute for the Advancement of Medicine and Science at Caltech
- HMRI-15-09-01
- University of Maryland, College Park
- UMD Libraries' Open Access Publishing Fund