A Caltech Library Service

Self-Calibrating Neural Radiance Fields

Jeong, Yoonwoo and Ahn, Seokjun and Choy, Christopher and Anandkumar, Animashree and Cho, Minsu and Park, Jaesik (2021) Self-Calibrating Neural Radiance Fields. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE , Piscataway, NJ, pp. 5826-5834. ISBN 978-1-6654-2812-5.

[img] PDF - Submitted Version
Creative Commons Attribution Share Alike.


Use this Persistent URL to link to this item:


In this work, we propose a camera self-calibration algorithm for generic cameras with arbitrary non-linear distortions. We jointly learn the geometry of the scene and the accurate camera parameters without any calibration objects. Our camera model consists of a pinhole model, a fourth order radial distortion, and a generic noise model that can learn arbitrary non-linear camera distortions. While traditional self-calibration algorithms mostly rely on geometric constraints, we additionally incorporate photometric consistency. This requires learning the geometry of the scene, and we use Neural Radiance Fields (NeRF). We also propose a new geometric loss function, viz., projected ray distance loss, to incorporate geometric consistency for complex non-linear camera models. We validate our approach on standard real image datasets and demonstrate that our model can learn the camera intrinsics and extrinsics (pose) from scratch without COLMAP initialization. Also, we show that learning accurate camera models in a differentiable manner allows us to improve PSNR over baselines. Our module is an easy-to-use plugin that can be applied to NeRF variants to improve performance. The code and data are currently available at

Item Type:Book Section
Related URLs:
URLURL TypeDescription Paper ItemCode and data
Anandkumar, Animashree0000-0002-6974-6797
Additional Information:© 2021 IEEE. This work was supported by the IITP grants (2019-0-01906: AI Grad. School Prog. - POSTECH and 2021-0-00537: visual common sense through selfsupervised learning for restoration of invisible parts in images) funded by Ministry of Science and ICT, Korea.
Funding AgencyGrant Number
Institute of Information & Communications Technology Planning & Evaluation (IITP)2019-0-01906
Institute of Information & Communications Technology Planning & Evaluation (IITP)2021-0-00537
Ministry of Science and ICT (Korea)UNSPECIFIED
Record Number:CaltechAUTHORS:20220715-171630599
Persistent URL:
Official Citation:Y. Jeong, S. Ahn, C. Choy, A. Anandkumar, M. Cho and J. Park, "Self-Calibrating Neural Radiance Fields," 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 5826-5834, doi: 10.1109/ICCV48922.2021.00579
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:115616
Deposited By: George Porter
Deposited On:15 Jul 2022 23:30
Last Modified:12 Aug 2022 16:57

Repository Staff Only: item control page