A Caltech Library Service

Vision without the Image

Chen, Bo and Perona, Pietro (2016) Vision without the Image. Sensors, 16 (4). Art. No. 484. ISSN 1424-8220. PMCID PMC4850998.

[img] PDF - Published Version
Creative Commons Attribution.


Use this Persistent URL to link to this item:


Novel image sensors transduce the stream of photons directly into asynchronous electrical pulses, rather than forming an image. Classical approaches to vision start from a good quality image and therefore it is tempting to consider image reconstruction as a first step to image analysis. We propose that, instead, one should focus on the task at hand (e.g., detection, tracking or control) and design algorithms that compute the relevant variables (class, position, velocity) directly from the stream of photons. We discuss three examples of such computer vision algorithms and test them on simulated data from photon-counting sensors. Such algorithms work just-in-time, i.e., they complete classification, search and tracking with high accuracy as soon as the information is sufficient, which is typically before there are enough photons to form a high-quality image. We argue that this is particularly useful when the photons are few or expensive, e.g., in astronomy, biological imaging, surveillance and night vision.

Item Type:Article
Related URLs:
URLURL TypeDescription CentralArticle
Chen, Bo0000-0001-5566-7361
Perona, Pietro0000-0002-7583-5809
Additional Information:© 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons by Attribution (CC-BY) license ( Received: 2 February 2016 / Revised: 26 March 2016 / Accepted: 4 April 2016 / Published: 6 April 2016. This work was funded by Office of Naval Research (ONR) N00014-10-1-0933 and Gordon and Betty Moore Foundation (Palo Alto, CA, USA). Author Contributions: B. Chen and P. Perona conceived and designed the experiments. B. Chen performed the experiments and analyzed the data. B. Chen and P. Perona wrote the paper. The authors declare no conflict of interest.
Funding AgencyGrant Number
Office of Naval Research (ONR)N00014-10-1-0933
Gordon and Betty Moore FoundationUNSPECIFIED
Subject Keywords:photon-counting sensors; visual recognition; low-light computer vision
Issue or Number:4
PubMed Central ID:PMC4850998
Record Number:CaltechAUTHORS:20160411-082516943
Persistent URL:
Official Citation:Chen, B.; Perona, P. Vision without the Image. Sensors 2016, 16, 484
Usage Policy:No commercial reproduction, distribution, display or performance rights in this work are provided.
ID Code:66041
Deposited By: Tony Diaz
Deposited On:12 Apr 2016 23:24
Last Modified:27 Mar 2020 19:18

Repository Staff Only: item control page