Image sensing has a long history but its evolution to computational sensors allows cameras to produce higher resolution images than ever before. Advancements in image sensor technology help undo blurring in photos and allow us to capture images like never before with amazing accuracy in depth and exposure.
Image sensing has had a long history, starting from the earliest film cameras of the daguerreotype that recorded photons of light impinging on polished metals to the analog rolls of film and darkrooms to expose them in the 20th century.
However, the rise of computation with image sensing really took off with the invention of the Complementary Metal Oxide Semiconductor - more commonly known by the acronym CMOS - image sensor in the 1990s at JPL, where circuit electronics could be placed side-by-side, as equals, with photosensitive pixels. This led to a revolution where cameras could be miniaturized, placed into mobile devices and carried in our pockets.
Central to the sensor is the idea of exposure: how long the sensor can accumulate photons of light to create a bright, good-looking image. Computational sensors now can better manipulate their exposure to achieve more new, unseen imagery than ever before.
You may have seen the acronym “HDR” in your phone’s camera settings. High dynamic range (or HDR) photography creates images from multiple exposures of different levels. These multiple exposures are then stitched together by an algorithm (usually on board the camera hardware) to create a final stunning image.
However, we can do more than just capture prettier pictures. Coded exposure is a technique in which the camera’s exposure is randomly modulated so that the blur looks jagged and noisy. However, using algorithms, we can undo this blurring and achieve sharp images – useful for deblurring an image CSI-style.
As sensor technology improves, new sensors are coming out into the market everyday. These include sensors that sense at wavelengths of light different than the visible spectrum (for example, hyperspectral and infrared cameras) and new sensors such as single-photon avalanche diodes (or SPADS for short) that can perform time-of-flight imaging, recover depth accurately, and even be utilized to see around corners.
Image sensors are becoming more computational in nature, and will continue to do so, leveraging advances in machine learning and image and signal processing to become parts of intelligent camera systems in the future.
