Intelligent Sensors Directly Analyze and Interpret Information in Light

The Science

Beams of light consist of a range of wavelengths, collectively known as the optical spectrum. These spectra (plural of spectrum) contain detailed information that extends beyond the limits of human vision. Advanced sensors and computers can take in and process this information. When certain sensors are exposed to light, they produce an electrical current called a photocurrent. Computers then process this photocurrent. However, leveraging such information for intelligent vision requires generating and computing dense information. These processes limit speed and power efficiency. This work overcomes such bottlenecks by using intelligent sensors called spectral kernel machines (SKMs). SKMs directly convert a photocurrent into object and chemical identification without needing additional processing. Scientists validated the concept with conventional silicon detectors and emerging black phosphorus detectors. They tested it across visible, near-infrared, and mid-infrared spectral ranges.

The Impact

Robots, drones, and satellites must recognize their surrounding environment with high speed and limited battery life. They also need to recognize objects as they adapt to unpredictable situations. SKMs offer a promising solution that is orders of magnitude faster and more power efficient than current technologies. They can also learn from examples and perform inference tasks. This function is like the 'sniff-and-seek' of retriever dogs. This ability allows SKMs to resolve, recognize, and compress high-dimensional spectral-spatial information. These findings will inspire new intelligent sensors with applications such as precision agriculture, autonomous driving, chemical sensing, and scientific measurements.

Summary

Spectral machine vision processes spectral and spatial data into three-dimensional hypercubes for scene recognition. It has major challenges related to data bottlenecks that affect power efficiency, frame rate, and spectral and spatial resolution. Researchers have found a way to solve this problem by developing the novel SKM technology. It integrates spectral machine learning analysis directly into the photodetection architecture. The experimental verification of various SKMs in the visible to mid-infrared demonstrated capabilities for diverse tasks. These tasks included chemometrics, semiconductor metrology, and plant hydration level identification. The technology can achieve orders of magnitude faster speed and better energy efficiency than current hyperspectral imaging technologies. It shows great potential for applications in mobile devices, robotics, and satellite technologies.

Contact

Ali Javey
Lawrence Berkeley National Laboratory
University of California, Berkeley
[email protected]

Funding

Material preparation, optical characterizations, and device design and fabrication were supported by the US Department of Energy, Office of Science, Office of Basic Energy Sciences, Materials Sciences and Engineering Division. Part of the SKM measurements were supported by the US Department of Energy, MEERCAT-Nanoscale Hybrids: A New Paradigm for Energy Efficient Optoelectronics. One of the scientists also received support from the US Army Research Office (ARO).

Publications

Zhang, D., et al. "Spectral kernel machines with electrically tunable photodetectors". Science, 390.6776, eady6571, (2025). [DOI: 10.1126/science.ady6571]

Related Links

Intelligent photodetectors 'sniff and seek' like retriever dogs to recognize materials directly from light spectra, TechXplore