Rajesh Menon
This process is slow, however, meaning that hyperspectral cameras can only take still images, or videos with very low frame rates, or frames per second (fps). But what if a high-fps video camera could capture dozens of wavelengths at once, revealing details invisible to the naked eye?
Now, researchers at the University of Utah’s John and Marcia Price College of Engineering and College of Science have developed a new way of taking a high-definition snapshot that encodes spectral data into images, much like a traditional camera encodes color. Instead of a filter that divides light into three color channels, their specialized filter divides it into 25. Each pixel stores compressed spectral information along with its spatial information, which computer algorithms can later reconstruct into a “cube” of 25 separate images—each representing a distinct slice of the visible spectrum.
This instantaneous encoding enables the researchers’ camera system—small enough to fit into a cellphone—to take high-definition video, and the compressed nature of the component images opens up new real-world applications.
Fernando Guevara Vasquez
A study demonstrating the camera was led by research assistant professor Apratim Majumder and professor Rajesh Menon, both in the Department of Electrical & Computer Engineering. Coauthors include Fernando Guevara Vasquez, a U professor of mathematics. Funded by the Office of Naval Research, this research was conducted in collaboration with the U tech startup Lumos Imaging, which Menon and Guevara Vasquez founded in 2015.
The results are reported in the journal Optica.
The camera’s design represents a leap forward in how spectral data can be captured.
“We introduce a compact camera that captures both color and fine spectral details in a single snapshot, producing a ‘spectral fingerprint’ for every pixel,” Menon said.
Hyperspectral cameras have long been used in agriculture, astronomy and medicine, where subtle differences in color can make a big difference. But these cameras have historically been bulky, expensive and limited to still images.
“When we started out on this research, our intention was to demonstrate a compact, fast, megapixel resolution hyperspectral camera, able to record highly compressed spatial-spectral information from scenes at video-rates, which did not exist,” Majumder said.
The U team’s breakthrough lies in how it captures and processes the data. The key component is a diffractive element that is placed directly over the camera’s sensor. It’s the element’s repeating nanoscale patterns that diffract incoming light and encodes both spatial and spectral information for each pixel on the sensor. By encoding the scene into a single, compact two-dimensional image rather than a massive three-dimensional data cube, the camera makes hyperspectral imaging faster and more efficient.
“One of the primary advantages of our camera is its ability to capture the spatial-spectral information in a highly compressed two-dimensional image instead of a three-dimensional data cube and use sophisticated computer algorithms to extract the full data cube at a later point,” Majumder explained. “This allows for fast, highly compressed data capture.”
Instead of a filter that divides light into three color channels, the University of Utah engineers and mathematicians have developed a diffractive element that divides it into 25.
The researchers’ diffractive element contains a repeating pattern of nanoscale features.
Guevara Vasquez, of the Mathematics Department, modeled and designed the diffractive filter array and co-designed, with co-author Fernando Gonzalez del Cueto, the algorithm to reconstruct the hyperspectral images from the raw data captured by the sensor.
The camera’s streamlined approach also cuts costs dramatically.
“Our camera costs many times less, is very compact and captures data much faster than most available commercial hyperspectral cameras,” Majumder said. “We have also shown the ability to post-process the data as per the need of the application and implement different classifiers suited to different fields such as agriculture, astronomy and bio-imaging.”
Data storage is another advantage.
“Satellites would have trouble beaming down full image cubes, but since we extract the cubes in post-processing, the original files are much smaller,” Majumder added.
To demonstrate the camera’s capabilities, the researchers tried three real-world applications: telling different types of tissue apart in a surgical scene; predicting the age of strawberries as they decayed over time; and mimicking a series of spectral filters that are used in astronomy.
The current prototype takes images at just over one megapixel in size and can break them down into 25 separate wavelengths across the spectrum. But the team is already working on improvements.
“This work demonstrates a first snapshot megapixel hyperspectral camera,” Majumder said. “Next, we are developing a more improved version of the camera that will allow us to capture images at a larger image size and an increased number of wavelength channels, while also making the nano-structured diffractive element much simpler in design.”
By making hyperspectral imaging cheaper, faster and more compact, the U engineers have opened the door for technologies that could change the way we see the world and uncover details hidden across the spectrum.
