The team of Professor Zeng Heping and Researcher Huang Kun of the State Key Laboratory of Precision Spectroscopic Science and Technology, East China Normal University, has made progress in mid-infrared spectral imaging, combining nonlinear upconversion imaging with tunable acousto-optic filtering technology, effectively improving the acquisition speed of spatial-wavelength three-dimensional spectral information, and realising the mid-infrared hyperspectral video imaging with ultra-sensitive, large field-of-view, and high frame rate, which could provide support for the analysis of chemical transient processes, It can provide powerful support for applications such as chemical transient process analysis, biological in situ imaging detection, medical real-time spectral imaging and rapid diagnosis of combustion field. The results were published online in Nature Communications on 28 February 2024 under the title of Wide-field mid-infrared hyperspectral imaging beyond video rate, with East China Normal University (ECNU) as the first author, PhD student Canan Fang as the first author, Prof. Heping Zeng and Kun Huang as the first author, and Prof. Heping Zeng and Kun Huang as the first authors. Dr Canan Fang is the first author of the paper, and Prof Heping Zeng and Kun Huang are the co-corresponding authors.
Figure 1: Nature Communications published the research results of Prof. Zeng Heping and Researcher Huang Kun's team.
Hyperspectral imaging is a multi-dimensional information acquisition method combining imaging and spectral technologies, which can non-invasively image the target in hundreds or even more spectral bands, and generate spectral data cubes containing spatial and spectral information. Therefore, hyperspectral images have the important feature of "one image, one spectrum", where each pixel corresponds to a set of spectral information, and the rich information contained therein is capable of determining and characterising the chemical composition, content and distribution of the sample. In particular, the mid-infrared (MIR) band is located in the fingerprint spectral region of molecules, which contains the absorption peaks of many functional groups, and the implementation of hyperspectral imaging in this band allows for the precise identification of the target to be measured in a label-free manner. Therefore, mid-infrared hyperspectral imaging technology has been widely used in the fields of trace analysis, environmental monitoring, biomedicine, and materials science.
Figure 2: Conceptual diagram of the principle of mid-infrared high-speed hyperspectral imaging.
However, the infrared hyperspectral imaging system, which combines multi-spectral bands and large format, has long been limited to observing static samples or low-speed motion scenes, and is difficult to be used for rapid target measurement or dynamic process capture. On the one hand, the spectral data generated by hyperspectral imaging provides rich target information, which helps to accurately analyse and identify the samples; on the other hand, the huge amount of data acquisition greatly limits the rate of hyperspectral imaging. For example, traditional pendulum-scanning and push-scanning hyperspectral imaging systems mainly rely on grating, prisms and other devices to achieve signal dispersion spectroscopy, and often need to rely on point scanning or line scanning to achieve two-dimensional image coverage in spatial information acquisition. In order to overcome the lengthy mechanical scanning, full-width spectral imaging technology came into being, which adopts tunable narrow-band light sources (e.g., optical parametric oscillators, quantum cascade lasers) or wavelength-tunable filters (e.g., acousto-optic and liquid-crystal filters) to carry out spectral scanning, which effectively enhances the efficiency of multi-pixel image acquisition. Even so, the speed of mid-infrared hyperspectral imaging is still largely limited by the operating frame rate of focal plane detector arrays in this wavelength band (especially for large-array multi-pixel cameras), and the typical value of the frame rate for monochromatic spectral image acquisition is 50 Hz @ 512 × 512 pixels. Accordingly, the acquisition of hyperspectral imaging with more than a hundred wavelength channels often takes several seconds or even longer, and there is still a quantum gap between the video frame rates that can be observed in real time. Currently, the realisation of mid-infrared hyperspectral imaging with a large field of view, multiple wavelengths, and high frame rate is still quite challenging, and requires the simultaneous realisation of high-speed spectral scanning and high-speed image acquisition.
Figure 3: Mid-infrared high-speed hyperspectral imaging device map
To this end, the research team innovatively combines nonlinear wide-angle imaging technology and high-speed acousto-optic filtering technology, which can simultaneously improve the infrared image acquisition rate and infrared spectral switching rate, overcoming the shortcomings of the traditional scheme in the acquisition of spectral information, and realising a three-dimensional spectral refresh rate as high as one hundred Hertz, which is at least two orders of magnitude better than the previous record in terms of the number of spectral segments and the size of pixels in the same spectral range. Specifically, the researchers used a specially designed chirp-polarised lithium niobate crystal to achieve broadband nonlinear optics and frequency, converting infrared signals in the supercontinuum spectrum to the visible band in one go. The process is characterised by large-field-of-view spatial mapping and high-fidelity spectral conversion, which preserves the complete target mapping information in both spatial and spectral dimensions. In order to achieve high-rate and high-precision wavelength tuning, the researchers used acousto-optic tunable filtering technology to obtain microsecond wavelength switching speed with nanometer-scale narrowband filtering bandwidth. The filtered monochrome image is captured by a high-performance silicon-based camera, which circumvents the shortcomings of the existing infrared focal plane detection arrays in terms of sensitivity, pixel count, frame rate, etc., and thus achieves infrared image acquisition with a large field of view, multi-pixel, and high frame rate.
Figure 4: High frame rate mid-infrared hyperspectral video imaging (A) Experimentally determined infrared absorption spectra of benzene and ethanol. (B) Each hyperspectral data cube contains 100 fine spectral bands, and the monochrome image capturing time is only 100 μs.(C-D) Different spectral channels can be selected to easily differentiate the display of different substance compositions. (E) RGB colour synthesis of monochromatic maps corresponding to the absorption peaks of two liquids can clearly demonstrate the dynamic process of diffusion and fusion of different media.
The hyperspectral imaging system constructed in the experiment operates at a wavelength of 2.4-4.1 μm, covering the infrared telescopic and vibrational absorption spectra of a variety of CH/OH chemical bonds, which are important spectral bands for the identification of organic materials. In order to demonstrate the application of hyperspectral imaging in material identification and dynamic scenarios, the researchers chose two chemical samples, ethanol and benzene, which are colourless and transparent to the naked eye, but can be measured by hyperspectral imaging to obtain very different infrared spectra (Figure 4A), which can be used to achieve the effective screening of sample composition by using the unique molecular selectivity. In hyperspectral 3D data acquisition, the integration time for single-wavelength large-field-of-view imaging (nearly megapixel frame) is only 100 μs, and the acquisition of 100 spectral bands of spectral cubic data is only 10 ms (Fig. 4B), thus realising large-field-of-view hyperspectral images at 100 Hz level. Unlike the traditional mechanical wavelength tuning method, the acousto-optic tunable filter is not limited by mechanical inertia, and can perform fast dynamic tuning of the spectrum to achieve continuous and uninterrupted cyclic wavelength scanning, which provides the possibility of real-time spectral video imaging. As shown in Figs. 4C-4E, multiple monochrome greyscale images can be selected for RGB color-filling synthesis according to the absorption spectral characteristics of the sample to achieve a more intuitive visualisation of the chemical differences and concentration distribution of the sample. Figure 5 demonstrates the dynamic process of diffusion and fusion of two different liquid media.
Figure 5: Real-time visualisation of the dynamic mixing process of ethanol and benzene.
Although both chemical liquids are colourless and transparent in appearance, their infrared absorption spectra are significantly different. The dynamic mixing process of the two liquids can be clearly observed by artificial colour synthesis of the grey-scale images corresponding to the respective absorption peaks (ethanol: 3350 cm-1, benzene: 3050 cm-1 ).
It is worth mentioning that the developed upconversion spectral imaging technique benefits from the phase matching conditions required in the nonlinear optical mixing process, which makes the monochromatic upconversion images of different wavelengths have different spatial scaling factors, thus forming a unique imaging effect of wavelength-space coupling, which, combined with the specific information encoding and computational imaging algorithms, can recover the three-dimensional spectral information from a single grey-scale image, and then develop a single-shot Snapshot infrared hyperspectral imaging provides an effective way to achieve ultra-high-speed spectral photography. In addition, this technology can be extended to the long-wave infrared or terahertz bands to meet the urgent demand for high-speed spectral imaging in this spectral band, which can provide attractive means of spectral image analysis in the fields of materials, chemistry, biology, medicine and so on.
In recent years, Prof. Heping Zeng and Kun Huang have carried out a series of innovative researches in the field of mid-infrared multi-dimensional imaging, and have successively developed mid-infrared nonlinear wide-angle imaging [Nature Comm. 13, 1077 (2022)], mid-infrared single-photon single-pixel imaging [Nature Comm. 14, 1073 (2023)], and mid-infrared single-photon 3D imaging [Light Sci. Appl. [Light Sci. Appl. 12, 144 (2023)], and so on. The work was supported by grants from the Ministry of Science and Technology, the Foundation, Shanghai, Chongqing and East China Normal University.
Link to paper: Wide-field mid-infrared hyperspectral imaging beyond video rate | Nature Communications
