Difference between revisions of "Hyperspectral Imaging"

Latest revision as of 11:54, 27 September 2023

A hyperspectral camera captures a scene in many frequency bands across the spectrum, providing rich information and facilitating numerous applications in industrial, military, and commercial domains. Example applications of hyperspectral imaging include medical diagnosis (e.g., early detection of skin cancer), food-quality inspection, artwork authentication, forest monitoring, material identification, and remote sensing. The potential of hyperspectral imaging has been established for decades now. However, to date, hyperspectral imaging has only seen success in specialized and large-scale industrial and military applications. This is because of three main challenges: (i) the sheer volume of hyperspectral data which makes it hard to transmit such data in real-time and thus limiting its usefulness for many applications, (ii) the negative impact of the environmental conditions (e.g., rain, fog, and snow) which reduces the utility of the captured hyper-spectral data, and (iii) the high cost of hyperspectral cameras (upwards of $20K USD) which makes the technology out of reach for many commercial and end-user applications. The goal of this project is to address these challenges to enable wide adoption of hyperspectral imaging in many applications.

MobiSpectral: Hyperspectral Imaging on Mobile Devices

Hyperspectral imaging systems capture information in multiple wavelength bands across the electromagnetic spectrum. These bands provide substantial details based on the optical properties of the materials present in the captured scene. The high cost of hyperspectral cameras and their strict illumination requirements make the technology out of reach for end-user and small-scale commercial applications. We propose MobiSpectral, which turns a low-cost phone into a simple hyperspectral imaging system, without any changes in the hardware. We design deep learning models that take regular RGB images and near-infrared (NIR) signals (which are used for face identification on recent phones) and reconstruct multiple hyperspectral bands in the visible and NIR ranges of the spectrum. Our experimental results show that MobiSpectral produces accurate bands that are comparable to ones captured by actual hyperspectral cameras. The availability of hyperspectral bands that reveal hidden information enables the development of novel mobile applications that are not currently possible. To demonstrate the potential of MobiSpectral, we use it to identify organic solid foods, which is a challenging food fraud problem that is currently partially addressed by laborious, unscalable, and expensive processes. We collect large datasets in real environments under diverse illumination conditions to evaluate MobiSpectral. Our results show that MobiSpectral can identify organic foods, e.g., apples, tomatoes, kiwis, strawberries, and blueberries, with an accuracy of up to 94% from images taken by phones.

Overview of MobiSpectral

People

Neha Sharma
Muhammad Shahzaib Waseem
Shahrzad Mirzaei
Mariam Bebawy

Code and Datasets

Publications

N. Sharma, M. Waseem, S. Mirzaei, and M. Hefeeda, MobiSpectral: Hyperspectral Imaging on Mobile Devices, In Proc. of ACM Conference on Mobile Computing and Networking (MobiCom'23), Madrid, Spain, October 2023. (Artifacts Evaluated and Functional)
P. Moghadam, N. Sharma, M. Hefeeda, Enabling Hyperspectral Imaging in Diverse Illumination Conditions for Indoor Applications, In Proc. of ACM Multimedia Systems Conference (MMSys’21), Istanbul, Turkey, September 2021. (Artifacts Evaluated and Functional)

N. Sharma, M. Hefeeda, Hyperspectral Reconstruction from RGB Images for Vein Visualization, In Proc. of ACM Multimedia Systems Conference (MMSys'20), Istanbul, Turkey, June 2020. (Artifacts Evaluated and Functional)

M. Arab, K. Calagari, M. Hefeeda, Band and Quality Selection for Efficient Transmission of Hyperspectral Images, In Proc. of ACM Conference on Multimedia (MM'19), Nice, France. October 2019.

@@ Line 1: / Line 1: @@
-Hyperspectral imaging (HSI) is a powerful tool that can provide substantial information about a scene through remote sensing. This project addresses different challenges in the pipeline of capturing, transmitting, and identifying remote scenes using hyperspectral images. One of the main significant challenges in hyperspectral imaging is analyzing the data and extracting the required information. This is mainly because of the extremely high dimensionality of hyperspectral images, which limits our ability to identify spectral signatures fast and accurately. In addition, noise in hyperspectral images makes matters even more complicated.
+A hyperspectral camera captures a scene in many frequency bands across the spectrum, providing rich information and facilitating numerous applications in industrial, military, and commercial domains. Example applications of hyperspectral imaging include medical diagnosis (e.g., early detection of skin cancer), food-quality inspection, artwork authentication, forest monitoring, material identification, and remote sensing. The potential of hyperspectral imaging has been established for decades now. However, to date, hyperspectral imaging has only seen success in specialized and large-scale industrial and military applications. This is because of three main challenges: (i) the sheer volume of hyperspectral data which makes it hard to transmit such data in real-time and thus limiting its usefulness for many applications, (ii) the negative impact of the environmental conditions (e.g., rain, fog, and snow) which reduces the utility of the captured hyper-spectral data, and (iii) the high cost of hyperspectral cameras (upwards of $20K USD) which makes the technology out of reach for many commercial and end-user applications. The goal of this project is to address these challenges to enable wide adoption of hyperspectral imaging in many applications.
+'''MobiSpectral: Hyperspectral Imaging on Mobile Devices'''
+Hyperspectral imaging systems capture information in multiple wavelength bands across the electromagnetic spectrum. These bands provide substantial details based on the optical properties of the materials present in the captured scene. The high cost of hyperspectral cameras and their strict illumination requirements make the technology out of reach for end-user and small-scale commercial applications. We propose MobiSpectral, which turns a low-cost phone into a simple hyperspectral imaging system, without any changes in the hardware. We design deep learning models that take regular RGB images and near-infrared (NIR) signals (which are used for face identification on recent phones) and reconstruct multiple hyperspectral bands in the visible and NIR ranges of the spectrum. Our experimental results show that MobiSpectral produces accurate bands that are comparable to ones captured by actual hyperspectral cameras. The availability of hyperspectral bands that reveal hidden information enables the development of novel mobile applications that are not currently possible. To demonstrate the potential of MobiSpectral, we use it to identify organic solid foods, which is a challenging food fraud problem that is currently partially addressed by laborious, unscalable, and expensive processes. We collect large datasets in real environments under diverse illumination conditions to evaluate MobiSpectral. Our results show that MobiSpectral can identify organic foods, e.g., apples, tomatoes, kiwis, strawberries, and blueberries, with an accuracy of up to 94% from images taken by phones.
+[[File:Mobispectral.png|thumb|center|700px|Overview of MobiSpectral]]
 == People ==
-* Mohammad Amin Arab
+* [https://www.sfu.ca/~nsa84/ Neha Sharma]
-* Kiana Calagari
+* Muhammad Shahzaib Waseem
-* [https://www.cs.sfu.ca/~mhefeeda/ Mohamed Hefeeda]
+* Shahrzad Mirzaei
+* Mariam Bebawy
-== Adaptive Identification of Remote Scenes using HSI ==
+== Code and Datasets ==
+* [https://github.com/mobispectral/mobicom23_mobispectral MobiSpectral: Hyperspectral Imaging on Mobile Devices]
+* [https://github.com/pazadimo/HS_In_Diverse_Illuminations Enabling Hyperspectral Imaging in Diverse Illumination Conditions for Indoor Applications]
+* [https://github.com/nehasharma512/vein-visualization Hyperspectral Reconstruction from RGB Images for Vein Visualization]
+* [https://github.com/maarab-sfu/BQSETHI Band and Quality Selection for Efficient Transmission of Hyperspectral Images]
-''' Abstract '''
+== Publications ==
+* N. Sharma, M. Waseem, S. Mirzaei, and M. Hefeeda, [https://www2.cs.sfu.ca/~mhefeeda/Papers/mobiCom23_MobiSpectral.pdf MobiSpectral: Hyperspectral Imaging on Mobile Devices], In Proc. of ACM Conference on Mobile Computing and Networking (MobiCom'23), Madrid, Spain, October 2023. (Artifacts Evaluated and Functional)
+* P.  Moghadam, N. Sharma, M. Hefeeda, [https://www2.cs.sfu.ca/~mhefeeda/Papers/mmsys21_HS_illumination.pdf Enabling Hyperspectral Imaging in Diverse Illumination Conditions for Indoor Applications], In Proc. of ACM Multimedia Systems Conference (MMSys’21), Istanbul, Turkey, September 2021. (Artifacts Evaluated and Functional)
-In this project we aim to thoroughly investigate and explore the scene through remote sensing. We do so by using remotely controlled drones to capture videos of a desired site. The captured information will be then transferred to a base station, where it will be processed and analyzed. As the desired site may be a remote outland with limited available bandwidth and given the size of hyperspectral data, all our methods and techniques are designed adaptively. Our methods prioritize data transfer based on the amount of detail required such that maximum accuracy is achieved using a specific amount of transferred data. This is in contrast to the state-of-the-art methods, which assume having all the data is in hand before starting the analysis.
+* N. Sharma, M. Hefeeda, [https://www2.cs.sfu.ca/~mhefeeda/Papers/mmsys20_vein.pdf Hyperspectral Reconstruction from RGB Images for Vein Visualization], In Proc. of ACM Multimedia Systems Conference (MMSys'20), Istanbul, Turkey, June 2020. (Artifacts Evaluated and Functional)
-[[Adaptive Identification of Remote Scenes using HSI|More info and demo]] ...
+* M. Arab, K. Calagari, M. Hefeeda, [https://www2.cs.sfu.ca/~mhefeeda/Papers/mm19_bandSelection.pdf Band and Quality Selection for Efficient Transmission of Hyperspectral Images], In Proc. of ACM Conference on Multimedia (MM'19),  Nice, France. October 2019.