Difference between revisions of "Hyperspectral Imaging"
| Line 1: | Line 1: | ||
A hyperspectral camera captures a scene in many frequency bands across the spectrum, providing rich information and facilitating numerous applications in industrial, military, and commercial domains. Example applications of hyperspectral imaging include medical diagnosis (e.g., early detection of skin cancer), food-quality inspection, artwork authentication, forest monitoring, material identification, and remote sensing. The potential of hyperspectral imaging has been established for decades now. However, to date, hyperspectral imaging has only seen success in specialized and large-scale industrial and military applications. This is because of three main challenges: (i) the sheer volume of hyperspectral data which makes it hard to transmit such data in real-time and thus limiting its usefulness for many applications, (ii) the negative impact of the environmental conditions (e.g., rain, fog, and snow) which reduces the utility of the captured hyper-spectral data, and (iii) the high cost of hyperspectral cameras (upwards of $20K USD) which makes the technology out of reach for many commercial and end-user applications. The goal of this project is to address these challenges to enable wide adoption of hyperspectral imaging in many applications. | A hyperspectral camera captures a scene in many frequency bands across the spectrum, providing rich information and facilitating numerous applications in industrial, military, and commercial domains. Example applications of hyperspectral imaging include medical diagnosis (e.g., early detection of skin cancer), food-quality inspection, artwork authentication, forest monitoring, material identification, and remote sensing. The potential of hyperspectral imaging has been established for decades now. However, to date, hyperspectral imaging has only seen success in specialized and large-scale industrial and military applications. This is because of three main challenges: (i) the sheer volume of hyperspectral data which makes it hard to transmit such data in real-time and thus limiting its usefulness for many applications, (ii) the negative impact of the environmental conditions (e.g., rain, fog, and snow) which reduces the utility of the captured hyper-spectral data, and (iii) the high cost of hyperspectral cameras (upwards of $20K USD) which makes the technology out of reach for many commercial and end-user applications. The goal of this project is to address these challenges to enable wide adoption of hyperspectral imaging in many applications. | ||
| + | |||
MobiSpectral: Hyperspectral Imaging on Mobile Devices | MobiSpectral: Hyperspectral Imaging on Mobile Devices | ||
Hyperspectralimagingsystemscaptureinformationinmultiplewavelengthbandsacrosstheelectromagneticspectrum. Thesebandsprovidesubstantialdetailsbasedontheoptical propertiesofthematerialspresentinthecapturedscene. Thehighcostofhyperspectralcamerasandtheirstrictilluminationrequirementsmakethetechnologyoutofreach forend-userandsmall-scalecommercialapplications.We proposeMobiSpectral,whichturnsalow-costphoneintoa simplehyperspectralimagingsystem,withoutanychanges inthehardware.Wedesigndeeplearningmodelsthattake regularRGBimagesandnear-infrared(NIR)signals(which areusedforfaceidentificationonrecentphones)andreconstructmultiplehyperspectralbandsinthevisibleandNIR rangesofthespectrum.Ourexperimentalresultsshowthat MobiSpectralproducesaccuratebandsthatarecomparable toonescapturedbyactualhyperspectralcameras.Theavailabilityofhyperspectralbandsthatrevealhiddeninformation enablesthedevelopmentofnovelmobileapplicationsthat arenotcurrentlypossible.Todemonstratethepotentialof MobiSpectral,weuseittoidentifyorganicsolidfoods,which isachallengingfoodfraudproblemthatiscurrentlypartially addressedbylaborious,unscalable,andexpensiveprocesses. Wecollectlargedatasetsinrealenvironmentsunderdiverse illuminationconditionstoevaluateMobiSpectral.OurresultsshowthatMobiSpectralcanidentifyorganicfoods,e.g., apples,tomatoes,kiwis,strawberries,andblueberries,with anaccuracyofupto94%fromimagestakenbyphones. | Hyperspectralimagingsystemscaptureinformationinmultiplewavelengthbandsacrosstheelectromagneticspectrum. Thesebandsprovidesubstantialdetailsbasedontheoptical propertiesofthematerialspresentinthecapturedscene. Thehighcostofhyperspectralcamerasandtheirstrictilluminationrequirementsmakethetechnologyoutofreach forend-userandsmall-scalecommercialapplications.We proposeMobiSpectral,whichturnsalow-costphoneintoa simplehyperspectralimagingsystem,withoutanychanges inthehardware.Wedesigndeeplearningmodelsthattake regularRGBimagesandnear-infrared(NIR)signals(which areusedforfaceidentificationonrecentphones)andreconstructmultiplehyperspectralbandsinthevisibleandNIR rangesofthespectrum.Ourexperimentalresultsshowthat MobiSpectralproducesaccuratebandsthatarecomparable toonescapturedbyactualhyperspectralcameras.Theavailabilityofhyperspectralbandsthatrevealhiddeninformation enablesthedevelopmentofnovelmobileapplicationsthat arenotcurrentlypossible.Todemonstratethepotentialof MobiSpectral,weuseittoidentifyorganicsolidfoods,which isachallengingfoodfraudproblemthatiscurrentlypartially addressedbylaborious,unscalable,andexpensiveprocesses. Wecollectlargedatasetsinrealenvironmentsunderdiverse illuminationconditionstoevaluateMobiSpectral.OurresultsshowthatMobiSpectralcanidentifyorganicfoods,e.g., apples,tomatoes,kiwis,strawberries,andblueberries,with anaccuracyofupto94%fromimagestakenbyphones. | ||
Revision as of 14:46, 21 September 2023
A hyperspectral camera captures a scene in many frequency bands across the spectrum, providing rich information and facilitating numerous applications in industrial, military, and commercial domains. Example applications of hyperspectral imaging include medical diagnosis (e.g., early detection of skin cancer), food-quality inspection, artwork authentication, forest monitoring, material identification, and remote sensing. The potential of hyperspectral imaging has been established for decades now. However, to date, hyperspectral imaging has only seen success in specialized and large-scale industrial and military applications. This is because of three main challenges: (i) the sheer volume of hyperspectral data which makes it hard to transmit such data in real-time and thus limiting its usefulness for many applications, (ii) the negative impact of the environmental conditions (e.g., rain, fog, and snow) which reduces the utility of the captured hyper-spectral data, and (iii) the high cost of hyperspectral cameras (upwards of $20K USD) which makes the technology out of reach for many commercial and end-user applications. The goal of this project is to address these challenges to enable wide adoption of hyperspectral imaging in many applications.
MobiSpectral: Hyperspectral Imaging on Mobile Devices Hyperspectralimagingsystemscaptureinformationinmultiplewavelengthbandsacrosstheelectromagneticspectrum. Thesebandsprovidesubstantialdetailsbasedontheoptical propertiesofthematerialspresentinthecapturedscene. Thehighcostofhyperspectralcamerasandtheirstrictilluminationrequirementsmakethetechnologyoutofreach forend-userandsmall-scalecommercialapplications.We proposeMobiSpectral,whichturnsalow-costphoneintoa simplehyperspectralimagingsystem,withoutanychanges inthehardware.Wedesigndeeplearningmodelsthattake regularRGBimagesandnear-infrared(NIR)signals(which areusedforfaceidentificationonrecentphones)andreconstructmultiplehyperspectralbandsinthevisibleandNIR rangesofthespectrum.Ourexperimentalresultsshowthat MobiSpectralproducesaccuratebandsthatarecomparable toonescapturedbyactualhyperspectralcameras.Theavailabilityofhyperspectralbandsthatrevealhiddeninformation enablesthedevelopmentofnovelmobileapplicationsthat arenotcurrentlypossible.Todemonstratethepotentialof MobiSpectral,weuseittoidentifyorganicsolidfoods,which isachallengingfoodfraudproblemthatiscurrentlypartially addressedbylaborious,unscalable,andexpensiveprocesses. Wecollectlargedatasetsinrealenvironmentsunderdiverse illuminationconditionstoevaluateMobiSpectral.OurresultsshowthatMobiSpectralcanidentifyorganicfoods,e.g., apples,tomatoes,kiwis,strawberries,andblueberries,with anaccuracyofupto94%fromimagestakenbyphones.
People
- Neha Sharma
- Puria Azadi
- Mohammad Amin Arab
- Kiana Calagari
- Mohamed Hefeeda
Code and Datasets
- MobiSpectral: Hyperspectral Imaging on Mobile Devices
- Enabling Hyperspectral Imaging in Diverse Illumination Conditions for Indoor Applications
- Hyperspectral Reconstruction from RGB Images for Vein Visualization
- Band and Quality Selection for Efficient Transmission of Hyperspectral Images
Publications
- N. Sharma, M. Waseem, S. Mirzaei, and M. Hefeeda, MobiSpectral: Hyperspectral Imaging on Mobile Devices, In Proc. of ACM Conference on Mobile Computing and Networking (MobiCom'23), Madrid, Spain, October 2023. (Artifacts Evaluated and Functional)
- P. Moghadam, N. Sharma, M. Hefeeda, Enabling Hyperspectral Imaging in Diverse Illumination Conditions for Indoor Applications, In Proc. of ACM Multimedia Systems Conference (MMSys’21), Istanbul, Turkey, September 2021. (Artifacts Evaluated and Functional)
- N. Sharma, M. Hefeeda, Hyperspectral Reconstruction from RGB Images for Vein Visualization, In Proc. of ACM Multimedia Systems Conference (MMSys'20), Istanbul, Turkey, June 2020. (Artifacts Evaluated and Functional)
- M. Arab, K. Calagari, M. Hefeeda, Band and Quality Selection for Efficient Transmission of Hyperspectral Images, In Proc. of ACM Conference on Multimedia (MM'19), Nice, France. October 2019.
