This book presents how metasurfaces are exploited to develop new low-cost single sensor based multispectral cameras. Multispectral cameras extend the concept of conventional colour cameras to capture images with multiple color bands and with narrow spectral passbands. Images from a multispectral camera can extract significant amount of additional information that the human eye or a normal camera fails to capture and thus have important applications in precision agriculture, forestry, medicine, object identifications, and classifications. Conventional multispectral cameras are made up of multiple image sensors each externally fitted with a narrow passband wavelength filters, optics and multiple electronics. The need for multiple sensors for each band results in a number of problems such as being bulky, power hungry and suffering from image co-registration problems which in turn limits their wide usage. The above problems can be eliminated if a multispectral camera is developed using one single image sensor.
A conventional color imaging system provides high resolution spatial information and low resolution spectral data. In contrast, a multispectral imaging system is able to provide both the spectral and spatial information of a scene in high resolution. A multispectral imaging system is complex and it is not easy to use it as a hand held device for acquisition of data in uncontrolled conditions. The use of multispectral imaging for computer vision applications has started recently but is not very efficient due to these limitations. Therefore, most of the computer vision systems still rely on traditional color imaging and the potential of multispectral imaging for these applications has yet to be explored.With the advancement in sensor technology, hand held multispectral imaging systems are coming in market. One such example is the snapshot multispectral filter array camera. So far, data acquisition from multispectral imaging systems require specific imaging conditions and their use is limited to a few applications including remote sensing and indoor systems. Knowledge of scene illumination during multispectral image acquisition is one of the important conditions. In color imaging, computational color constancy deals with this condition while the lack of such a framework for multispectral imaging is one of the major limitation in enabling the use of multispectral cameras in uncontrolled imaging environments.In this work, we extend some methods of computational color imaging and apply them to the multispectral imaging systems. A major advantage of color imaging is the ability of providing consistent color of objects and surfaces across varying imaging conditions. In this work, we extend the concept of color constancy and white balancing from color to multispectral images, and introduce the term multispectral constancy.The validity of proposed framework for consistent representation of multispectral images is demonstrated through spectral reconstruction of material surfaces from the acquired images. We have also presented a new hyperspectral reflectance images dataset in this work. The framework of multispectral constancy will make it one step closer for the use of multispectral imaging in computer vision applications, where the spectral information, as well as the spatial information of a surface will be able to provide distinctive useful features for material identification and classification tasks.
This document shows the current state of the research work done around the SmartSpectra project. The SmartSpectra project is a Research, Technological development and Demonstration (RTD) project funded under EU s Fifth Framework Programme (FP5) by the Information Society Technologies (IST) Programme. The project pursues the development of a Smart Multispectral System for Commercial Applications. SmartSpectra is an acronym of "Smart Multispectral System for Commercial Applications." In this project, a Smart Multispectral System will be designed and implemented. The system will allow multispectral imaging with an aff ordable cost and proven robustness, in order to achieve a broad use of multispectral techniques in several commercial areas and applications. The system will have the capability to be integrated in currently established production systems. Moreover,it will be flexible enough to be applicable to a wide range of applications. The document is organized as follows: In the introductory chapter, we present the SmartSpectra project, emphasizing the parts in which we are involved. Next chapter is devoted to explain the concept of the SmartSpectra camera, defining its specifications. Chapters 3 and 4 detail the realised work regarding Workpackages 2 and 3 of the project. Workpackage 2 deals with the optical and sensing part of the SmartSpectra system. It summarizes the state of the art in VIS & NIR technologies, lists the purchased sensors for the project and describes the AOTF (Acousto-Optic Tunable Filter) technology. Workpackage 3 explains the sensor electronics and interface. Two different prototypes of the SmartSpectra camera are described, along with the Firewire subsystem. These chapters are followed by a Progress Review and Future Work. Last chapter is a final summary of the work. The document ends with a group of annexes showing the outcomes of the work.
What is Multispectral Imaging Multispectral imaging captures image data within specific wavelength ranges across the electromagnetic spectrum. The wavelengths may be separated by filters or detected with the use of instruments that are sensitive to particular wavelengths, including light from frequencies beyond the visible light range, i.e. infrared and ultra-violet. It can allow extraction of additional information the human eye fails to capture with its visible receptors for red, green and blue. It was originally developed for military target identification and reconnaissance. Early space-based imaging platforms incorporated multispectral imaging technology to map details of the Earth related to coastal boundaries, vegetation, and landforms. Multispectral imaging has also found use in document and painting analysis. How you will benefit (I) Insights, and validations about the following topics: Chapter 1: Multispectral imaging Chapter 2: Infrared Chapter 3: Remote sensing Chapter 4: Thermographic camera Chapter 5: Satellite imagery Chapter 6: Spectral signature Chapter 7: Spectral imaging Chapter 8: Hyperspectral imaging Chapter 9: Chemical imaging Chapter 10: Normalized difference vegetation index (II) Answering the public top questions about multispectral imaging. (III) Real world examples for the usage of multispectral imaging in many fields. Who this book is for Professionals, undergraduate and graduate students, enthusiasts, hobbyists, and those who want to go beyond basic knowledge or information for any kind of Multispectral Imaging.
Multispectral imaging is traditionally performed using a combination of an imaging device with a filter bank such as a filter wheel or a form of tunable filter, or a combination of many imaging devices with various spectral beam splitting optics. The complexity and size of these devices seem to be the limiting factor of their adoption and use in various fields that could potentially benefit from this imaging modality. With the advent of nanophotonics, there has been a surge in single camera, snapshot, multispectral imaging exploiting the capabilities of nanotechnology to devise pixel-based spectral filters. This new form of sensing, which can be classified as compressive sensing, has its limitations. The work presented here will describe an optical design that would enable a single-camera, simultaneous multispectral imaging via multiview computational compressive sensing.
This book provides a cutting-edge research overview on the latest developments in the field of Optics and Photonics. All chapters are authored by the pioneers in their field and will cover the developments in Quantum Photonics, Optical properties of 2D Materials, Optical Sensors, Organic Opto-electronics, Nanophotonics, Metamaterials, Plasmonics, Quantum Cascade lasers, LEDs, Biophotonics and biomedical photonics and spectroscopy.
Considered a major field of photonics, plasmonics offers the potential to confine and guide light below the diffraction limit and promises a new generation of highly miniaturized photonic devices. This book combines a comprehensive introduction with an extensive overview of the current state of the art. Coverage includes plasmon waveguides, cavities for field-enhancement, nonlinear processes and the emerging field of active plasmonics studying interactions of surface plasmons with active media.
This book presents a comprehensive overview of state-of-the-art quantum dot photodetectors, including device fabrication technologies, optical engineering/manipulation strategies, and emerging photodetectors with building blocks of novel quantum dots (e.g. perovskite) as well as their hybrid structured (e.g. 0D/2D) materials. Semiconductor quantum dots have attracted much attention due to their unique quantum confinement effect, which allows for the facile tuning of optical properties that are promising for next-generation optoelectronic applications. Among these remarkable properties are large absorption coefficient, high photosensitivity, and tunable optical spectrum from ultraviolet/visible to infrared region, all of which are very attractive and favorable for photodetection applications. The book covers both fundamental and frontier research in order to stimulate readers' interests in developing novel ideas for semiconductor photodetectors at the center of future developments in materials science, nanofabrication technology and device commercialization. The book provides a knowledge sharing platform and can be used as a reference for researchers working in the fields of photonics, materials science, and nanodevices.
Due to recent advancements in technology, consumer digital cameras are becoming cheaper and easier to use. These consumer digital cameras, with Bayer color filter arrays (CFAs), allow for simultaneous capture of the red, green and blue (RGB) channels. To achieve higher spectral resolution, multispectral imaging systems use methods such as filter wheels and tunable filters to capture data in a sequential manner. However, in order to capture transient phenomena, one would need to capture spectral information of a 2D scene in a simultaneous manner. Therefore, there has been an on-going trend towards creating a simultaneous multispectral imaging system that uses a conventional consumer digital camera with a Bayer CFA. Such a system allows for a effective imaging of transient or dynamic phenomena with a low-cost and compact system. Currently, the main method to accomplish this is known as Wiener estimation which uses statistical assumptions of the relationship between the incoming spectra and the RGB measurements. However, these assumptions limit the ability to accurately predict the incoming spectra. Therefore, we leverage a comprehensive framework based on numerical demultiplexing of sensor measurements via spectral characterization of the image sensor CFA and non-linear random forest modeling. To create this numerical demultiplexing system we create a forward model from the spectral sensitivity of the imaging system, which is accomplished with a monochrometer. This forward model is then used to create a mapping of 10,000 randomly generated spectra to their corresponding RGB values. This mapping acts as our training set for our non-linear inverse model which utilizes the random forest modeling framework. Having constructed the numerical demultiplexer, we test the performance against the state-of-the-art Wiener estimation for both quantitative and qualitative experiments. In the first set of experiments, we performed a quantitative performance assessment of the proposed framework within a controlled simulation environment. The second set of experiments, validated the observations made from the first set of controlled simulation experiments within a real-world setting. More specifically, we used an icon with different colors as well as a scene of different color flowers to perform quantitative analysis. In these experiments, we show that the proposed numerical demultiplexer outperforms the state-of-the art and is a more robust and reliable way to infer higher spectra from RGB measurements. Having validated the numerical demultiplexer, we use it for two applications which are photoplethysmogrpahic imaging and multispectral microscopy. For photoplethysmogrpahic imaging we found that decomposing the RGB camera measurements into narrow-band spectral information can noticeably improve the prediction of heart rate estimation. In addition, we used the numerical demultiplexer for both a bright-field multispectral microscope as well as a dark-field fluorescence multispectral microscope, which illustrates its potential as a low-cost, portable, point-of-care system.
