Search

Remote Photoplethysmography (remote PPG) enables contactless monitoring of the cardiac rhythm using video cameras. Prior research has shown the feasibility of video-based atrial fibrillation (AF) and/or flutter (Aflutter) detection in some scenarios, but most exclude patient movement. In this work, we investigate the feasibility of detecting these two cardiac arrhythmias in a regular hospital environment using an RGB camera, where patients were not limited in movement during the recording process. Data of 56 patients was collected before and after a scheduled cardioversion treatment. Using the data and machine learning models, we developed three models: First, a model to detect only AF from the data excluding any Aflutter cases. Here we report a sensitivity of 94.5% and a specificity of 89.3% with an AUC of 0.966. Second, a model to classify if a cardiac arrhythmia (AF or Aflutter) is present or not. There we report there a sensitivity of 95.6% and a specificity of 91.2% with an AUC of 0.975. Finally, we develop a multi rhythm model, where we classify the data in AF, Aflutter and sinus rhythm separately. The performance of arrhythmia detection is close to the second model, but we note that the distinction between AF and Aflutter is still a challenge. Here we theorize that remote PPG is more sensitive to noise during Aflutter, which will lead to features in Aflutter which are closer to those of AF. To confirm this, we will extensively review the reason of misclassification of Aflutter as AF in future work.

For augmented reality applications, the micro display panels are generally used, as a straightforward method for display engines. But there are the light efficiency and vergence accommodation conflict problems, which affect the performance and application of the AR glasses, HMD and HUDs. One way to solve the problem is to use holography display instead of direct display, which exploit the diffraction light instead of reflection light of the liquid crystal panel. One of the big problems is the form factor of holographic systems which use SLM. Most of current SLMs are composed of a driver board with FPGA and a liquid crystal on silicon panel. The FPGA driver board is big and very power consuming, which makes it better for general SLM systems that cares more about changing the driving algorithm flexibly rather than make it a wearable gadget. A compact SLM system is designed specifically for AR applications, which replaced the normally used FPGA system with a driver IC and greatly reduce they whole system size and power consumption while realize the same function for holographic display. The driver IC could receive the RGB signal or MIPI signal and transform the standard picture signal to a high frame rate color sequential signal, also the gray scale could be tuned through on chip registers either beforehand or instantly, which makes it good for liquid crystals working in different wavelength. The driver board is 2*6 cm2 with a type-C connector which could easily transmit image data from a laptop or a cellphone. This form factor is almost the same with monocular glasses. The power consumption of the driver IC is around 250mW for the driver IC comparing to the normally 5W power of FPGAs. Also, the frame rate could be from 60 to 120 Hz. A simple Fourier holography optical system is used to demonstrate the holographic display, with a lens tear apart from a smart glass on the market. The 3D holographic is displayed in the lens.

In recent years, 3D vision robot picking has been widely used in the fields of manufacturing, logistics, and warehouse due to its flexibility in handling various models. In this paper, we propose a robust and precise 3D vision guided robot picking method using multi-channel image information fusion. The major challenges are the failure or inaccurate of object localization and pose estimation caused by environment. Various considerations, we utilize a 3D camera based on binocular structed light to obtain the high accuracy RGB-D image with an additional customized illumination unit. In addition, we introduce the fusion method using multi-exposure control and multi-channel model from multi-color images to establish the reliable model matching for all conditions. Our system calculates the location and pose of the stove according to the rectangular template and the normal vector of the bottom surface to realize visual guidance. By projecting the point cloud onto the stove plane, only the height information is retained to generate a pseudo-image. Our 3D vision system reaches ≥99.8% successful pickup rate with the ±2mm accuracy along X, Y, and Z directions and±0.2°rotational accuracy along RX, RY, and RZ for both bare and painted surfaces under various illumination conditions during production. Our stove-picking systems installed in the welding and assembly stations improve the flexibility and the efficiency of the stove production line (100,000 unit/year).

Nighttime clouds are detected using a camera without a NIR-cut filter by exploring the pixel value distribution in each red, green, and blue (RGB) color space from clear and 100% cloudy sky images. The removal of the NIR-cut filter enhances the pixel values of each color space due to the additional NIR signals. Cloud pixels can be separated from clear sky pixels by only using each color space and applying the appropriate threshold. Exploring different cloud detection methods is vital in our ongoing research activity collecting clear and cloudy sky images in other countries using this camera.

The defect inspection in traditional metal processing usually adopts the manual visual inspection, which is easy to cause problems such as false detection and leakage. Besides, the detection rate of existing metal defect inspection has been affected by the high reflectivity of metal surface. For avoiding the problems of existing detection algorithms, a metal defect detection system based on laser triangulation and laser vision is given. The system is composed of industrial camera and strip light source. In order to avoid the interference of metal reflection on defect detection, the green strip light is used as the light source of the system. Based on the proposed system, this paper presents a metal surface detection algorithm with three primary color channels, that is, metal surface defects are identified by identifying the RGB values of metal surface pixels. Finally, through static experiments, the detection rate of the metal defect surface detection system based on laser vision is 100%. The defect detection method proposed in this paper can effectively avoid the influence of high reflectivity of metal surface on the detection rate, and can be widely used in the field of metal processing.

The harmonic sawtooth microstructure allows the diffractive lens to operate with polychromatic radiation, in particular, with the radiation of RGB LEDs, but only with restrictions due to axial color and specific field curvature. Its reason is the jump in the working diffraction order with an increase of the wave angle of incidence. The permissible wave angles of incidence at the microstructure were estimated using the example of a harmonic diffractive lens made of crown-like plastic E48R.

Compared with images taken in the air, underwater images are susceptible due to the adverse effects of complex water conditions, such as absorption, attenuation, scattering, various noises, and uneven lighting, etc. These effects severely reduce the quality of underwater imaging and give rise to many problems among which is image color distortion. To fully acknowledge the underwater scenes, it is important to acquire the images with accurate colors, especially for marine rescue, underwater target recognition, and marine ecological research. To address this problem, this paper proposes an underwater imaging technology based on a light field camera array. The application of this technology can not only perform accurate color correction processing on the captured images but also expand the dynamic range of underwater images, thereby obtain enhanced high-quality, true-color underwater images. The experimental system in this paper contains a 3*3 camera array that simultaneously captures images of underwater targets with different exposure times. As the light attenuation coefficient of different wavelengths in water varies with the wavelength, during the color restoration process, we measure the relative value of the underwater spectral power distribution concerning the air, calculate the tristimulus value change according to the colorimetric principle, and convert the tristimulus value change in the CIEXYZ color space to the CIE La*b* color space. The color on the underwater images obtained by the camera array is then compensated. Then, in the image fusion process, images with different exposure are used to recover the response function of the imaging process, and multiple images are fused into a single, high-dynamic-range radiance map using high quality tone mapping technique in the radiance value dynamic display high-contrast images on devices with limited range. Experimental results show that our light-field imaging color correction and dynamic range expansion processing methods have obvious advantages over single cameras in underwater color correction and underwater moving target imaging detection, which provide advanced technology and equipment for underwater in-situ applications.

The target tracking method based on RGB image is affected by many factors such as light and haze weather, so it is difficult to distinguish the tracking target from the background, and it is easy to lead to the drift or even loss of the tracking target. Target tracking based on infrared images is not affected by light, haze and other illumination factors, but the target's color, texture and other characteristic information will be missing. Therefore, in order to obtain the target's color, texture and other characteristic information in a poorly illuminated environment, while achieving accurate and fast tracking of the object, this paper proposes a RGB and infrared image fusion tracking method based on a deep convolutional network. Firstly, the fusion method of RGB and infrared images is studied; secondly, a target tracking network based on the Siamese network is established to extract the image convolution features of the target template and the current target; finally, the response map is calculated by the deep cross-correlation module. At last, the performance test of the target tracking algorithm is carried out on the VOT2019RGBT data set. Experimental results show that the algorithm can effectively solve the problem of target tracking when the target is partially occluded, the tracking scene has no suitable illumination or the light changes strongly, and it is of great significance to improve the accuracy of target tracking under complex backgrounds.

The intensity of flowering of the holm oak trees is important for the annual phenological monitoring and as a predictive index of final acorn production. Their male flowers present in long catkins of intense yellow color and the estimation of their abundance in the field is a time-consuming task that becomes unfeasible at a large scale. In this work, a methodology to estimate the intensity of flowering of oak trees using RGB (Red Green Blue) images, provided by an unmanned aerial vehicle, was tested. During the spring of 2019, three aerial zenith images of 3 cm spatial resolution were taken in three selected dehesa sites, together with simultaneous ground digital photographs per tree (50 at each site). The intensity of flowering was visually estimated using the ground digital photographs in three categories, ranging from 1 (little or no flowering) to 3 (high flowering). A simple flowering intensity index, based on the closeness to pure yellow within a Cartesian RGB space, was developed to check the relationship between the drone images and the visually analyzed photographs. The results showed that those trees with lower flowering intensity were grouped in higher yellow distances and the high flowering intensity trees in the lower ones. As a result, it can be concluded that this index was able to identify qualitatively the flowering intensity of holm oaks at the farm level and could be useful for future phenological or productivity applications.

Images acquired by RGB cameras on Unmanned Aerial Vehicles (UAVs) can be particularly useful to detect crowd in urban areas when restrictive conditions are imposed for the sake of public safety or health , such as during the Covid 19 pandemic. Together with acquired images, opportune pattern recognition techniques have to be considered to extract useful information. In this framework, features capturing the semantic rich information encompassed in VHR images have to be computed. In particular, Deep Neural Networks (DNN) have been recently proved to be able to extract useful features from data [1]. Moreover, in a transfer learning approach, a DNN pre-trained on a data set can be used to extract opportune features, named deep-features, from another data set, belonging to a different applicative domain [1], [2]. Here, a transfer learning technique is presented to produce change maps, detecting how people gathering increases, from VHR images. It is based on deep-features computed by using some pre-trained convolutional layers of AlexNet. The proposed methodology has been tested on a data set composed of several synthetic VHR images, that simulate crowd collecting in a park, as they can be acquired by RGB camera on a UAV flying at 10 meters height from the ground. The experimental results show that the proposed technique is able to efficiently detect change due to new people incoming in the scene or people that get away, with a low computational cost and in a near-real time operative mode.

Colored light for visual communication in cars at daylight conditions has recently started: A light guide with RGB LEDs is placed at the bottom of the windshield. Usage is safe visualization of autonomous mode and warnings beside e.g. wayfinding support, directional highlighting and turn signals. Today’s white luminance of light guides is about 1,250 cd/m2 for daylight with a RGB luminance ratio of typically 28:65:7. A new proposal from an OEM requires the same luminance just for blue. Therefore we started an evaluation with different aspects as basis: The luminance of blue is low due to V(λ) curve but its color perception is high by Color Matching Functions, the lightness L* (perceived brightness) is related to luminance L by L* ~ L0.44 and the RGB luminance ratio of traffic signs (acc. EN12966) is 35:50:15. We designed our car mock-up for a white luminance of 13,000 cd/m² using a RGB LED stripe reaching 1,500 cd/m2 for blue. The surrounding simulates night to sunlight conditions with additional 100,000 cd/m2 blinding sun. We tested 18 subjects (pandemic restrictions). The luminance of blue for “annoying” (= safe recognition) reached 900 cd/m2 (sunlight + blinding). Additionally, the subjects had to adjust two neighboring colors to the same brightness or judge (brighter, same, and darker) for different colors. Our survey resulted in a reasonable luminance of 500 cd/m2 for blue (emphasized by blinking) and a RGB luminance ratio of 35:50:15. The portion of blue is nearly doubled which forces a redesign of today’s automotive RGB LEDs.

This research focuses on the possibility of building an alternative mixed reality (MR) system, which will eliminate all its main causes of visual discomfort and form a model of the real world in the virtual space, as much as possible corresponding to it. The relationship between virtual reality (VR) systems, which are limited to models of their own virtual world, and MR systems, which add virtual objects to the real world, is examined. This paper presents an approach based on generating a point cloud of static objects using RGB-D sensors of the MR device, its further classification and segmentation, and then searching for a similar CAD object in the appropriate database. The found virtual analogue, after appropriate transformations, replaces the real object of the scene. To create more realism, RGB images of real objects can be superimposed as textures on the corresponding virtual scene objects. The paper proposes a multimodal approach, which consists in searching for objects with similar modalities in databases. A virtual scene created in a single space using this approach eliminates the possibility of forming unnatural lighting and observation conditions for all objects, including virtual copies of real objects and added virtual objects.

This paper shows three experiments from our HyperGreding’19 campaign that combine multi-temporal hyper�spectral data to address several essential questions in target detection. The experiments were conducted over Greding, Germany, using a Headwall VNIR/SWIR co-aligned sensor mounted on a drone with a flight altitude of 80 m. Additionally, high-resolution aerial RGB data, GPS measurements, and reference data from a field spectrometer were recorded to support the hyperspectral data pre-processing and the evaluation process for the individual experiments. The focus of the experiments is the detectability of camouflage materials and camou�flaged objects. When the goal is to transfer hyperspectral analysis to a practical setting, the analysis must be robust regarding realistic and changing conditions. The first experiment investigates the SAM and the SAMZID approaches for change detection to demonstrate their usefulness for target detection of moving objects within the recorded scene. The goal is to eliminate unwanted changes like shadow areas. The second experiment eval�uates the detection of different camouflage net types over two days. This includes camouflage nets in shadows during one flight and brightly illuminated in another due to varying solar elevation angles during the day. We demonstrate the performance of typical hyperspectral target detection and classification approaches for robust detection under these conditions. Finally, the third experiment aims to detect objects and materials behind the cover of camouflage nets by using a camouflage garage. We show that some materials can be detected using an unmixing approach.

Due to an increased occurrence frequency of drought events and pest infestations, large amounts of deadwood are a current issue in temperate forests. Accurate monitoring of deadwood and analysis of its spatial and temporal distribution is, therefore, more important than ever, as it facilitates faster response to pest outbreaks or increased risk of forest fires. As highlighted by previous studies, state-of-the-art remote sensing platforms, such as UAVs, provide great synergies for deadwood monitoring in combination with machine- and deep-learning approaches of computer vision. Key challenges that remain are the acquisition of sufficient amounts of labeled data for model training and identifying deadwood on the single-tree level, which is required to estimate the deadwood volume in an area. The presented work demonstrates how it is possible to obtain very accurate instance segmentation in the combined RGB and elevation domain and with limited training data. A high-performance Mask R-CNN model was trained to map standing and lying deadwood instances in German forests, achieving outstanding results with an overall accuracy of 92.4% and a mean average precision of 43.4%. To compensate for the possibly insufficient amount of annotated images, we performed experiments with a semi-supervised active learning pipeline. Here, each time after the model predicted a batch of new data, only the instances that achieved a high prediction score were added to the pool of training data to re-compute the model for the next iteration step. Even though the application of the fully supervised approach led to superior results, overall, this study proves that the proposed method can reliably map individual deadwood objects. The approach not only represents an end-to-end framework for image annotation, model acquisition, and large-scale mapping of deadwood, but also is adoptable with reasonable effort to solve similar problems in the future.

The Philippines used to be one of the prime exporters of coffee from different parts of the world. However due to lack of technology and the absence of standard the production and exportation of coffee diminish through the years. Until now, the coffee farmers are relying on manual operations of classifying and profiling coffee beans intended to level and match the global standard. Hence, the researchers created a system that will automatically classify and profile coffee beans without human intervention based on the different features of coffee beans using integrated image processing algorithms. The focus of this research is to create a device that can evaluate the size, quality, and roast level of a batch of the coffee beans through the use of image processing techniques and Back Propagation Neural Network. To determine these features, BPNN would serve as the method to develop the brain of the device. The integrated processing algorithms used in this research include K-mean shift, Blob, and Canny Edge to extract the features of the coffee beans and Red Green Blue Analysis, Hu's Moment, and Blob Analysis to make use of these features and feed it into the BPNN. Based on the standard set by the Philippine Coffee Board Inc., the prototype in this research was able to classify and profile different coffee beans with up to 100% accuracy.

We propose quad-layered transmissive structural color filters that can generate RGB primary colors with high purity and high brightness by utilizing interferences in dual Fabry-Perot (FP) cavities. Since there is a trade-off between color purity and brightness in a conventional single FP cavity, a peak separation in multiple FP cavities is exploited to achieve a more square-shaped spectral curve. Besides, controlling a resonance order in each cavity leads to a great suppression of a higher-order resonance for a red color filter. The presented results may be applied to various applications including display panels, decorations, and image sensors.

This paper presents the implementation of a simultaneous localization and mapping (SLAM) algorithm for autonomous mobile robot navigation. The proposed implementation uses an RGB-D camera to detect the environment and map an occupancy grid that allows the mobile robot to perform autonomous navigation through the environment. The implementation employs the Robot Operating System (ROS) and the Adaptive Monte Carlo Localization to estimate the mobile robot’s current position in the environment with the data retrieved from the RGB-D camera and the odometry data. The mobile robot performs autonomous navigation considering if the robot can safely navigate while avoiding obstacles. Experimental results are presented to validate the implementation.

With the continuous development of remote sensing technology, the resolution of remote sensing image is gradually improving, and the information contained in high resolution remote sensing image is also more rich, which is widely used in various fields. The extraction of buildings using high resolution remote sensing image has also become one of the research hotspots. In order to solve the problems of incomplete or missing buildings detected by traditional building detection algorithms in high resolution remote sensing images, and the similarity of spectral features between buildings and roads and bright soil is difficult to distinguish. The purpose of this study is to develop an effective building extraction algorithm based on improved adaptive MBI index based on GF-2 satellite data. First radiation calibration in remote sensing image preprocessing, such as the original image from the RGB space transformation to do after YCbCr space information, and to adaptive image enhancement processing, feature extraction with adaptive body mass index, and using vegetation index set threshold value of buildings and shadow region, the preliminary extraction result post-processing of the building, So as to get the final extraction result. Compared with the traditional building morphological index MBI algorithm, the error of error and error of missing classification are reduced, and the overall accuracy is improved by about 20%.

Digital photoelasticity is used for evaluating the stress in loaded bodies. However, when dynamic analyses are needed, the motions of optical elements are an experimental challenge. This new computational hybrid approach calculates the stress field by extract the phase steps from RGB color channels of a photoelastic color image. Our approach integrated the load stepping strategy with a computational hybrid phase algorithm, hence only bright field images are required. Although, our method has a lower performance than phase shifting methods evaluated, the principal advantage of this hybrid strategy is that only a color- image is required to analyze stress field, avoided capture multiple images for analyzing phase maps.

Visual SLAM is widely known in robotics for computing, concurrently, the odometry of a robot and construct a 3D navigation map with only a camera. In visual SLAM systems, detection and description of local features are extremely important because they identify unique and invariant points in an observed frame. Although there are various detectors and descriptors, the proper detector/descriptor combination for extraction has not yet been generalized for the problem. In this work, a comprehensive performance evaluation of combinations for different feature detectors and descriptors is presented. This evaluation will help determine the best detector/descriptor combination for designing a visual SLAM system based on RGB-D data. The considered methods are evaluated in terms of accuracy and robustness in both, a single and overall visual SLAM system.

We present dual-cavity based reflective structural color filters that can produce RGB additive colors featuring high purity and high efficiency. We decided to design a filter with a new wavelength by combining two classical MDM structure Fabry-Perot cavities with different resonant wavelength. A cavity medium with high refractive index leads to angle-invariant performance up to 50˚. Moreover, only deposition is involved so that the device can be easily scaled over a large area. The described approach may provide new avenues for various applications, such as reflective displays and surface coatings for decoration and solar cells.

In view of the fusion of SAR image and RGB image, different methods based on wavelet transform (WT) were studied. Based on WT, SAR images are fused with different channel of RGB images respectively. In order to suppress the speckle in SAR images, Speckle Reducing Anisotropic Diffusion (SRAD) and nonlocal mean filter (NLM) were introduced respectively. The image fusion effect was comprehensively measured by using several image quality indexes. The results show that WT and NLM are more helpful to improve the quality of image fusion.

In this paper, an approach for the estimation of relative positions and orientations of multiple rigidly coupled RGB-D sensors is presented. The proposed methodology does not impose constraints on the relative pose among the cameras, except for the rigidity: relative poses among cameras do not change in time. We assume that each camera captures an individual sequence which is afterward processed by a Simultaneous Localization And Mapping (SLAM) algorithm resulting in positions and orientations describing the trajectory followed by each sensor. Several simulations have been run considering a two-camera set having known input trajectories, affected by increasing levels of white noise. The results prove the capability of the algorithm in estimating the relative translations and rotations between the two cameras. Furthermore, experimental laboratory tests corroborate the effectiveness of the proposed method.

Bayfol® HX photopolymer films prove themselves as easy-to-process recording materials for volume holographic optical elements (vHOEs) and are available in customized grades at industrial scale. Their full-color (RGB) recording and replay capabilities are two of their major advantages. Moreover, the adjustable diffraction efficiency, tunable angular and spectral selectivity of vHOEs recorded into Bayfol® HX as well as their unmatched optical clarity enables superior invisible “off Bragg” optical functionality. As a film product, the replication of vHOEs in Bayfol® HX can be carried out in a highly cost-efficient and purely photonic roll-to-roll (R2R) process. Utilizing thermoplastic substrates, Bayfol® HX was demonstrated to be compatible to state-of-the-art plastic processing techniques like thermoforming, film insert molding and casting, all enabled using a variety of industry-proven integration technologies for vHOEs. Therefore, Bayfol® HX made its way in applications in the field of augmented reality such as Head-up-Displays (HUD) and Headmounted- Displays (HMD), in free-space combiners, in plastic optical waveguides, and in transparent screens. Also, vHOEs made from Bayfol® HX are utilized in highly sophisticated spectrometers in astronomy as well as in narrow band notch filters for eyeglasses against laser strikes. Based on a well-established toolbox, Bayfol® HX can be adopted for a variety of applications. To further offer access to more applications in sensing and continuously improve the performance in existing applications, we recently extended our chemical toolbox to address both the sensitization beyond RGB into the Near Infrared Region (NIR) and increase the achievable index modulation Δn1 beyond 0.06. In this paper, we will report on our latest developments in these fields.

We demonstrate a miniaturized, full-color RGB light source module for near-to-eye display systems, incorporating three semiconductor laser diodes (LDs) that are integrated on a free-space, micro-optical bench together with collimation optics and wavelength filters. The ultra-compact package has a footprint of 4.4 mm x 4.15 mm with a height of 2.9 mm (0.053 cm3) and an optical output window for the collimated and collinearly aligned RGB beams. The light source module delivers up to 10 mW per color at low power dissipation values of 640 mW and provides low-divergent output beams having a high circularity and a diameter of 250-650 μm at a reference distance of 50 mm.

Achieving precise information on the position of a subject without changing the luminaire infrastructure is a big challenge in positioning approaches that rely on visible light positioning. Achieving high positioning accuracy on a centimeter scale is done by implementing complex receiver unit designs or adapting the existing luminaries. In this context, we suggest a visible light positioning based approach that can determine the position of a person in certain areas of a room without the need of lighting infrastructure modifications. With this approach, one can identify the position with the help of the existing luminaires for the obligatory room lighting. The receiver, represented by a RGB sensitive photodiode, is positioned in an optimized way in order to support both the positioning task as well as the comfort of the user. Based on received signal strength measurements in the red, green and blue channels, we achieve the positioning task by a segmentation of the room into different areas corresponding to the respective impinging light and by utilizing machine learning clustering. Our results show the influence of different segmentation strategies and parameters on the number and size of the distinguishable areas inside the room. Then, we demonstrate the achievable accuracy of our solution approach in real world experiments. Our results show that such light-based positioning data can be fused with IMU sensor data for recognizing human activity.

The classic Retinex algorithm assumes that the image illumination changes uniformly, and uses a Gaussian filter as the center/surround function to estimate the illumination component. However, there may be light jumps at the edges of the image, and blurring of details and edge halo will occur. when the Retinex algorithm processes color images, there are obvious color distortions. In response to the above problems, this paper improves Retinex by using gradient domain guided filtering and multi-scale detail enhancement algorithms. First, the image to be processed is converted to HSI color space, and gradient domain guided filtering is used as an estimation function to decompose the I channel image into brightness and reflection components, It is fused after brightness enhancement and denoising respectively, and then multi-scale detail enhancement of the fused image, and finally the image is converted from HSI space back to RGB space. The experimental results show that the proposed method can effectively enhance the texture details in the dark regions of the image while improving the image brightness, and outperforms other algorithms in terms of objective evaluation metrics

One of the common methods to increase data bitrate using visible light communication (VLC) fibers system is to use wavelength division multiplexing (WDM). However, the implementation of WDM requires additional components and more energy consumption which can limit the system performances. To overcome these issues, we propose a new method for designing an RGB multiplexer based on multicore polymer optical fiber (MC-POF). The new design is based on replacing seven air-holes areas with polycarbonate (PC) layers along the fiber length. The PC layer length sizes are suitable to the light coupling of the operating wavelengths which allows us to control the light switching between closer PC layers and to obtain an RGB multiplexer device without adding more devices. Results show that after a 20 mm light propagation the PC MC-POF RGB multiplexer can be obtained with a low power loss of 0.6 to 1.02 dB, large bandwidth of 7.3 to 28.4 nm and good isolation between the transmission of the input RGB channels.

Our goal is to develop stable, accurate, and robust semantic scene understanding methods for wide-area scene perception and understanding, especially in challenging outdoor environments. To achieve this, we are exploring and evaluating a range of related technology and solutions, including AI-driven multimodal scene perception, fusion, processing, and understanding. This work reports our efforts on the evaluation of a state-of-the-art approach for semantic segmentation with multiple RGB and depth sensing data. We employ four large datasets composed of diverse urban and terrain scenes and design various experimental methods and metrics. In addition, we also develop new strategies of multi-datasets learning to improve the detection and recognition of unseen objects. Extensive experiments, implementations, and results are reported in the paper., 12 pages, 5 figures

Target detection is one of the most important applications utilizing the rich spectral information from hyperspectral imaging systems. Data fusion algorithms applied on hyperspectral datasets address the inherent spatial-spectral resolution tradeoff in these imaging systems by combining spectral information from hyperspectral data with spatial information from hi-res panchromatic or multispectral images (e.g., hi-res RGB). This paper presents the first attempt at using an iterative target implantation technique as a modification to Wald's protocol to assess the performance of data fusion algorithms in target detection tasks. More specifically, this paper looks at how the sharpening process localizes and discriminates the subpixel target from its background, and characterizes an image-wide detectability of any single subpixel target independent of location in the image. We used NNDi use as our pansharpener to perform HRPAN+LRHSI data fusion and the adaptive coherence estimator (ACE) as our target detector. Results show that our methodology is effective at assessing (1) how the sharpening process enhances target-background separability within any 5x5 window anywhere on the image and (2) how the sharpening process enhances the detectability of a single subpixel target over the entire hyperspectral image.

This paper explores a multimodal deep learning network based on SqueezeSeg. We extend the standard SqueezeSeg architecture to enable camera and lidar fusion. The sensor processing method is termed pixel-block point-cloud fusion. Using co-registered camera and lidar sensors, the input section of the proposed network creates a feature vector by extracting information from a block of RGB pixels from each point in the point-cloud that is also in the camera’s field of view. Essentially, each lidar point is paired with neighboring RGB data so the feature extractor has more meaningful information from the image. This fusion method adds rich information on object color and texture from the camera data to enhance the overall performance. The image pixel blocks will not only add color information to the lidar data, but it will also add information about texture. The proposed pixel-block point-cloud fusion method yields better results than single-pixel fusion.

Estimating building height from satellite imagery is important for digital surface modeling while also providing rich information for change detection and building footprint detection. The acquisition of building height usually requires a LiDAR system, which is not often available in many satellite systems. In this paper, we describe a building height estimation method that does not require building height annotation. Our method estimates building height using building shadows and satellite image metadata given a single RGB satellite image. To reduce the data annotation needed, we design a multi-stage instance detection method for building and shadow detection with both supervised and semi-supervised training. Given the detected building and shadow instances, we can then estimate the building height with satellite image metadata. Building height estimation is done by maximizing the overlap between the projected shadow region given a query height and the detected shadow region. We evaluate our method on the xView2 and Urban Semantic 3D datasets and show that the proposed method achieves accurate building detection, shadow detection, and height estimation.

Automated and cost-effective phenotyping pipelines are needed to efficiently characterize new lines and hybrids developed in plant breeding programs. In this study, we employ deep neural networks (DNNs) to model individual maize plants using 3D point cloud data derived from unmanned aerial systems (UAS) imagery by PointNet network. The experimental setup was performed at the Indiana Corn and Soybean Innovation Center at the Agronomy Center for Research and Education (ACRE) in West Lafayette, Indiana, USA. On June 17th, 2020 a flight was carried out over maize trials using a custom designed UAS platform with a Sony Alpha ILCE-7R photogrammetric sensor. RGB images were processed by a standard photogrammetric pipeline by Structure from Motion (SfM) to reconstruct the study field into a final scaled 3D point cloud. 50 individual maize plants were manually segmented from the point cloud to train the DNN and subsequently individual plants were extracted over a test trial with more than 5,000 plants. Moreover, to reduce overfitting in the fully-connected layers, we employed data augmentation not only in translation, but also in color intensity. Results show a successful rate for the extraction of the individual plants of 72.4%. Our test trial demonstrates the possibility of using deep learning to overcome the individual maize extraction challenge on the basis of UAS data.

AR Smart Glasses need to be comparable to normal glasses in respect of size, weight and esthetics. By using Laser Beam Scanning (LBS) the optical engine i.e. the image projector can be miniaturized down to sizes of 1cc only, allowing smart integration into the glasses temples. In contrast to 2D microdisplays like LCoS, DLP or µOLED the size of the LBS optical engine is not directly scaling with display performance i.e. luminance, field of view and angular resolution. So far single laser packages of TO-can type have been used for each color red, green and blue. We present a compact package containing the laser diodes chips of all three colors. This RGB laser module combines all the necessary features for a robust and small form factor light source (surface mountable, hermetically sealed etc.). By choosing appropriate power classes for the single mode laser diodes the module can be used for both waveguide combiner and direct retina scanning optical architectures. Beside the optical power also the geometrical arrangement of the colors can be optimized according to the needs of the system integrator.

The RGB laser sources integrated with waveguide type combiner is introduced. Compact full color beam scanning modules are developed by using the RGB laser light sources. RGB laser beams are completely aligned and clear full color high-definition video images are obtained.

Virtual and augmented reality are rapidly developing technologies, but their large-scale penetration will require breakthroughs in the optical architecture. Unlike electronics that has rapidly evolved and shrunk in size following Moore’s law, the appearance and the underlying physics of lenses have not changed much. In this talk, we would like to show a path to a future VR/AR platform based on the meta-optics. We show millimeter-scale-diameter, high-NA, sub-micron thin, metasurface-based lenses that achieve diffraction-limited achromatic focusing of the primary colors by exploiting constructive interference of light from multiple zones and dispersion engineering meta-atoms. We further demonstrate a VR system based on a home-built fiber scanning near-eye display..

The recently reported “Angular Spatial Light Modulator” (ASLM) light engine, using pulsed illumination synchronized to a Digital Micromirror Device (DMD), shows significant promise to enhance pixel counts of Near-to-Eye Displays (NED) without increasing package volume, but requires an uncommon illumination driver. We present a field effect transistor based constant-current driver that is fast, compact, and scalable to RGB illumination. The digital-to-analog convertor modulates intensity on-the-fly for illumination-based multiplexing. The driver outputs 100 ns pulses, up to 24 kHz repetition rate. The circuit is demonstrated for two laser diodes and for two LEDs in an ASLM-enhanced pixel count display.

As of today, display performance is a major development criteria in the quest to deliver consumer-ready, high-quality XR glasses. Laser beam scanners in comparison with other display technologies are among the most promising high-dynamic range RGB display engine architectures, e.g., because the size of these devices remains unchanged when increasing the display resolution and field of view. This is in sharp contrast to competing display engines where each pixel constitutes an individual component and these technologies at some point seem to reach their physical limits. On the other hand, manufacturing state-of-the-art laser beam scanners including optics seems especially labor intensive, exhibiting a low yield, therefore driving up the price of XR glasses. This paper addresses the potential benefits and pitfalls of using laser beam scanners in XR and gives an insight into new solutions in next-gen laser beam scanning devices like, e.g., replacing cumbersome hardware beam combination by mere software solutions.

A Visible Light Communication (VLC) transmitter that can transmit data efficiently without utilizing a blue optical filter is needed while also be able to produce high color and excellent lighting performance. A dichromatic LED configuration consisting of a data transmitter (Blue LED/Red LED) and illumination light source (LED Lime) is modulated with OOKNRZ signal and then compared to traditional LED Transmitters. From a lighting perspective, Lime-Red and Lime-Blue WLED configuration can achieve Color Rendering Index (CRI) value of 81.8 and 76.4 respectively, compared to the highest CRI value of RGB WLED that is 74.5. From a communication perspective, Lime-Red and Lime-Blue WLED can achieve data rate of 1.31 Mbps (red modulated) and 1.28 Mbps (blue modulated) compared to data rate of 1.09 Mbps by traditional Phosphorous Cool White LED. Lime-Red and Lime-Blue WLED does not have the bandwidth limitation that is prevalent in Phosphorous White LEDs, which makes these 2 configurations able to trespass the traditional data rate limit without the use of blue optical filter. Thus, by only using 2 LEDs, VLC transmitters with high CRIs, and capability of transmitting a high data rate without the use of blue optical filter is obtained.

Remote sensing image river extraction can provide effective data information for flood disaster prevention and control, water resource pollution etc. River extraction of remote sensing image can be regarded as an image segmentation problem. Due to the large amount of information contained in RGB (Red, Green and Blue) color images, the existing remote sensing image river extraction algorithms extract rivers based on grayscale images. However, human eyes have a high recognition for color images, and at the same time, rich information is conducive to more accurate image extraction. In this paper, HSV (Hue, Saturation and value) color image segmentation is used in remote sensing image river segmentation, the RGB color image is converted to HSV color image, H image component and S image component are multiplied, and set the threshold image is multiplied with the original image to obtain the river extraction image. At last, combined with river shape detection extracts the river area and water area from the remote sensing image to obtain a complete river area image.

Vernona amygdalina has been scientifically proven to have activity against various diseases i.e. anti-inflammatory, antimicrobial, antioxidant, and anti-allergic. Detection of chlorophyll content in the leaves by non-invasive sensing is very important to estimate the antioxidant content. The purpose of this study was to predict the chlorophyll content of Vernona amygdalina leaves using computer vision as non-invasive sensing method. Artificial neural network (ANN) was used to model RGB colour co-occurrence matrix (CCM) and grey level co-occurrence matrix (GLCM) textural features as input and leaf chlorophyll content as output. Performance comparisons in each ANN model were carried out to find the best model in predicting leaf chlorophyll content, indicated by the smallest prediction error value. The results showed that ANN can describe the relationship between textural features and leaf chlorophyll content. Red CCM textural features-subset showed the best results when compared to Green CCM, Blue CCM, and GLCM. The learning process in the training set data showed the MSE value of 0.0099, while the MSE value of the validation set data was 0.0472. The ANN model structure that can be used to predict chlorophyll content in Vernona amygdalina leaves consisted of 3 layers with 30 hidden nodes.

Recently, the research and exploration of the ocean has attracted the attention of a large number of scholars. Due to the light scattering and absorption in underwater imaging, the fish observation images cause imaging blurred and details lost. To handle the above limitation, an underwater polarization imaging recovery technique is proposed to recover visibility restoration and color correction of images. First, in the CIE-Lab model, morphological filtering is used to repair the missing details of the L channel, and color correction is performed on A channel. Secondly, in the RGB model, adaptive contrast stretching is performed to enhance the contrast of the image. To further improve the image clarity, dark channel dehazing is implemented to reduce the atomization caused by scattering. Extensive experiments demonstrate that the method can significantly enhance the visual effect, clarity, and detail information of the fish image. The restoration and division of the contour information of fish in the image are improved.

Visual object tracking has attracted a lot of interests due to its applications in numerous fields such as industry and security. Because the change of illumination could lead to RGB tracking failure, more and more researchers focus on RGB-T tracking methods based on fusion of visible and thermal infrared spectrums and hasten their development in recent years. In order to utilize dual-modal complementary information adaptively, we design a weight-aware dual-modal feature aggregation mechanism, and the WF DiMP algorithm for RGB-T tracking is therefore proposed in this paper. In WF DiMP, deep features of visible and thermal infrared images are extracted by ResNet50 and are leveraged to produce heterogenous response maps, from which dual-modal weights are learned adaptively. Weighted deep features are then concatenated as input of classifier and bounding box estimation module respectively in DiMP (Discriminative Model Prediction) network to obtain the final confidence map and an object bounding box. Experiments on VOT-RGBT2019 dataset are carried out. The results show that WF DiMP algorithm has higher tracking accuracy and robustness. The evaluation indexes PR, SR reach 82.1% and 56.3% respectively, which prove the effectiveness of our mechanism given in the paper.

Rolling quality is a key index of the cotton quality, which directly influences the quality of the lint and textiles, however, it is mainly decided through visual classification by skilled personnel. In order to realize the intelligent rapid classification of cotton quality, this paper proposed a decision-level fusion recognition method for the cotton quality grade based on colored-image information. After the preprocessing of images, RGB and HSV features were calculated, respectively. The features are normalization processed and principal component analysis (PCA) is employed to extract the greater contribution features of RGB and HSV images, which are adopted as BP neural network (BPNN) input parameters to identify the quality grade recognition of cotton, respectively, and then output parameters of BPNN are used as independent evidence to construct Basic Probability Assignment (BPA). Finally, D-S Theory is used to obtain the decision fusion and realize the high accuracy the recognition of cotton quality grades. The compared experimental results show that the precision of proposed method is significantly superior to classification using RGB and HSV features respectively. The method provided in this paper can realize the intelligent rapid classification of cotton quality, and proves the feasibility of cotton-graded artificial intelligent classification.

To evaluate the effect of illumination, camera orientation, and camera distance on color consistency of different skin tones, several swatches from a Pantone SkinTone Guide deck were selected and photographed at varying distances, illuminations, and angles. The RGB values from each selected swatch from each image were converted to LAB units and compared with the converted “true” values provided by Pantone. The ability of various color references to correct different skin tones was tested by comparing color values from a subject’s forearm skin to “true” color values of the closest Pantone SkinTone swatch found by visual comparison.

Bayfol HX photopolymer films prove themselves as easy-to-process recording materials for volume holographic optical elements (vHOEs) and are available at industrial scale. Their full-color (RGB) recording and replay capabilities are two of their major advantages. Bayfol HX is compatible to plastic processing techniques like thermoforming, film insert molding and casting. Therefore, Bayfol HX made its way in applications in the field of augmented reality such as Head-up-Displays (HUD) and Head-mounted-Displays (HMD), in free-space combiners, in plastic optical waveguides, and in transparent screens. Bayfol HX can be adopted for a variety of applications. To offer access to more applications, we address the sensitization into the Near Infrared Region (NIR) and increase the achievable index modulation Δn1 beyond 0.06. In this paper, we will report on our latest developments in these fields.

Current fingerprint anti-spoofing and liveness detection techniques heavily relied on image processing algorithms and did not fully explore other hardware of a smartphone. In this study, we utilize the phone’s front-facing teardrop notch camera and LED display for multi-spectral imaging and liveness detection. We programmed the phone to illuminate different RGB colors patterns while the camera captures images of the illuminated finger. A custom-designed software allows for the distinct detection between spoof and live fingers based on resulting images and video. The method was implemented and tested in a Samsung Galaxy A50 with a teardrop notch camera. Utilizing a MATLAB program, we were able to distinguish a real finger from eight different-colored spoofs based on the images captured from the front-facing camera.

MEMS Mirrors provide a great way to spatially scan light and is being used for a wide range of applications such as raster scanning RGB light for Display Engines or scanning infra-red light for depth cameras (3D sensing) by using structured light or Time of Flight (TOF) techniques. Yet, in some cases it is required to deploy multiple MEMS Mirrors that ought to work in tandem, such as implementing a Display Engine for a binocular HMD (Head Mount Display, AR/VR/MR) generating two images, one per eye. In such a case, it is necessary to synchronize the movement of the MEMS mirrors, so that the images that are presented to the viewer, would create a homogenous and consistent image. Another possible use case is to increase the Field Of View, such as for an Automotive Lidar application, so that each Scanning Engine with a mounted MEMS Mirror would cover a subset of the scanned scene, yet collectively would provide an increased scanned FOV. The scanning of each segment by one Scanning Engine would be done sequentially and in a synchronized manner with the other Scanning Engines. This paper will present a viable synchronization mechanism for two main types of MEMS Mirror: Resonant Mirror and Quasi Static Mirrors. The paper will discuss the challenges and considerations of the System Clock, Master / Slave modes, overcoming jitter and noise and modeling by simulations the expected results. Additionally, the paper will present the actual results of such system implementation as a result of this work.

We reported Ag deposition-based electrochromic device which showed reversible color change from transparent to chromatic coloration, black and silver-mirror in a single device. This device newly enabled us to achieve multi-coloration including CMY and RGB. We analyzed relationship between optical properties and morphology of silver nanoparticles toward plasmonic nanophotonics.

Smart Glasses, a subcategory of Head Mounted Display devices, are getting popular in the recent years, offering a convenient way to add information in addition to what the wearer see. Designers are seeking ways to build a Display Engine that can meet the targeted image performance (such as resolution, filed of view) yet keeping the total device size and weight as low as possible. This paper presents an innovative Display Engine based on Laser Beam Scanning (LBS) and outlines the main building blocks that were design and prototyped. It presents the use of MEMS Micro Mirrors (Resonant mirror and Linear mirror) and a newly developed RGB laser light source that integrates the three laser diodes into a single package. This paper outlines the opto-mechanical design considerations, beam and spot size analysis, the driving electronics, mirror and color control loops, the procedures and steps to assemble and calibrate the optical engine and finally presents the achieved results of this work.