Search

Diffractive multispectral optical imaging plays an essential role in optical sensing, which typically suffers from the image blurring problem caused by the spatially variant point spread function. Here, we propose a novel high-quality and efficient hybrid space calibrated 3D network "HSC3D" for spatially variant diffractive multispectral imaging that utilizes the 3D U-Net structure combined with space calibration modules of magnification and rotation effects to achieve high-accuracy eight-channel multispectral restoration. The algorithm combines the advantages of the space calibrated module and U-Net architecture with 3D convolutional layers to improve the image quality of diffractive multispectral imaging without the requirements of complex equipment modifications and large amounts of data. A diffractive multispectral imaging system is established by designing and manufacturing one diffractive lens and four refractive lenses, whose monochromatic aberration is carefully corrected to improve imaging quality. The mean peak signal-to-noise ratio and mean structural similarity index of the reconstructed multispectral images are improved by 3.33 dB and 0.08, respectively, presenting obviously improved image quality compared with a typical Unrolled Network algorithm. The new algorithm with high space calibrated ability and imaging quality has great application potential in diffraction lens spectroscopy and paves a new method for complex practical diffractive multispectral image sensing. [ABSTRACT FROM AUTHOR]

Multispectral remote sensing images contain abundant information about the distribution and reflectance of ground objects, playing a crucial role in target detection, environmental monitoring, and resource exploration. However, due to the complexity of the imaging process in multispectral remote sensing, image blur is inevitable, and the blur kernel is typically unknown. In recent years, many researchers have focused on blind image deblurring, but most of these methods are based on single-band images. When applied to CASEarth satellite multispectral images, the spectral correlation is unutilized. To address this limitation, this paper proposes a novel approach that leverages the characteristics of multispectral data more effectively. We introduce an inter-band gradient similarity prior and incorporate it into the patch-wise minimal pixel (PMP)-based deblurring model. This approach aims to utilize the spectral correlation across bands to improve deblurring performance. A solution algorithm is established by combining the half-quadratic splitting method with alternating minimization. Subjectively, the final experiments on CASEarth multispectral images demonstrate that the proposed method offers good visual effects while enhancing edge sharpness. Objectively, our method leads to an average improvement in point sharpness by a factor of 1.6, an increase in edge strength level by a factor of 1.17, and an enhancement in RMS contrast by a factor of 1.11. [ABSTRACT FROM AUTHOR]

In the field of remote sensing image processing, pansharpening technology stands as a critical advancement. This technology aims to enhance multispectral images that possess low resolution by integrating them with high-spatial-resolution panchromatic images, ultimately producing multispectral images with high resolution that are abundant in both spatial and spectral details. Thus, there remains potential for improving the quality of both the spectral and spatial domains of the fused images based on deep-learning-based pansharpening methods. This work proposes a new method for the task of pansharpening: the Multi-Stage Progressive Pansharpening Network with Detail Injection with Redundancy Reduction Mechanism (MSPPN-DIRRM). This network is divided into three levels, each of which is optimized for the extraction of spectral and spatial data at different scales. Particular spectral feature and spatial detail extraction modules are used at each stage. Moreover, a new image reconstruction module named the DRRM is introduced in this work; it eliminates both spatial and channel redundancy and improves the fusion quality. The effectiveness of the proposed model is further supported by experimental results using both simulated data and real data from the QuickBird, GaoFen1, and WorldView2 satellites; these results show that the proposed model outperforms deep-learning-based methods in both visual and quantitative assessments. Among various evaluation metrics, performance improves by 0.92–18.7% compared to the latest methods. [ABSTRACT FROM AUTHOR]

Crop height and biomass are the two important phenotyping traits to screen forage population types at local and regional scales. This study aims to compare the performances of multispectral and RGB sensors onboard drones for quantitative retrievals of forage crop height and biomass at very high resolution. We acquired the unmanned aerial vehicle (UAV) multispectral images (MSIs) at 1.67 cm spatial resolution and visible data (RGB) at 0.31 cm resolution and measured the forage height and above-ground biomass over the alfalfa (Medicago sativa L.) breeding trials in the Canadian Prairies. (1) For height estimation, the digital surface model (DSM) and digital terrain model (DTM) were extracted from MSI and RGB data, respectively. As the resolution of the DTM is five times less than that of the DSM, we applied an aggregation algorithm to the DSM to constrain the same spatial resolution between DSM and DTM. The difference between DSM and DTM was computed as the canopy height model (CHM), which was at 8.35 cm and 1.55 cm for MSI and RGB data, respectively. (2) For biomass estimation, the normalized difference vegetation index (NDVI) from MSI data and excess green (ExG) index from RGB data were analyzed and regressed in terms of ground measurements, leading to empirical models. The results indicate better performance of MSI for above-ground biomass (AGB) retrievals at 1.67 cm resolution and better performance of RGB data for canopy height retrievals at 1.55 cm. Although the retrieved height was well correlated with the ground measurements, a significant underestimation was observed. Thus, we developed a bias correction function to match the retrieval with the ground measurements. This study provides insight into the optimal selection of sensor for specific targeted vegetation growth traits in a forage crop. [ABSTRACT FROM AUTHOR]

In maritime transportation, a ship's draft survey serves as a primary method for weighing bulk cargo. The accuracy of the ship's draft reading determines the fairness of bulk cargo transactions. Human visual-based draft reading methods face issues such as safety concerns, high labor costs, and subjective interpretation. Therefore, some image processing methods are utilized to achieve automatic draft reading. However, due to the limitations in the spectral characteristics of RGB images, existing image processing methods are susceptible to water surface environmental interference, such as reflections. To solve this issue, we obtained and annotated 524 multispectral images of a ship's draft as the research dataset, marking the first application of integrating NIR information and RGB images for automatic draft reading tasks. Additionally, a dual-branch backbone named BIF is proposed to extract and combine spectral information from RGB and NIR images. The backbone network can be combined with the existing segmentation head and detection head to perform waterline segmentation and draft detection. By replacing the original ResNet-50 backbone of YOLOv8, we reached a mAP of 99.2% in the draft detection task. Similarly, combining UPerNet with our dual-branch backbone, the mIoU of the waterline segmentation task was improved from 98.9% to 99.3%. The inaccuracy of the draft reading is less than ±0.01 m, confirming the efficacy of our method for automatic draft reading tasks. [ABSTRACT FROM AUTHOR]

Canopy imaging offers a non-destructive, efficient way to objectively measure canopy size, detect stress symptoms, and assess pigment concentrations. While it is faster and easier than traditional destructive methods, manual image analysis, including segmentation and evaluation, can be time-consuming. To make imaging more widely accessible, it's essential to reduce the cost of imaging systems and automate the analysis process. We developed a low-cost imaging system with automated analysis using an embedded microcomputer equipped with a monochrome camera and a filter for a total hardware cost of ~USD 500. Our imaging system takes images under blue, green, red, and infrared light, as well as chlorophyll fluorescence. The system uses a Python-based program to collect and analyze images automatically. The multi-spectral imaging system separates plants from the background using a chlorophyll fluorescence image, which is also used to quantify canopy size. The system then generates normalized difference vegetation index (NDVI, "greenness") images and histograms, providing quantitative, spatially resolved information. We verified that these indices correlate with leaf chlorophyll content and can easily add other indices by installing light sources with the desired spectrums. The low cost of the system can make this imaging technology widely available. [ABSTRACT FROM AUTHOR]

Fire blight is an infectious disease found in apple and pear orchards. While managing the disease is critical to maintaining orchard health, identifying symptoms early is a challenging task which requires trained expert personnel. This paper presents an inspection technique that targets individual symptoms via deep learning and density estimation. We evaluate the effects of including multi-spectral sensors in the model's pipeline. Results show that adding near infrared (NIR) channels can help improve prediction performance and that density estimation can detect possible symptoms when severity is in the mid-high range. [ABSTRACT FROM AUTHOR]

(1) Background: Yield-monitoring systems are widely used in grain crops but are less advanced for hay and forage. Current commercial systems are generally limited to weighing individual bales, limiting the spatial resolution of maps of hay yield. This study evaluated an Uncrewed Aerial Vehicle (UAV)-based imaging system to estimate hay yield. (2) Methods: Data were collected from three 0.4 ha plots and a 35 ha hay field of red clover and timothy grass in September 2020. A multispectral camera on the UAV captured images at 30 m (20 mm pixel−1) and 50 m (35 mm pixel−1) heights. Eleven Vegetation Indices (VIs) and five texture features were calculated from the images to estimate biomass yield. Multivariate regression models (VIs and texture features vs. biomass) were evaluated. (3) Results: Model R2 values ranged from 0.31 to 0.68. (4) Conclusions: Despite strong correlations between standard VIs and biomass, challenges such as variable image resolution and clarity affected accuracy. Further research is needed before UAV-based yield estimation can provide accurate, high-resolution hay yield maps. [ABSTRACT FROM AUTHOR]

Spectral imaging has many applications, from methane detection using satellites to disease detection on crops. However, spectral cameras remain a costly solution ranging from 10 thousand to 100 thousand euros for the hardware alone. Here, we present a low-cost multispectral camera (LC-MSC) with 64 LEDs in eight different colors and a monochrome camera with a hardware cost of 340 euros. Our prototype reproduces spectra accurately when compared to a reference spectrometer to within the spectral width of the LEDs used and the ±1 σ variation over the surface of ceramic reference tiles. The mean absolute difference in reflectance is an overestimate of 0.03 for the LC-MSC as compared to a spectrometer, due to the spectral shape of the tiles. In environmental light levels of 0.5 W m−2 (bright artificial indoor lighting) our approach shows an increase in noise, but still faithfully reproduces discrete reflectance spectra over 400 nm–1000 nm. Our approach is limited in its application by LED bandwidth and availability of specific LED wavelengths. However, unlike with conventional spectral cameras, the pixel pitch of the camera itself is not limited, providing higher image resolution than typical high-end multi- and hyperspectral cameras. For sample conditions where LED illumination bands provide suitable spectral information, our LC-MSC is an interesting low-cost alternative approach to spectral imaging. [ABSTRACT FROM AUTHOR]

Accurately extracting large-scale offshore floating raft aquaculture (FRA) areas is crucial for supporting scientific planning and precise aquaculture management. While remote sensing technology offers advantages such as wide coverage, rapid imaging, and multispectral capabilities for FRA monitoring, the current methods face challenges in terms of establishing spatial–spectral correlations and extracting multiscale features, thereby limiting their accuracy. To address these issues, we propose an innovative multiscale spatial–spectral fusion network (MSSFNet) designed specifically for extracting offshore FRA areas from multispectral remote sensing imagery. MSSFNet effectively integrates spectral and spatial information through a spatial–spectral feature extraction block (SSFEB), significantly enhancing the accuracy of FRA area identification. Additionally, a multiscale spatial attention block (MSAB) captures contextual information across different scales, improving the ability to detect FRA areas of varying sizes and shapes while minimizing edge artifacts. We created the CHN-YE7-FRA dataset using Sentinel-2 multispectral remote sensing imagery and conducted extensive evaluations. The results showed that MSSFNet achieved impressive metrics: an F1 score of 90.76%, an intersection over union (IoU) of 83.08%, and a kappa coefficient of 89.75%, surpassing those of state-of-the-art methods. The ablation results confirmed that the SSFEB and MSAB modules effectively enhanced the FRA extraction accuracy. Furthermore, the successful practical applications of MSSFNet validated its generalizability and robustness across diverse marine environments. These findings highlight the performance of MSSFNet in both experimental and real-world scenarios, providing reliable, precise FRA area monitoring. This capability provides crucial data for scientific planning and environmental protection purposes in coastal aquaculture zones. [ABSTRACT FROM AUTHOR]

A snapshot multi-wavelength birefringence imaging measurement method was proposed in this study. The RGB-LEDs at wavelengths 463 nm, 533 nm, and 629 nm were illuminated with circularly polarized light after passing through a circular polarizer. The transmitted light through the birefringent sample was captured by a color polarization camera. A single imaging process captured light intensity in four polarization directions (0°, 45°, 90°, and 135°) for each of the three RGB spectral wavelength channels, and subsequently measured the first three elements of Stokes vectors (S0, S1, and S2) after the sample. The birefringence retardance and fast-axis azimuthal angle were determined simultaneously. An experimental setup was constructed, and polarization response matrices were calibrated for each spectral wavelength channel to ensure the accurate detection of Stokes vectors. A polymer true zero-order quarter-wave plate was employed to validate measurement accuracy and repeatability. Additionally, stress-induced birefringence in a PMMA arch-shaped workpiece was measured both before and after the application of force. Experimental results revealed that the repeatability of birefringence retardance and fast-axis azimuthal angle was better than 0.67 nm and 0.08°, respectively. This approach enables multispectral wavelength, high-speed, high-precision, and high-repeatability birefringence imaging measurements through a single imaging session. [ABSTRACT FROM AUTHOR]

Visual object tracking, pivotal for applications like earth observation and environmental monitoring, encounters challenges under adverse conditions such as low light and complex backgrounds. Traditional tracking technologies often falter, especially when tracking dynamic objects like aircraft amidst rapid movements and environmental disturbances. This study introduces an innovative adaptive multimodal image object-tracking model that harnesses the capabilities of multispectral image sensors, combining infrared and visible light imagery to significantly enhance tracking accuracy and robustness. By employing the advanced vision transformer architecture and integrating token spatial filtering (TSF) and crossmodal compensation (CMC), our model dynamically adjusts to diverse tracking scenarios. Comprehensive experiments conducted on a private dataset and various public datasets demonstrate the model's superior performance under extreme conditions, affirming its adaptability to rapid environmental changes and sensor limitations. This research not only advances visual tracking technology but also offers extensive insights into multisource image fusion and adaptive tracking strategies, establishing a robust foundation for future enhancements in sensor-based tracking systems. [ABSTRACT FROM AUTHOR]

Objective:The objective of this study is to explore and enhance the diagnostic process of unipolar and bipolar disorders. The primary focus is on leveraging automated processes to improve the accuracy and accessibility of diagnosis. The study aims to introduce an audio corpus collected from patients diagnosed with these disorders, annotated using the Clinical Global Impressions Scale (CGI) by psychiatrists. Methods and procedures: Traditional diagnostic methods rely on the clinician's expertise and consideration of co-existing mental disorders. However, this study proposes the implementation of automated processes in the diagnosis, providing quantitative measures and enabling prolonged observation of patients. The paper introduces a speech signal pipeline for CGI state classification, with a specific focus on selecting the most discriminative features. Acoustic features such as prosodies, MFCC, and LPC coefficients are examined in the study. The classification process utilizes common machine learning methods. Results: The results of the study indicate promising outcomes for the automated diagnosis of bipolar and unipolar disorders using the proposed speech signal pipeline. The audio corpus annotated with CGI by psychiatrists achieved a classification accuracy of 95% for the two-class classification. For the four- and seven-class classifications, the results were 77.3% and 73%, respectively, demonstrating the potential of the developed method in distinguishing different states of the disorders. [ABSTRACT FROM AUTHOR]

Radar sensors, leveraging the Doppler effect, enable the nonintrusive capture of kinetic and physiological motions while preserving privacy. Deep learning (DL) facilitates radar sensing for healthcare applications such as gait recognition and vital-sign measurement. However, band-dependent patterns, indicating variations in patterns and power scales associated with frequencies in time–frequency representation (TFR), challenge radar sensing applications using DL. Frequency-dependent characteristics and features with lower power scales may be overlooked during representation learning. This paper proposes an Enhanced Band-Dependent Learning framework (E-BDL) comprising an adaptive sub-band filtering module, a representation learning module, and a sub-view contrastive module to fully detect band-dependent features in sub-frequency bands and leverage them for classification. Experimental validation is conducted on two radar datasets, including gait abnormality recognition for Alzheimer's disease (AD) and AD-related dementia (ADRD) risk evaluation and vital-sign monitoring for hemodynamics scenario classification. For hemodynamics scenario classification, E-BDL-ResNet achieves competitive performance in overall accuracy and class-wise evaluations compared to recent methods. For ADRD risk evaluation, the results demonstrate E-BDL-ResNet's superior performance across all candidate models, highlighting its potential as a clinical tool. E-BDL effectively detects salient sub-bands in TFRs, enhancing representation learning and improving the performance and interpretability of DL-based models. [ABSTRACT FROM AUTHOR]

Liver fibrosis, a major global health issue, is marked by excessive collagen deposition that impairs liver function. Noninvasive methods for the direct visualization of collagen content are crucial for the early detection and monitoring of fibrosis progression. This study investigates the potential of spectral photoacoustic imaging (sPAI) to monitor collagen development in liver fibrosis. Utilizing a novel data-driven superpixel photoacoustic unmixing (SPAX) framework, we aimed to distinguish collagen presence and evaluate its correlation with fibrosis progression. We employed an established diethylnitrosamine (DEN) model in rats to study liver fibrosis over various time points. Our results revealed a significant correlation between increased collagen photoacoustic signal intensity and advanced fibrosis stages. Collagen abundance maps displayed dynamic changes throughout fibrosis progression. These findings underscore the potential of sPAI for the noninvasive monitoring of collagen dynamics and fibrosis severity assessment. This research advances the development of noninvasive diagnostic tools and personalized management strategies for liver fibrosis. [ABSTRACT FROM AUTHOR]

Human action recognition based on optical and infrared video data is greatly affected by the environment, and feature extraction in traditional machine learning classification methods is complex; therefore, this paper proposes a method for human action recognition using Frequency Modulated Continuous Wave (FMCW) radar based on an asymmetric convolutional residual network. First, the radar echo data are analyzed and processed to extract the micro-Doppler time domain spectrograms of different actions. Second, a strategy combining asymmetric convolution and the Mish activation function is adopted in the residual block of the ResNet18 network to address the limitations of linear and nonlinear transformations in the residual block for micro-Doppler spectrum recognition. This approach aims to enhance the network's ability to learn features effectively. Finally, the Improved Convolutional Block Attention Module (ICBAM) is integrated into the residual block to enhance the model's attention and comprehension of input data. The experimental results demonstrate that the proposed method achieves a high accuracy of 98.28% in action recognition and classification within complex scenes, surpassing classic deep learning approaches. Moreover, this method significantly improves the recognition accuracy for actions with similar micro-Doppler features and demonstrates excellent anti-noise recognition performance. [ABSTRACT FROM AUTHOR]

Hyperspectral imaging was used to predict the total polyphenol content in low-temperature stressed tomato seedlings for the development of a multispectral image sensor. The spectral data with a full width at half maximum (FWHM) of 5 nm were merged to obtain FWHMs of 10 nm, 25 nm, and 50 nm using a commercialized bandpass filter. Using the permutation importance method and regression coefficients, we developed the least absolute shrinkage and selection operator (Lasso) regression models by setting the band number to ≥11, ≤10, and ≤5 for each FWHM. The regression model using 56 bands with an FWHM of 5 nm resulted in an R2 of 0.71, an RMSE of 3.99 mg/g, and an RE of 9.04%, whereas the model developed using the spectral data of only 5 bands with a FWHM of 25 nm (at 519.5 nm, 620.1 nm, 660.3 nm, 719.8 nm, and 980.3 nm) provided an R2 of 0.62, an RMSE of 4.54 mg/g, and an RE of 10.3%. These results show that a multispectral image sensor can be developed to predict the total polyphenol content of tomato seedlings subjected to low-temperature stress, paving the way for energy saving and low-temperature stress damage prevention in vegetable seedling production. [ABSTRACT FROM AUTHOR]

Internet of Things (IoT) applications and resources are highly vulnerable to flood attacks, including Distributed Denial of Service (DDoS) attacks. These attacks overwhelm the targeted device with numerous network packets, making its resources inaccessible to authorized users. Such attacks may comprise attack references, attack types, sub-categories, host information, malicious scripts, etc. These details assist security professionals in identifying weaknesses, tailoring defense measures, and responding rapidly to possible threats, thereby improving the overall security posture of IoT devices. Developing an intelligent Intrusion Detection System (IDS) is highly complex due to its numerous network features. This study presents an improved IDS for IoT security that employs multimodal big data representation and transfer learning. First, the Packet Capture (PCAP) files are crawled to retrieve the necessary attacks and bytes. Second, Spark-based big data optimization algorithms handle huge volumes of data. Second, a transfer learning approach such as word2vec retrieves semantically-based observed features. Third, an algorithm is developed to convert network bytes into images, and texture features are extracted by configuring an attention-based Residual Network (ResNet). Finally, the trained text and texture features are combined and used as multimodal features to classify various attacks. The proposed method is thoroughly evaluated on three widely used IoT-based datasets: CIC-IoT 2022, CIC-IoT 2023, and Edge-IIoT. The proposed method achieves excellent classification performance, with an accuracy of 98.2%. In addition, we present a game theory-based process to validate the proposed approach formally. [ABSTRACT FROM AUTHOR]

In complex environments a single visible image is not good enough to perceive the environment, this paper proposes a novel dual-stream real-time detector designed for target detection in extreme environments such as nighttime and fog, which is able to efficiently utilise both visible and infrared images to achieve Fast All-Weatherenvironment sensing (FAWDet). Firstly, in order to allow the network to process information from different modalities simultaneously, this paper expands the state-of-the-art end-to-end detector YOLOv8, the backbone is expanded in parallel as a dual stream. Then, for purpose of avoid information loss in the process of network deepening, a cross-modal feature enhancement module is designed in this study, which enhances each modal feature by cross-modal attention mechanisms, thus effectively avoiding information loss and improving the detection capability of small targets. In addition, for the significant differences between modal features, this paper proposes a three-stage fusion strategy to optimise the feature integration through the fusion of spatial, channel and overall dimensions. It is worth mentioning that the cross-modal feature fusion module adopts an end-to-end training approach. Extensive experiments on two datasets validate that the proposed method achieves state-of-the-art performance in detecting small targets. The cross-modal real-time detector in this study not only demonstrates excellent stability and robust detection performance, but also provides a new solution for target detection techniques in extreme environments. [ABSTRACT FROM AUTHOR]

Aircraft failures can result in the leakage of fuel, hydraulic oil, or other lubricants onto the runway during landing or taxiing. Damage to fuel tanks or oil lines during hard landings or accidents can also contribute to these spills. Further, improper maintenance or operational errors may leave oil traces on the runway before take-off or after landing. Identifying oil spills in airport runway videos is crucial to flight safety and accident investigation. Advanced image processing techniques can overcome the limitations of conventional RGB-based detection, which struggles to differentiate between oil spills and sewage due to similar coloration; given that oil and sewage have distinct spectral absorption patterns, precise detection can be performed based on multispectral images. In this study, we developed a method for spectrally enhancing RGB images of oil spills on airport runways to generate HSI images, facilitating oil spill detection in conventional RGB imagery. To this end, we employed the MST++ spectral reconstruction network model to effectively reconstruct RGB images into multispectral images, yielding improved accuracy in oil detection compared with other models. Additionally, we utilized the Fast R-CNN oil spill detection model, resulting in a 5% increase in Intersection over Union (IOU) for HSI images. Moreover, compared with RGB images, this approach significantly enhanced detection accuracy and completeness by 25.3% and 26.5%, respectively. These findings clearly demonstrate the superior precision and accuracy of HSI images based on spectral reconstruction in oil spill detection compared with traditional RGB images. With the spectral reconstruction technique, we can effectively make use of the spectral information inherent in oil spills, thereby enhancing detection accuracy. Future research could delve deeper into optimization techniques and conduct extensive validation in real airport environments. In conclusion, this spectral reconstruction-based technique for detecting oil spills on airport runways offers a novel and efficient approach that upholds both efficacy and accuracy. Its wide-scale implementation in airport operations holds great potential for improving aviation safety and environmental protection. [ABSTRACT FROM AUTHOR]

Multi-source remote sensing-derived information on crops contributes significantly to agricultural monitoring, assessment, and management. In Africa, some challenges (i.e., small-scale farming practices associated with diverse crop types and agricultural system complexity, and cloud coverage during the growing season) can imped agricultural monitoring using multi-source remote sensing. The combination of optical remote sensing and synthetic aperture radar (SAR) data has emerged as an opportune strategy for improving the precision and reliability of crop type mapping and monitoring. This work aims to conduct an extensive review of the challenges of agricultural monitoring and mapping in Africa in great detail as well as the current research progress of agricultural monitoring based on optical and Radar satellites. In this context optical data may provide high spatial resolution and detailed spectral information, which allows for the differentiation of different crop types based on their spectral signatures. However, synthetic aperture radar (SAR) satellites can provide important contributions given the ability of this technology to penetrate cloud cover, particularly in African tropical regions, as opposed to optical data. This review explores various combination techniques employed to integrate optical and SAR data for crop type classification and their applicability and limitations in the context of African countries. Furthermore, challenges are discussed in this review as well as and the limitations associated with optical and SAR data combination, such as the data availability, sensor compatibility, and the need for accurate ground truth data for model training and validation. This study also highlights the potential of advanced modelling (i.e., machine learning algorithms, such as support vector machines, random forests, and convolutional neural networks) in improving the accuracy and automation of crop type classification using combined data. Finally, this review concludes with future research directions and recommendations for utilizing optical and SAR data combination techniques in crop type classification for African agricultural systems. Furthermore, it emphasizes the importance of developing robust and scalable classification models that can accommodate the diversity of crop types, farming practices, and environmental conditions prevalent in Africa. Through the utilization of combined remote sensing technologies, informed decisions can be made to support sustainable agricultural practices, strengthen nutritional security, and contribute to the socioeconomic development of the continent. [ABSTRACT FROM AUTHOR]

This work presents an approach for the recognition of plastics using a low-cost spectroscopy sensor module together with a set of machine learning methods. The sensor is a multi-spectral module capable of measuring 18 wavelengths from the visible to the near-infrared. Data processing and analysis are performed using a set of ten machine learning methods (Random Forest, Support Vector Machines, Multi-Layer Perceptron, Convolutional Neural Networks, Decision Trees, Logistic Regression, Naive Bayes, k-Nearest Neighbour, AdaBoost, Linear Discriminant Analysis). An experimental setup is designed for systematic data collection from six plastic types including PET, HDPE, PVC, LDPE, PP and PS household waste. The set of computational methods is implemented in a generalised pipeline for the validation of the proposed approach for the recognition of plastics. The results show that Convolutional Neural Networks and Multi-Layer Perceptron can recognise plastics with a mean accuracy of 72.50% and 70.25%, respectively, with the largest accuracy of 83.5% for PS plastic and the smallest accuracy of 66% for PET plastic. The results demonstrate that this low-cost near-infrared sensor with machine learning methods can recognise plastics effectively, making it an affordable and portable approach that contributes to the development of sustainable systems with potential for applications in other fields such as agriculture, e-waste recycling, healthcare and manufacturing. [ABSTRACT FROM AUTHOR]

Automotive radar is one of the key sensors for intelligent driving. Radar image sequences contain abundant spatial and temporal information, enabling target classification. For existing radar spatiotemporal classifiers, multi-view radar images are usually employed to enhance the information of the target and 3D convolution is employed for spatiotemporal feature extraction. These models consume significant hardware resources and are not applicable to real-time applications. In this paper, RadarTCN, a novel lightweight network, is proposed that achieves high-accuracy online target classification using single-view radar image sequences only. In RadarTCN, 2D convolution and 3D-TCN are employed to extract spatiotemporal features sequentially. To reduce data dimensionality and computational complexity, a multi-layer max pooling down-sampling method is designed in a 2D convolution module. Meanwhile, the 3D-TCN module is improved through residual pruning and causal convolution is introduced for leveraging the performance of online target classification. The experimental results demonstrate that RadarTCN can achieve high-precision online target recognition for both range-angle and range-Doppler map sequences. Compared to the reference models on the CARRADA dataset, RadarTCN exhibits better classification performance, with fewer parameters and lower computational complexity. [ABSTRACT FROM AUTHOR]

This paper proposes a workflow to assess the uncertainty of the Normalized Difference Vegetation Index (NDVI), a critical index used in precision agriculture to determine plant health. From a metrological perspective, it is crucial to evaluate the quality of vegetation indices, which are usually obtained by processing multispectral images for measuring vegetation, soil, and environmental parameters. For this reason, it is important to assess how the NVDI measurement is affected by the camera characteristics, light environmental conditions, as well as atmospheric and seasonal/weather conditions. The proposed study investigates the impact of atmospheric conditions on solar irradiation and vegetation reflection captured by a multispectral UAV camera in the red and near-infrared bands and the variation of the nominal wavelengths of the camera in these bands. Specifically, the study examines the influence of atmospheric conditions in three scenarios: dry–clear, humid–hazy, and a combination of both. Furthermore, this investigation takes into account solar irradiance variability and the signal-to-noise ratio (SNR) of the camera. Through Monte Carlo simulations, a sensitivity analysis is carried out against each of the above-mentioned uncertainty sources and their combination. The obtained results demonstrate that the main contributors to the NVDI uncertainty are the atmospheric conditions, the nominal wavelength tolerance of the camera, and the variability of the NDVI values within the considered leaf conditions (dry and fresh). [ABSTRACT FROM AUTHOR]

The present research had two aims. The first was to evaluate the effect of height and exposure on the vegetative response of olive canopies' vertical axis studied through a multispectral sensor and on the qualitative and quantitative product characteristics. The second was to examine the relationship between multispectral data and productive characteristics. Six olive plants were sampled, and their canopy's vertical axis was subdivided into four sectors based on two heights (Top and Low) and two exposures (West and East). A ground-vehicle-mounted multispectral proximal sensor (OptRx from AgLeader®) was used to investigate the different behaviours of the olive canopy vegetation index (VI) responses in each sector. A selective harvest was performed, in which each plant and sector were harvested separately. Product characterisation was conducted to investigate the response of the products (both olives and oils) in each sector. The results of Tukey's test (p > 0.05) showed a significant effect of height for the VI responses, with the Low sector obtaining higher values than the Top sector. The olive product showed some height and exposure effect, particularly for the olives' dimension and resistance to detachment, which was statistically higher in the upper part of the canopies. The regression studies highlighted some relationships between the VIs and product characteristics, particularly for resistance to detachments (R2 = 0.44–0.63), which can affect harvest management. In conclusion, the results showed the complexity of the olive canopies' response to multispectral data collection, highlighting the need to study the vertical axis to assess the variability of the canopy itself. The relationship between multispectral data and product characteristics must be further investigated. [ABSTRACT FROM AUTHOR]

Relative radiometric normalization (RRN) is a critical pre-processing step that enables accurate comparisons of multitemporal remote-sensing (RS) images through unsupervised change detection. Although existing RRN methods generally have promising results in most cases, their effectiveness depends on specific conditions, especially in scenarios with land cover/land use (LULC) in image pairs in different locations. These methods often overlook these complexities, potentially introducing biases to RRN results, mainly because of the use of spatially aligned pseudo-invariant features (PIFs) for modeling. To address this, we introduce a location-independent RRN (LIRRN) method in this study that can automatically identify non-spatially matched PIFs based on brightness characteristics. Additionally, as a fast and coregistration-free model, LIRRN complements keypoint-based RRN for more accurate results in applications where coregistration is crucial. The LIRRN process starts with segmenting reference and subject images into dark, gray, and bright zones using the multi-Otsu threshold technique. PIFs are then efficiently extracted from each zone using nearest-distance-based image content matching without any spatial constraints. These PIFs construct a linear model during subject–image calibration on a band-by-band basis. The performance evaluation involved tests on five registered/unregistered bitemporal satellite images, comparing results from three conventional methods: histogram matching (HM), blockwise KAZE, and keypoint-based RRN algorithms. Experimental results consistently demonstrated LIRRN's superior performance, particularly in handling unregistered datasets. LIRRN also exhibited faster execution times than blockwise KAZE and keypoint-based approaches while yielding results comparable to those of HM in estimating normalization coefficients. Combining LIRRN and keypoint-based RRN models resulted in even more accurate and reliable results, albeit with a slight lengthening of the computational time. To investigate and further develop LIRRN, its code, and some sample datasets are available at link in Data Availability Statement. [ABSTRACT FROM AUTHOR]

The fusion of infrared and visible images is a well-researched task in computer vision. These fusion methods create fused images replacing the manual observation of single sensor image, often deployed on edge devices for real-time processing. However, there is an issue of information imbalance between infrared and visible images. Existing methods often fail to emphasize temperature and edge texture information, potentially leading to misinterpretations. Moreover, these methods are computationally complex, and challenging for edge device adaptation. This paper proposes a method that calculates the distribution proportion of infrared pixel values, allocating fusion weights to adaptively highlight key information. It introduces a weight allocation mechanism and MobileBlock with a multispectral information complementary module, innovations which strengthened the model's fusion capabilities, made it more lightweight, and ensured information compensation. Training involves a temperature-color-perception loss function, enabling adaptive weight allocation based on image pair information. Experimental results show superiority over mainstream fusion methods, particularly in the electric power equipment scene and publicly available datasets. [ABSTRACT FROM AUTHOR]

Light Detection and Ranging (LiDAR) is a well-established active technology for the direct acquisition of 3D data. In recent years, the geometric information collected by LiDAR sensors has been widely combined with optical images to provide supplementary spectral information to achieve more precise results in diverse remote sensing applications. The emergence of active Multispectral LiDAR (MSL) systems, which operate on different wavelengths, has recently been revolutionizing the simultaneous acquisition of height and intensity information. So far, MSL technology has been successfully applied for fine-scale mapping in various domains. However, a comprehensive review of this modern technology is currently lacking. Hence, this study presents an exhaustive overview of the current state-of-the-art in MSL systems by reviewing the latest technologies for MSL data acquisition. Moreover, the paper reports an in-depth analysis of the diverse applications of MSL, spanning across fields of "ecology and forestry", "objects and Land Use Land Cover (LULC) classification", "change detection", "bathymetry", "topographic mapping", "archaeology and geology", and "navigation". Our systematic review uncovers the potentials, opportunities, and challenges of the recently emerged MSL systems, which integrate spatial–spectral data and unlock the capability for precise multi-dimensional (nD) mapping using only a single-data source. [ABSTRACT FROM AUTHOR]

The field of plant phenotype is used to analyze the shape and physiological characteristics of crops in multiple dimensions. Imaging, using non-destructive optical characteristics of plants, analyzes growth characteristics through spectral data. Among these, fluorescence imaging technology is a method of evaluating the physiological characteristics of crops by inducing plant excitation using a specific light source. Through this, we investigate how fluorescence imaging responds sensitively to environmental stress in garlic and can provide important information on future stress management. In this study, near UV LED (405 nm) was used to induce the fluorescence phenomenon of garlic, and fluorescence images were obtained to classify and evaluate crops exposed to abiotic environmental stress. Physiological characteristics related to environmental stress were developed from fluorescence sample images using the Chlorophyll ratio method, and classification performance was evaluated by developing a classification model based on partial least squares discrimination analysis from the image spectrum for stress identification. The environmental stress classification performance identified from the Chlorophyll ratio was 14.9% in F673/F717, 25.6% in F685/F730, and 0.209% in F690/F735. The spectrum-developed PLS-DA showed classification accuracy of 39.6%, 56.2% and 70.7% in Smoothing, MSV, and SNV, respectively. Spectrum pretreatment-based PLS-DA showed higher discrimination performance than the existing image-based Chlorophyll ratio. [ABSTRACT FROM AUTHOR]

Beijing Satellite 3 is a high-performance optical remote sensing satellite with a spatial resolution of 0.3–0.5 m. It can provide timely and independent ultra-high-resolution spatial big data and comprehensive spatial information application services. At present, there is no relevant research on the fusion method of BJ-3A satellite images. In many applications, high-resolution panchromatic images alone are insufficient. Therefore, it is necessary to fuse them with multispectral images that contain spectral color information. Currently, there is a lack of research on the fusion method of BJ-3A satellite images. This article explores six traditional pixel-level fusion methods (HPF, HCS, wavelet, modified-IHS, PC, and Brovey) for fusing the panchromatic image and multispectral image of the BJ-3A satellite. The fusion results were analyzed qualitatively from two aspects: spatial detail enhancement capability and spectral fidelity. Five indicators, namely mean, standard deviation, entropy, correlation coefficient, and average gradient, were used for quantitative analysis. Finally, the fusion results were comprehensively evaluated from three aspects: spectral curves of ground objects, absolute error figure, and object-oriented classification effects. The findings of the research suggest that the fusion method known as HPF is the optimum and appropriate technique for fusing panchromatic and multispectral images obtained from BJ-3A. These results can be utilized as a guide for the implementation of BJ-3A panchromatic and multispectral data fusion in real-world scenarios. [ABSTRACT FROM AUTHOR]

This article introduces an algorithm for detecting glucose and silicon levels in solution. The research focuses on addressing the critical need for accurate and efficient glucose monitoring, particularly in the context of diabetic management. Understanding and monitoring silicon levels in the body is crucial due to its significant role in various physiological processes. Silicon, while often overshadowed by other minerals, plays a vital role in bone health, collagen formation, and connective tissue integrity. Moreover, recent research suggests its potential involvement in neurological health and the prevention of certain degenerative diseases. Investigating silicon levels becomes essential for a comprehensive understanding of its impact on overall health and well-being and paves the way for targeted interventions and personalized healthcare strategies. The approach presented in this paper is based on the integration of hyperspectral data and artificial intelligence techniques. The algorithm investigates the effectiveness of two distinct models utilizing SVMR and a perceptron independently. SVMR is employed to establish a robust regression model that maps input features to continuous glucose and silicon values. The study outlines the methodology, including feature selection, model training, and evaluation metrics. Experimental results demonstrate the algorithm's effectiveness at accurately predicting glucose and silicon concentrations and showcases its potential for real-world application in continuous glucose and silicon monitoring systems. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification is a highly challenging task, particularly in fields like crop yield prediction and agricultural infrastructure detection. These applications often involve complex image types, such as soil, vegetation, water bodies, and urban structures, encompassing a variety of surface features. In HSI, the strong correlation between adjacent bands leads to redundancy in spectral information, while using image patches as the basic unit of classification causes redundancy in spatial information. To more effectively extract key information from this massive redundancy for classification, we innovatively proposed the CESA-MCFormer model, building upon the transformer architecture with the introduction of the Center Enhanced Spatial Attention (CESA) module and Morphological Convolution (MC). The CESA module combines hard coding and soft coding to provide the model with prior spatial information before the mixing of spatial features, introducing comprehensive spatial information. MC employs a series of learnable pooling operations, not only extracting key details in both spatial and spectral dimensions but also effectively merging this information. By integrating the CESA module and MC, the CESA-MCFormer model employs a "Selection–Extraction" feature processing strategy, enabling it to achieve precise classification with minimal samples, without relying on dimension reduction techniques such as PCA. To thoroughly evaluate our method, we conducted extensive experiments on the IP, UP, and Chikusei datasets, comparing our method with the latest advanced approaches. The experimental results demonstrate that the CESA-MCFormer achieved outstanding performance on all three test datasets, with Kappa coefficients of 96.38%, 98.24%, and 99.53%, respectively. [ABSTRACT FROM AUTHOR]

Vegetation in East Antarctica, such as moss and lichen, vulnerable to the effects of climate change and ozone depletion, requires robust non-invasive methods to monitor its health condition. Despite the increasing use of unmanned aerial vehicles (UAVs) to acquire high-resolution data for vegetation analysis in Antarctic regions through artificial intelligence (AI) techniques, the use of multispectral imagery and deep learning (DL) is quite limited. This study addresses this gap with two pivotal contributions: (1) it underscores the potential of deep learning (DL) in a field with notably limited implementations for these datasets; and (2) it introduces an innovative workflow that compares the performance between two supervised machine learning (ML) classifiers: Extreme Gradient Boosting (XGBoost) and U-Net. The proposed workflow is validated by detecting and mapping moss and lichen using data collected in the highly biodiverse Antarctic Specially Protected Area (ASPA) 135, situated near Casey Station, between January and February 2023. The implemented ML models were trained against five classes: Healthy Moss, Stressed Moss, Moribund Moss, Lichen, and Non-vegetated. In the development of the U-Net model, two methods were applied: Method (1) which utilised the original labelled data as those used for XGBoost; and Method (2) which incorporated XGBoost predictions as additional input to that version of U-Net. Results indicate that XGBoost demonstrated robust performance, exceeding 85% in key metrics such as precision, recall, and F1-score. The workflow suggested enhanced accuracy in the classification outputs for U-Net, as Method 2 demonstrated a substantial increase in precision, recall and F1-score compared to Method 1, with notable improvements such as precision for Healthy Moss (Method 2: 94% vs. Method 1: 74%) and recall for Stressed Moss (Method 2: 86% vs. Method 1: 69%). These findings contribute to advancing non-invasive monitoring techniques for the delicate Antarctic ecosystems, showcasing the potential of UAVs, high-resolution multispectral imagery, and ML models in remote sensing applications. [ABSTRACT FROM AUTHOR]

Multi-spectral imaging technologies have made great progress in the past few decades. The development of snapshot cameras equipped with a specific multi-spectral filter array (MSFA) allow dynamic scenes to be captured on a miniaturized platform across multiple spectral bands, opening up extensive applications in quantitative and visualized analysis. However, a snapshot camera based on MSFA captures a single band per pixel; thus, the other spectral band components of pixels are all missed. The raw images, which are captured by snapshot multi-spectral imaging systems, require a reconstruction procedure called demosaicing to estimate a fully defined multi-spectral image (MSI). With increasing spectral bands, the challenge of demosaicing becomes more difficult. Furthermore, the existing demosaicing methods will produce adverse artifacts and aliasing because of the adverse effects of spatial interpolation and the inadequacy of the number of layers in the network structure. In this paper, a novel multi-spectral demosaicing method based on a deep convolution neural network (CNN) is proposed for the reconstruction of full-resolution multi-spectral images from raw MSFA-based spectral mosaic images. The CNN is integrated with the channel attention mechanism to protect important channel features. We verify the merits of the proposed method using 5 × 5 raw mosaic images on synthetic as well as real-world data. The experimental results show that the proposed method outperforms the existing demosaicing methods in terms of spatial details and spectral fidelity. [ABSTRACT FROM AUTHOR]

This paper proposes a method for demosaicing raw images captured by multispectral cameras. The proposed method estimates a pseudo-panchromatic image (PPI) via an iterative-linear-regression model and utilizes the estimated PPI for multispectral demosaicing. The PPI is estimated through horizontal and vertical guided filtering, with the subsampled multispectral-filter-array-(MSFA) image and low-pass-filtered MSFA as the guide image and filtering input, respectively. The number of iterations is automatically determined according to a predetermined criterion. Spectral differences between the estimated PPI and MSFA are calculated for each channel, and each spectral difference is interpolated using directional interpolation. The weights are calculated from the estimated PPI, and each interpolated spectral difference is combined using the weighted sum. The experimental results indicate that the proposed method outperforms the State-of-the-Art methods with regard to spatial and spectral fidelity for both synthetic and real-world images. [ABSTRACT FROM AUTHOR]

It is important to maintain the safety of road driving by automatically performing a series of processes to automatically measure and repair damage to the road pavement. However, road pavements include not only damages such as longitudinal cracks, transverse cracks, alligator cracks, and potholes, but also various elements such as manholes, road marks, oil marks, shadows, and joints. Therefore, in order to separate categories that exist in various road pavements, in this paper, 13,500 digital, IR, and MSX images were collected and nine categories were automatically classified by DarkNet. The DarkNet classification accuracies of digital images, IR images, and MSX images are 97.4%, 80.1%, and 91.1%, respectively. The MSX image is a enhanced image of the IR image and showed an average of 6% lower accuracy than the digital image but an average of 11% higher accuracy than the IR image. Therefore, MSX images can play a complementary role if DarkNet classification is performed together with digital images. In this paper, a method for detecting the directionality of each crack through a two-dimensional wavelet transform is presented, and this result can contribute to future research on detecting cracks in pavements. [ABSTRACT FROM AUTHOR]

This study aims to understand the dynamic changes in the coral reef habitats of Derawan Island over two decades (2003, 2011, and 2021) using advanced machine learning classification techniques. The motivation stems from the urgent need for accurate, detailed environmental monitoring to inform conservation strategies, particularly in ecologically sensitive areas like coral reefs. We employed non-parametric machine learning algorithms, including Random Forest (RF), Support Vector Machine (SVM), and Classification and Regression Tree (CART), to assess spatial and temporal changes in coral habitats. Our analysis utilized high-resolution data from Landsat 9, Landsat 7, Sentinel-2, and Multispectral Aerial Photos. The RF algorithm proved to be the most accurate, achieving an accuracy of 71.43% with Landsat 9, 73.68% with Sentinel-2, and 78.28% with Multispectral Aerial Photos. Our findings indicate that the classification accuracy is significantly influenced by the geographic resolution and the quality of the field and satellite/aerial image data. Over the two decades, there was a notable decrease in the coral reef area from 2003 to 2011, with a reduction to 16 hectares, followed by a slight increase in area but with more heterogeneous densities between 2011 and 2021. The study underscores the dynamic nature of coral reef habitats and the efficacy of machine learning in environmental monitoring. The insights gained highlight the importance of advanced analytical methods in guiding conservation efforts and understanding ecological changes over time. [ABSTRACT FROM AUTHOR]

In golf, the location of the impact, where the clubhead hits the ball, is of imperative nature for a successful ballflight. Direct feedback to the athlete where he/she hits the ball could improve a practice session. Currently, this information can be measured via, e.g., dual laser technology; however, this is a stationary and external method. A mobile measurement method would give athletes the freedom to gain the information of the impact location without the limitation to be stationary. Therefore, the aim of this study was to investigate whether it is possible to detect the impact location via a motion sensor mounted on the shaft of the golf club. To answer the question, an experiment was carried out. Within the experiment data were gathered from one athlete performing 282 golf swings with an 7 iron. The impact location was recorded and labeled during each swing with a Trackman providing the classes for a neural network. Simultaneously, the motion of the golf club was gathered with an IMU from the Noraxon Ultium Motion Series. In the next step, a neural network was designed and trained to estimate the impact location class based on the motion data. Based on the motion data, a classification accuracy of 93.8% could be achieved with a ResNet architecture. [ABSTRACT FROM AUTHOR]

In the food industry, quality and safety issues are associated with consumers' health condition. There is a growing interest in applying various noninvasive sensorial techniques to obtain quickly quality attributes. One of them, hyperspectral/multispectral imaging technique has been extensively used for inspection of various food products. In this paper, a stacking-based ensemble prediction system has been developed for the prediction of total viable counts of microorganisms in beef fillet samples, an essential cause to meat spoilage, utilizing multispectral imaging information. As the selection of important wavelengths from the multispectral imaging system is considered as an essential stage to the prediction scheme, a features fusion approach has been also explored, by combining wavelengths extracted from various feature selection techniques. Ensemble sub-components include two advanced clustering-based neuro-fuzzy network prediction models, one utilizing information from average reflectance values, while the other one from the standard deviation of the pixels' intensity per wavelength. The performances of neurofuzzy models were compared against established regression algorithms such as multilayer perceptron, support vector machines and partial least squares. Obtained results confirmed the validity of the proposed hypothesis to utilize a combination of feature selection methods with neurofuzzy models in order to assess the microbiological quality of meat products. [ABSTRACT FROM AUTHOR]

This article presents an integrated system that uses the capabilities of unmanned aerial vehicles (UAVs) to perform a comprehensive crop analysis, combining qualitative and quantitative evaluations for efficient agricultural management. A convolutional neural network-based model, Detectron2, serves as the foundation for detecting and segmenting objects of interest in acquired aerial images. This model was trained on a dataset prepared using the COCO format, which features a variety of annotated objects. The system architecture comprises a frontend and a backend component. The frontend facilitates user interaction and annotation of objects on multispectral images. The backend involves image loading, project management, polygon handling, and multispectral image processing. For qualitative analysis, users can delineate regions of interest using polygons, which are then subjected to analysis using the Normalized Difference Vegetation Index (NDVI) or Optimized Soil Adjusted Vegetation Index (OSAVI). For quantitative analysis, the system deploys a pre-trained model capable of object detection, allowing for the counting and localization of specific objects, with a focus on young lettuce crops. The prediction quality of the model has been calculated using the AP (Average Precision) metric. The trained neural network exhibited robust performance in detecting objects, even within small images. [ABSTRACT FROM AUTHOR]

Selecting training samples is crucial in remote sensing image classification. In this paper, we selected three images—Sentinel-2, GF-1, and Landsat 8—and employed three methods for selecting training samples: grouping selection, entropy-based selection, and direct selection. We then used the selected training samples to train three supervised classification models—random forest (RF), support-vector machine (SVM), and k-nearest neighbor (KNN)—and evaluated the classification results of the three images. According to the experimental results, the three classification models performed similarly. Compared with the entropy-based method, the grouping selection method achieved higher classification accuracy using fewer samples. In addition, the grouping selection method outperformed the direct selection method with the same number of samples. Therefore, the grouping selection method performed the best. When using the grouping selection method, the image classification accuracy increased with the increase in the number of samples within a certain sample size range. [ABSTRACT FROM AUTHOR]

To improve the management of multispectral sensor systems on small reconnaissance drones, this paper proposes an approach to predict the performance of a sensor band with respect to its ability to expose camouflaged targets under a given environmental context. As a reference for sensor performance, a new metric is introduced that quantifies the visibility of camouflaged targets in a particular sensor band: the Target Visibility Index (TVI). For the sensor performance prediction, several machine learning models are trained to learn the relationship between the TVI for a specific sensor band and an environmental context state extracted from the visual band by multiple image descriptors. Using a predicted measure of performance, the sensor bands are ranked according to their significance. For the training and evaluation of the performance prediction approach, a dataset featuring 853 multispectral captures and numerous camouflaged targets in different environments was created and has been made publicly available for download. The results show that the proposed approach can successfully determine the most informative sensor bands in most cases. Therefore, this performance prediction approach has great potential to improve camouflage detection performance in real-world reconnaissance scenarios by increasing the utility of each sensor band and reducing the associated workload of complex multispectral sensor systems. [ABSTRACT FROM AUTHOR]

Defect segmentation of apples is an important task in the agriculture industry for quality control and food safety. In this paper, we propose a deep learning approach for the automated segmentation of apple defects using convolutional neural networks (CNNs) based on a U-shaped architecture with skip-connections only within the noise reduction block. An ad-hoc data synthesis technique has been designed to increase the number of samples and at the same time to reduce neural network overfitting. We evaluate our model on a dataset of multi-spectral apple images with pixel-wise annotations for several types of defects. In this paper, we show that our proposal outperforms in terms of segmentation accuracy general-purpose deep learning architectures commonly used for segmentation tasks. From the application point of view, we improve the previous methods for apple defect segmentation. A measure of the computational cost shows that our proposal can be employed in real-time (about 100 frame-per-second on GPU) and in quasi-real-time (about 7/8 frame-per-second on CPU) visual-based apple inspection. To further improve the applicability of the method, we investigate the potential of using only RGB images instead of multi-spectral images as input images. The results prove that the accuracy in this case is almost comparable with the multi-spectral case. [ABSTRACT FROM AUTHOR]

This study introduces a prototype end-to-end Simulator software tool for simulating two-dimensional satellite multispectral imagery for a variety of satellite instrument models in aquatic environments. Using case studies, the impact of variable sensor configurations on the performance of value-added products for challenging applications, such as coral reefs and cyanobacterial algal blooms, is assessed. This demonstrates how decisions regarding satellite sensor design, driven by cost constraints, directly influence the quality of value-added remote sensing products. Furthermore, the Simulator is used to identify situations where retrieval algorithms require further parameterization before application to unsimulated satellite data, where error sources cannot always be identified or isolated. The application of the Simulator can verify whether a given instrument design meets the performance requirements of end-users before build and launch, critically allowing for the justification of the cost and specifications for planned and future sensors. It is hoped that the Simulator will enable engineers and scientists to understand important design trade-offs in phase 0/A studies easily, quickly, reliably, and accurately in future Earth observation satellites and systems. [ABSTRACT FROM AUTHOR]

Precise identification and spatial analysis of land salinity in China's Yellow River Delta are essential for the rational utilization and sustainable development of land resources. However, the accurate retrieval model construction for monitoring land salinity remains challenging. This study constructed a land salinity retrieval framework using a harmonized UAV and Landsat-9 multi-spectral dataset. The Kenli district of the Yellow River Delta was selected as the case study area, and a land salinity monitoring index (LSMI) was proposed based on field survey data and UAV multi-spectral image and applied to the reflectance-corrected Landsat-9 OLI image. The land salinity distribution patterns were then mapped and spatially analyzed using Moran's I and Getis-Ord GI* analysis. The results demonstrated the following: (1) The LSMI-based method can accurately retrieve land salinity content with a validation determination coefficient (R2), root mean square error (RMSE), and residual predictive deviation (RPD) of 0.75, 1.89, and 2.11, respectively. (2) Land salinization affected 93.12% of the cultivated land in the study area, and the severely saline soil grade (with a salinity content of 6–8 g/kg) covered 38.41% of the total cultivated land area and was widely distributed throughout the study area. (3) Saline land exhibited a positive spatial autocorrelation with a value of 0.311 at the p = 0.000 level; high–high cluster types occurred mainly in the Kendong and Huanghekou towns (80%), while low–low cluster types were mainly located in the Dongji, Haojia, Kenli, and Shengtuo towns (88.46%). The spatial characteristics of various salinity grades exhibit significant variations, and conducting separate spatial analyses is recommended for future studies. [ABSTRACT FROM AUTHOR]

Multispectral imaging is valuable in many vision-related fields as it provides an additional modality to observe the world. Cameras equipped with multispectral filter arrays (MSFAs) are typically impractical for everyday use due to their intractable demosaicking and chromatic reproduction processes, which restrict their applicability beyond academic research. In this work, a novel MSFA design is proposed to enable dual-mode imaging for multispectral cameras. In addition to a conventional multispectral image, the camera is also able to produce a Bayer-formed RGB image from a single shot by grouping and merging adjacent pixels in the proposed MSFA, making it suitable for scenarios where display-ready RGB images are required. Furthermore, a two-stage optimization scheme is implemented to jointly optimize objective functions for both imaging modes. The evaluation results on multiple datasets suggest that the proposed MSFA design is able to simultaneously achieve competitive spectral reconstruction accuracy compared to elaborate multispectral cameras and chromatic accuracy compared to commercial RGB cameras. [ABSTRACT FROM AUTHOR]

A geologic map is both a visual depiction of the lithologies and structures occurring at the Earth's surface and a representation of a conceptual model for the geologic history in a region. The work needed to capture such multifaced information in an accurate geologic map is time consuming. Remote sensing can complement traditional primary field observations, geochemistry, chronometry, and subsurface geophysical data in providing useful information to assist with the geologic mapping process. Two novel sources of remote sensing data are particularly relevant for geologic mapping applications: decameter-resolution imaging spectroscopy (spectroscopic imaging) and meter-resolution multispectral shortwave infrared (SWIR) imaging. Decameter spectroscopic imagery can capture important mineral absorptions but is frequently unable to spatially resolve important geologic features. Meter-resolution multispectral SWIR images are better able to resolve fine spatial features but offer reduced spectral information. Such disparate but complementary datasets can be challenging to integrate into the geologic mapping process. Here, we conduct a comparative analysis of spatial and spectral scaling for two such datasets: one Airborne Visible/Infrared Imaging Spectrometer—Classic (AVIRIS-classic) flightline, and one WorldView-3 (WV3) scene, for a geologically complex landscape in Anza-Borrego Desert State Park, California. To do so, we use a two-stage framework that synthesizes recent advances in the spectral mixture residual and joint characterization. The mixture residual uses the wavelength-explicit misfit of a linear spectral mixture model to capture low variance spectral signals. Joint characterization utilizes nonlinear dimensionality reduction (manifold learning) to visualize spectral feature space topology and identify clusters of statistically similar spectra. For this study area, the spectral mixture residual clearly reveals greater spectral dimensionality in AVIRIS than WorldView (99% of variance in 39 versus 5 residual dimensions). Additionally, joint characterization shows more complex spectral feature space topology for AVIRIS than WorldView, revealing information useful to the geologic mapping process in the form of mineralogical variability both within and among mapped geologic units. These results illustrate the potential of recent and planned imaging spectroscopy missions to complement high-resolution multispectral imagery—along with field and lab observations—in planning, collecting, and interpreting the results from geologic field work. [ABSTRACT FROM AUTHOR]

Multispectral satellite imagery offers a new perspective for spatial modelling, change detection and land cover classification. The increased demand for accurate classification of geographically diverse regions led to advances in object-based methods. A novel spatiotemporal method is presented for object-based land cover classification of satellite imagery using a Graph Neural Network. This paper introduces innovative representation of sequential satellite images as a directed graph by connecting segmented land region through time. The method's novel modular node classification pipeline utilises the Convolutional Neural Network as a multispectral image feature extraction network, and the Graph Neural Network as a node classification model. To evaluate the performance of the proposed method, we utilised EfficientNetV2-S for feature extraction and the GraphSAGE algorithm with Long Short-Term Memory aggregation for node classification. This innovative application on Sentinel-2 L2A imagery produced complete 4-year intermonthly land cover classification maps for two regions: Graz in Austria, and the region of Portorož, Izola and Koper in Slovenia. The regions were classified with Corine Land Cover classes. In the level 2 classification of the Graz region, the method outperformed the state-of-the-art UNet model, achieving an average F1-score of 0.841 and an accuracy of 0.831, as opposed to UNet's 0.824 and 0.818, respectively. Similarly, the method demonstrated superior performance over UNet in both regions under the level 1 classification, which contains fewer classes. Individual classes have been classified with accuracies up to 99.17%. [ABSTRACT FROM AUTHOR]

In this paper, a unified optimization model for medical image fusion based on tensor decomposition and the non-subsampled shearlet transform (NSST) is proposed. The model is based on the NSST method and the tensor decomposition method to fuse the high-frequency (HF) and low-frequency (LF) parts of two source images to obtain a mixed-frequency fused image. In general, we integrate low-frequency and high-frequency information from the perspective of tensor decomposition (TD) fusion. Due to the structural differences between the high-frequency and low-frequency representations, potential information loss may occur in the fused images. To address this issue, we introduce a joint static and dynamic guidance (JSDG) technique to complement the HF/LF information. To improve the result of the fused images, we combine the alternating direction method of multipliers (ADMM) algorithm with the gradient descent method for parameter optimization. Finally, the fused images are reconstructed by applying the inverse NSST to the fused high-frequency and low-frequency bands. Extensive experiments confirm the superiority of our proposed TDFusion over other comparison methods. [ABSTRACT FROM AUTHOR]

Rice lodging causes a loss of yield and leads to lower-quality rice. In Japan, Koshihikari is the most popular rice variety, and it has been widely cultivated for many years despite its susceptibility to lodging. Reducing basal fertilizer is recommended when the available nitrogen in soil (SAN) exceeds the optimum level (80–200 mg N kg−1). However, many commercial farmers prefer to simultaneously apply one-shot basal fertilizer at transplant time. This study investigated the relationship between the rice lodging and SAN content by assessing their spatial distributions from unmanned aircraft system (UAS) images in a Koshihikari paddy field where one-shot basal fertilizer was applied. We analyzed the severity of lodging using the canopy height model and spatially clarified a heavily lodged area and a non-lodged area. For the SAN assessment, we selected green and red band pixel digital numbers from multispectral images and developed a SAN estimating equation by regression analysis. The estimated SAN values were rasterized and compiled into a 1 m mesh to create a soil fertility map. The heavily lodged area roughly coincided with the higher SAN area. A negative correlation was observed between the rice inclination angle and the estimated SAN, and rice lodging occurred even within the optimum SAN level. These results show that the amount of one-shot basal fertilizer applied to Koshihikari should be reduced when absorbable nitrogen (SAN + fertilizer nitrogen) exceeds 200 mg N kg−1. [ABSTRACT FROM AUTHOR]