Search

The necessity for intelligent welding robots that meet the demand in real industrial production, according to the objectives of Industry 4.0, has been supported owing to the rapid development of computer vision and the use of new technologies. To improve the efficiency in weld location for industrial robots, this work focuses on trajectory extraction based on color features identification on three-dimensional surfaces acquired with a depth-RGB sensor. The system is planned to be used with a low-cost Intel RealSense D435 sensor for the reconstruction of 3D models based on stereo vision and the built-in color sensor to quickly identify the objective trajectory, since the parts to be welded are previously marked with different colors, indicating the locations of the welding trajectories to be followed. This work focuses on 3D color segmentation with which the points of the target trajectory are segmented by color thresholds in HSV color space and a spline cubic interpolation algorithm is implemented to obtain a smooth trajectory. Experimental results have shown that the RMSE error for V-type butt joint path extraction was under 1.1 mm and below 0.6 mm for a straight butt joint, in addition, the system seems to be suitable for welding beads of various shapes.

This study reviews obstacle detection technologies in vegetation for autonomous vehicles or robots. Autonomous vehicles used in agriculture and as lawn mowers face many environmental obstacles that are difficult to recognize for the vehicle sensor. This review provides information on choosing appropriate sensors to detect obstacles through vegetation, based on experiments carried out in different agricultural fields. The experimental setup from the literature consists of sensors placed in front of obstacles, including a thermal camera; red, green, blue (RGB) camera; 360° camera; light detection and ranging (LiDAR); and radar. These sensors were used either in combination or single-handedly on agricultural vehicles to detect objects hidden inside the agricultural field. The thermal camera successfully detected hidden objects, such as barrels, human mannequins, and humans, as did LiDAR in one experiment. The RGB camera and stereo camera were less efficient at detecting hidden objects compared with protruding objects. Radar detects hidden objects easily but lacks resolution. Hyperspectral sensing systems can identify and classify objects, but they consume a lot of storage. To obtain clearer and more robust data of hidden objects in vegetation and extreme weather conditions, further experiments should be performed for various climatic conditions combining active and passive sensors.

For many decades the multispectral images of the earth&rsquo, s surface and its objects were taken from multispectral sensors placed on satellites. In recent years, the technological evolution produced similar sensors (much smaller in size and weight) which can be placed on Unmanned Aerial Vehicles (UAVs), thereby allowing the collection of higher spatial resolution multispectral images. In this paper, Parrot&rsquo, s small Multispectral (MS) camera Sequoia+ is used, and its images are evaluated at two archaeological sites, on the Byzantine wall (ground application) of Thessaloniki city (Greece) and on a mosaic floor (aerial application) at the archaeological site of Dion (Greece). The camera receives RGB and MS images simultaneously, a fact which does not allow image fusion to be performed, as in the standard utilization procedure of Panchromatic (PAN) and MS image of satellite passive systems. In this direction, that is, utilizing the image fusion processes of satellite PAN and MS images, this paper demonstrates that with proper digital processing the images (RGB and MS) of small MS cameras can lead to a fused image with a high spatial resolution, which retains a large percentage of the spectral information of the original MS image. The high percentage of spectral fidelity of the fused images makes it possible to perform high-precision digital measurements in archaeological sites such as the accurate digital separation of the objects, area measurements and retrieval of information not so visible with common RGB sensors via the MS and RGB data of small MS sensors.

Leaf area index (LAI) is a vital parameter for predicting rice yield. Unmanned aerial vehicle (UAV) surveillance with an RGB camera has been shown to have potential as a low-cost and efficient tool for monitoring crop growth. Simultaneously, deep learning (DL) algorithms have attracted attention as a promising tool for the task of image recognition. The principal aim of this research was to evaluate the feasibility of combining DL and RGB images obtained by a UAV for rice LAI estimation. In the present study, an LAI estimation model developed by DL with RGB images was compared to three other practical methods: a plant canopy analyzer (PCA); regression models based on color indices (CIs) obtained from an RGB camera; and vegetation indices (VIs) obtained from a multispectral camera. The results showed that the estimation accuracy of the model developed by DL with RGB images (R2 = 0.963 and RMSE = 0.334) was higher than those of the PCA (R2 = 0.934 and RMSE = 0.555) and the regression models based on CIs (R2 = 0.802-0.947 and RMSE = 0.401–1.13), and comparable to that of the regression models based on VIs (R2 = 0.917–0.976 and RMSE = 0.332–0.644). Therefore, our results demonstrated that the estimation model using DL with an RGB camera on a UAV could be an alternative to the methods using PCA and a multispectral camera for rice LAI estimation.

Most scenes in practical applications are dynamic scenes containing moving objects, so accurately segmenting moving objects is crucial for many computer vision applications. In order to efficiently segment all the moving objects in the scene, regardless of whether the object has a predefined semantic label, we propose a two-level nested octave U-structure network with a multi-scale attention mechanism, called U2-ONet. U2-ONet takes two RGB frames, the optical flow between these frames, and the instance segmentation of the frames as inputs. Each stage of U2-ONet is filled with the newly designed octave residual U-block (ORSU block) to enhance the ability to obtain more contextual information at different scales while reducing the spatial redundancy of the feature maps. In order to efficiently train the multi-scale deep network, we introduce a hierarchical training supervision strategy that calculates the loss at each level while adding knowledge-matching loss to keep the optimization consistent. The experimental results show that the proposed U2-ONet method can achieve a state-of-the-art performance in several general moving object segmentation datasets.

The proportion of welding work in total man-hours required for shipbuilding processes has been perceived to be significant, and welding man-hours are greatly affected by working posture. Continuous research has been conducted to identify the posture in welding by utilizing the relationship between man-hours and working posture. However, the results that reflect the effect of the welding posture on man-hours are not available. Although studies on posture recognition based on depth image analysis are being positively reviewed, welding operation has difficulties in image interpretation because an external obstacle caused by arcs exists. Therefore, any obstacle element must be removed in advance. This study proposes a method to acquire work postures using a low-cost RGB-D camera and recognize the welding position through image analysis. It removes obstacles that appear as depth holes in the depth image and restores the removed part to the desired state. The welder’s body joints are extracted, and a convolution neural network is used to determine the corresponding welding position. The restored image showed significantly improved recognition accuracy. The proposed method acquires, analyzes, and automates the recognition of welding positions in real-time. It can be applied to all areas where image interpretation is difficult due to obstacles.

Local features extraction is a crucial technology for image matching navigation of an unmanned aerial vehicle (UAV), where it aims to accurately and robustly match a real-time image and a geo-referenced image to obtain the position update information of the UAV. However, it is a challenging task due to the inconsistent image capture conditions, which will lead to extreme appearance changes, especially the different imaging principle between an infrared image and RGB image. In addition, the sparsity and labeling complexity of existing public datasets hinder the development of learning-based methods in this research area. This paper proposes a novel learning local features extraction method, which uses local features extracted by deep neural network to find the correspondence features on the satellite RGB reference image and real-time infrared image. First, we propose a single convolution neural network that simultaneously extracts dense local features and their corresponding descriptors. This network combines the advantages of a high repeatability local feature detector and high reliability local feature descriptors to match the reference image and real-time image with extreme appearance changes. Second, to make full use of the sparse dataset, an iterative training scheme is proposed to automatically generate the high-quality corresponding features for algorithm training. During the scheme, the dense correspondences are automatically extracted, and the geometric constraints are added to continuously improve the quality of them. With these improvements, the proposed method achieves state-of-the-art performance for infrared aerial (UAV captured) image and satellite reference image, which shows 4–6% performance improvements in precision, recall, and F1-score, compared to the other methods. Moreover, the applied experiment results show its potential and effectiveness on localization for UAVs navigation and trajectory reconstruction application.

The integral imaging system has received considerable research attention because it can be applied to real-time three-dimensional image displays with a continuous view angle without supplementary devices. Most previous approaches place a physical micro-lens array in front of the image, where each lens looks different depending on the viewing angle. A computational integral imaging system with a virtual micro-lens arrays has been proposed in order to provide flexibility for users to change micro-lens arrays and focal length while reducing distortions due to physical mismatches with the lens arrays. However, computational integral imaging methods only represent part of the whole image because the size of virtual lens arrays is much smaller than the given large-scale images when dealing with large-scale images. As a result, the previous approaches produce sub-aperture images with a small field of view and need additional devices for depth information to apply to integral imaging pickup systems. In this paper, we present a single image-based computational RGB-D integral imaging pickup system for a large field of view in real time. The proposed system comprises three steps: deep learning-based automatic depth map estimation from an RGB input image without the help of an additional device, a hierarchical integral imaging system for a large field of view in real time, and post-processing for optimized visualization of the failed pickup area using an inpainting method. Quantitative and qualitative experimental results verify the proposed approach’s robustness.

In the recent decade, the rapid development of drone technologies has made many spatial problems easier to solve, including the problem of 3D reconstruction of large objects. A review of existing solutions has shown that most of the works lack the autonomy of drones because of nonscalable mapping techniques. This paper presents a method for centralized multi-drone 3D reconstruction, which allows performing a data capturing process autonomously and requires drones equipped only with an RGB camera. The essence of the method is a multiagent approach—the control center performs the workload distribution evenly and independently for all drones, allowing simultaneous flights without a high risk of collision. The center continuously receives RGB data from drones and performs each drone localization (using visual odometry estimations) and rough online mapping of the environment (using image descriptors for estimating the distance to the building). The method relies on a set of several user-defined parameters, which allows the tuning of the method for different task-specific requirements such as the number of drones, 3D model detalization, data capturing time, and energy consumption. By numerical experiments, it is shown that method parameters can be estimated by performing a set of computations requiring characteristics of drones and the building that are simple to obtain. Method performance was evaluated by an experiment with virtual building and emulated drone sensors. Experimental evaluation showed that the precision of the chosen algorithms for online localization and mapping is enough to perform simultaneous flights and the amount of captured RGB data is enough for further reconstruction.

Human action recognition (HAR) by skeleton data is considered a potential research aspect in computer vision. Three-dimensional HAR with skeleton data has been used commonly because of its effective and efficient results. Several models have been developed for learning spatiotemporal parameters from skeleton sequences. However, two critical problems exist: (1) previous skeleton sequences were created by connecting different joints with a static order, (2) earlier methods were not efficient enough to focus on valuable joints. Specifically, this study aimed to (1) demonstrate the ability of convolutional neural networks to learn spatiotemporal parameters of skeleton sequences from different frames of human action, and (2) to combine the process of all frames created by different human actions and fit in the spatial structure information necessary for action recognition, using multi-task learning networks (MTLNs). The results were significantly improved compared with existing models by executing the proposed model on an NTU RGB+D dataset, an SYSU dataset, and an SBU Kinetic Interaction dataset. We further implemented our model on noisy expected poses from subgroups of the Kinetics dataset and the UCF101 dataset. The experimental results also showed significant improvement using our proposed model.

The accurate estimation of grain yield in rice breeding is crucial for breeders to screen and select qualified cultivars. In this study, a low-cost unmanned aerial vehicle (UAV) platform mounted with an RGB camera was carried out to capture high-spatial resolution images of rice canopy in rice breeding. The random forest (RF) regression techniques were used to establish yield models by using (1) only color vegetation indices (VIs), (2) only phenological data, and (3) fusion of VIs and phenological data as inputs, respectively. Then, the performances of RF models were compared with the manual observation and CERES-Rice model. The results indicated that the RF model using VIs only performed poorly for estimating yield; the optimized RF model that combined the use of phenological data and color VIs performed much better, which demonstrated that the phenological data significantly improved the model performance. Furthermore, the yield estimation accuracy of 21 rice cultivars that were continuously planted over three years in the optimal RF model had no significant difference (p > 0.05) with that of the CERES-Rice model. These findings demonstrate that the RF model, by combining phenological data and color Vis, is a potential and cost-effective way to estimate yield in rice breeding.

Relative to standard red/green/blue (RGB) imaging systems, hyperspectral imaging systems offer superior capabilities but tend to be expensive and complex, requiring either a mechanically complex push-broom line scanning method, a tunable filter, or a large set of light emitting diodes (LEDs) to collect images in multiple wavelengths. This paper proposes a new methodology to support the design of a hypothesized system that uses three imaging modes—fluorescence, visible/near-infrared (VNIR) reflectance, and shortwave infrared (SWIR) reflectance—to capture narrow-band spectral data at only three to seven narrow wavelengths. Simulated annealing is applied to identify the optimal wavelengths for sparse spectral measurement with a cost function based on the accuracy provided by a weighted k-nearest neighbors (WKNN) classifier, a common and relatively robust machine learning classifier. Two separate classification approaches are presented, the first using a multi-layer perceptron (MLP) artificial neural network trained on sparse data from the three individual spectra and the second using a fusion of the data from all three spectra. The results are compared with those from four alternative classifiers based on common machine learning algorithms. To validate the proposed methodology, reflectance and fluorescence spectra in these three spectroscopic modes were collected from fish fillets and used to classify the fillets by species. Accuracies determined from the two classification approaches are compared with benchmark values derived by training the classifiers with the full resolution spectral data. The results of the single-layer classification study show accuracies ranging from ~68% for SWIR reflectance to ~90% for fluorescence with just seven wavelengths. The results of the fusion classification study show accuracies of about 95% with seven wavelengths and more than 90% even with just three wavelengths. Reducing the number of required wavelengths facilitates the creation of rapid and cost-effective spectral imaging systems that can be used for widespread analysis in food monitoring/food fraud, agricultural, and biomedical applications.

Hand-eye calibration and three-dimensional space target positioning are the keys to realize the automatic grasping of the manipulator. Aiming at the problems of a difficult camera manipulator calibration and poor real-time segmentation and positioning of stacked targets in industrial environment, a rapid target positioning method of self-calibration manipulator is proposed. Firstly, the spatial spherical autonomous path planning is carried out based on the quaternion linear interpolation method to calculate spatially symmetric path trajectory of the calibrator. The RGB-D camera mounted at the end of the manipulator is used to obtain multiple groups of RGB and depth images of the calibration plate. Combined with the position and attitude of the manipulator end, the internal and external parameters of the camera and the hand-eye conversion matrix are automatically calibrated. Then, based on KD tree algorithm, the holes in the point cloud are extracted to plan the shooting pose of the complementary image, and the target object is photographed from multiple symmetric angles. Combined with the iterative shooting pose of the manipulator, the rapid registration of the point cloud is realized and the complete outer surface model of the target object is obtained. Finally, the improved double pyramid feature fusion depth image is used to segment the RGB image through Mask R-CNN and mapped to the point cloud space, to achieve fast target segmentation of end-to-end 3D point cloud. The experimental results show that the eye-in-hand manipulator system can be self-calibrated to greatly simplify the calibration process and achieve the calibration accuracy of the traditional calibration method. The average error in each direction of the calibration result is less than 2 mm, which can achieve the acquisition accuracy of the vision sensor. It can also register and reconstruct point clouds for complex scenes in 1 s. The improved Mask-RCNN increases the segmentation accuracy of stacking objects by 8%. Compared with the physical error of hardware, the positioning error is no more than 0.89% and can meet the requirements of practical applications.

Vegetation inspection and monitoring is a time-consuming task. In the era of industrial revolution 4.0 (IR 4.0), unmanned aerial vehicles (UAV), commercially known as drones, are in demand, being adopted for vegetation inspection and monitoring activities. However, most off-the-shelf drones are least favoured by vegetation maintenance departments for on-site inspection due to limited spectral bands camera restricting advanced vegetation analysis. Most of these drones are normally equipped with a normal red, green, and blue (RGB) camera. Additional spectral bands are found to produce more accurate analysis during vegetation inspection, but at the cost of advanced camera functionalities, such as multispectral camera. Vegetation indices (VI) is a technique to maximize detection sensitivity related to vegetation characteristics while minimizing other factors which are not categorised otherwise. The emergence of machine learning has slowly influenced the existing vegetation analysis technique in order to improve detection accuracy. This study focuses on exploring VI techniques in identifying vegetation objects. The selected VIs investigated are Visible Atmospheric Resistant Index (VARI), Green Leaf Index (GLI), and Vegetation Index Green (VIgreen). The chosen machine learning technique is You Only Look Once (YOLO), which is a clever convolutional neural network (CNN) offering object detection in real time. The CNN model has a symmetrical structure along the direction of the tensor flow. Several series of data collection have been conducted at identified locations to obtain aerial images. The proposed hybrid methods were tested on captured aerial images to observe vegetation detection performance. Segmentation in image analysis is a process to divide the targeted pixels for further detection testing. Based on our findings, more than 70% of the vegetation objects in the images were accurately detected, which reduces the misdetection issue faced by previous VI techniques. On the other hand, hybrid segmentation methods perform best with the combination of VARI and YOLO at 84% detection accuracy.

The extraction of wheat lodging is of great significance to post-disaster agricultural production management, disaster assessment and insurance subsidies. At present, the recognition of lodging wheat in the actual complex field environment still has low accuracy and poor real-time performance. To overcome this gap, first, four-channel fusion images, including RGB and DSM (digital surface model), as well as RGB and ExG (excess green), were constructed based on the RGB image acquired from unmanned aerial vehicle (UAV). Second, a Mobile U-Net model that combined a lightweight neural network with a depthwise separable convolution and U-Net model was proposed. Finally, three data sets (RGB, RGB + DSM and RGB + ExG) were used to train, verify, test and evaluate the proposed model. The results of the experiment showed that the overall accuracy of lodging recognition based on RGB + DSM reached 88.99%, which is 11.8% higher than that of original RGB and 6.2% higher than that of RGB + ExG. In addition, our proposed model was superior to typical deep learning frameworks in terms of model parameters, processing speed and segmentation accuracy. The optimized Mobile U-Net model reached 9.49 million parameters, which was 27.3% and 33.3% faster than the FCN and U-Net models, respectively. Furthermore, for RGB + DSM wheat lodging extraction, the overall accuracy of Mobile U-Net was improved by 24.3% and 15.3% compared with FCN and U-Net, respectively. Therefore, the Mobile U-Net model using RGB + DSM could extract wheat lodging with higher accuracy, fewer parameters and stronger robustness.

New and emerging technologies, especially those based on non-invasive video and thermal infrared cameras, can be readily tested on robotic milking facilities. In this research, implemented non-invasive computer vision methods to estimate cow’s heart rate, respiration rate, and abrupt movements captured using RGB cameras and machine learning modelling to predict eye temperature, milk production and quality are presented. RGB and infrared thermal videos (IRTV) were acquired from cows using a robotic milking facility. Results from 102 different cows with replicates (n = 150) showed that an artificial neural network (ANN) model using only inputs from RGB cameras presented high accuracy (R = 0.96) in predicting eye temperature (°C), using IRTV as ground truth, daily milk productivity (kg-milk-day−1), cow milk productivity (kg-milk-cow−1), milk fat (%) and milk protein (%) with no signs of overfitting. The ANN model developed was deployed using an independent 132 cow samples obtained on different days, which also rendered high accuracy and was similar to the model development (R = 0.93). This model can be easily applied using affordable RGB camera systems to obtain all the proposed targets, including eye temperature, which can also be used to model animal welfare and biotic/abiotic stress. Furthermore, these models can be readily deployed in conventional dairy farms.

To implement the non-contact detection of contamination on insulators, a contamination severity assessment methodology using the deep learning of the colored image information of insulators can be used. For the insulator images taken at the substation site, a mathematical morphology-improved optimal entropic threshold (OET) method is utilized to extract the insulator from the background. By performing feature calculations of insulator images in RGB and HSI color spaces, sixty-six color features are obtained. By fusing the features of the two color spaces using kernel principal component analysis (KPCA), fused features are obtained. The recognition of contamination grades is then accomplished with a deep belief network (DBN) that consists of a three-layered restricted Boltzmann machine. The experimental results of the images taken on-site show that the fused features obtained by the KPCA can fully reflect the contamination state of the insulators. Compared with the identification obtained using RGB or HSI color-space features alone, accuracy is significantly improved, and insulator contamination grades can be effectively identified. The research provides a new method for the accurate, efficient, and non-contact detection of insulator contamination grades.

Hand pose estimation from RGB images has always been a difficult task, owing to the incompleteness of the depth information. Moon et al. improved the accuracy of hand pose estimation by using a new network, InterNet, through their unique design. Still, the network still has potential for improvement. Based on the architecture of MobileNet v3 and MoGA, we redesigned a feature extractor that introduced the latest achievements in the field of computer vision, such as the ACON activation function and the new attention mechanism module, etc. Using these modules effectively with our network, architecture can better extract global features from an RGB image of the hand, leading to a greater performance improvement compared to InterNet and other similar networks.

The detection of the spectral fingerprint of chemical sensors through the combined use of an LCD and a webcam is an alternative approach for chemical sensor characterization. This technique allows the development of more compact, cheap, and user-friendly measurement systems compared to the more classic instruments, such as spectrometers and gas chromatography systems. In the Computer Screen Photo-assisted Technique (CSPT), a display acts as a light source, and a conventional camera (e.g., a webcam) plays the role of a detector. The light from the LCD is reflected (or transmitted) by the chemical sensor, and the camera detects it. In the present contribution, we propose a compact and low-cost platform based on CSPT for the characterization of colorimetric sensor arrays. The system can provide spectral information of both reflected and transmitted light from the sample. Further, a 2.4-inch LCD and three different detector’s (a webcam, an RGB sensor, and a camera module) performances have been evaluated and discussed. The developed system includes a UDOO-based single board computer that makes it a stand-alone measurement system.

Accurate fruit segmentation in images is the prerequisite and key step for precision agriculture. In this article, aiming at the segmentation of grape cluster with different varieties, 3 state-of-the-art semantic segmentation networks, i.e., Fully Convolutional Network (FCN), U-Net, and DeepLabv3+ applied on six different datasets were studied. We investigated: (1) the segmentation performance difference of the 3 studied networks, (2) The impact of different input representations on segmentation performance, (3) The effect of image enhancement method to improve the poor illumination of images and further improve the segmentation performance, (4) The impact of the distance between grape clusters and camera on segmentation performance. The experiment results show that compared with FCN and U-Net the DeepLabv3+ combined with transfer learning is more suitable for the task with an intersection over union (IoU) of 84.26%. Five different input representations, namely RGB, HSV, L*a*b, HHH, and YCrCb obtained different IoU, ranging from 81.5% to 88.44%. Among them, the L*a*b got the highest IoU. Besides, the adopted Histogram Equalization (HE) image enhancement method could improve the model’s robustness against poor illumination conditions. Through the HE preprocessing, the IoU of the enhanced dataset increased by 3.88%, from 84.26% to 88.14%. The distance between the target and camera also affects the segmentation performance, no matter in which dataset, the closer the distance, the better the segmentation performance was. In a word, the conclusion of this research provides some meaningful suggestions for the study of grape or other fruit segmentation.

Graph Convolutional Networks (GCNs) have attracted a lot of attention and shown remarkable performance for action recognition in recent years. For improving the recognition accuracy, how to build graph structure adaptively, select key frames and extract discriminative features are the key problems of this kind of method. In this work, we propose a novel Adaptive Attention Memory Graph Convolutional Networks (AAM-GCN) for human action recognition using skeleton data. We adopt GCN to adaptively model the spatial configuration of skeletons and employ Gated Recurrent Unit (GRU) to construct an attention-enhanced memory for capturing the temporal feature. With the memory module, our model can not only remember what happened in the past but also employ the information in the future using multi-bidirectional GRU layers. Furthermore, in order to extract discriminative temporal features, the attention mechanism is also employed to select key frames from the skeleton sequence. Extensive experiments on Kinetics, NTU RGB+D and HDM05 datasets show that the proposed network achieves better performance than some state-of-the-art methods.

When reconstructing a 3D object, it is difficult to obtain accurate 3D geometric information using a single camera. In order to capture detailed geometric information of a 3D object, it is inevitable to increase the number of cameras to capture the object. However, cameras need to be synchronized in order to simultaneously capture frames. If cameras are incorrectly synchronized, many artifacts are produced in the reconstructed 3D object. The RealSense RGB-D camera, which is commonly used for obtaining geometric information of a 3D object, provides synchronization modes to mitigate synchronization errors. However, the synchronization modes provided by theRealSense cameras can only sync depth cameras and have limitations in the number of cameras that can be synchronized using a single host due to the hardware issue of stable data transmission. Therefore, in this paper, we propose a novel synchronization method that synchronizes an arbitrary number of RealSense cameras by adjusting the number of hosts to support stable data transmission. Our method establishes a master–slave architecture in order to synchronize the system clocks of the hosts. While synchronizing the system clocks, delays that resulted from the process of synchronization were estimated so that the difference between the system clocks could be minimized. Through synchronization of the system clocks, cameras connected to the different hosts can be synchronized based on the timestamp of the data received by the hosts. Thus, our method synchronizes theRealSense cameras to simultaneously capture accurate 3D information of an object at a constant frame rate without dropping it.

The development of UAV technologies offers practical methods to create landcover maps for monitoring and management of areas affected by natural disasters such as landslides. The present study aims at comparing the capability of two different types of UAV to deliver precise information, in order to characterize vegetation at landslide areas over a period of months. For the comparison, an RGB UAV and a Multispectral UAV were used to identify three different classes: vegetation, bare soil, and dead matter, from April to July 2021. The results showed high overall accuracy values (&gt, 95%) for the Multispectral UAV, as compared to the RGB UAV, which had lower overall accuracies. Although having lower overall accuracies, the vegetation class of the RGB UAV presented high producer’s and user’s accuracy over time, comparable to the Multispectral UAV results. Image quality played an important role in this study, where higher accuracy values were found on cloudy days. Both RGB and Multispectral UAVs presented similar patterns of vegetation, bare soil, and dead matter classes, where the increase in vegetation class was consistent with the decrease in bare soil and dead matter class. The present study suggests that the Multispectral UAV is more suitable in characterizing vegetation, bare soil, and dead matter classes on landslide areas while the RGB UAV can deliver reliable information for vegetation monitoring.

Colonoscopy screening and colonoscopic polypectomy can decrease the incidence and mortality rate of colorectal cancer (CRC). The adenoma detection rate and accuracy of diagnosis of colorectal polyp which vary in different experienced endoscopists have impact on the colonoscopy protection effect of CRC. The work proposed a colorectal polyp image detection and classification system through grayscale images and deep learning. The system collected the data of CVC-Clinic and 1000 colorectal polyp images of Linkou Chang Gung Medical Hospital. The red-green-blue (RGB) images were transformed to 0 to 255 grayscale images. Polyp detection and classification were performed by convolutional neural network (CNN) model. Data for polyp detection was divided into five groups and tested by 5-fold validation. The accuracy of polyp detection was 95.1% for grayscale images which is higher than 94.1% for RGB and narrow-band images. The diagnostic accuracy, precision and recall rates were 82.8%, 82.5% and 95.2% for narrow-band images, respectively. The experimental results show that grayscale images achieve an equivalent or even higher accuracy of polyp detection than RGB images for lightweight computation. It is also found that the accuracy of polyp detection and classification is dramatically decrease when the size of polyp images small than 1600 pixels. It is recommended that clinicians could adjust the distance between the lens and polyps appropriately to enhance the system performance when conducting computer-assisted colorectal polyp analysis.

Potato tuber shape is an important quality trait for breeding and variety development. Length to width (L/W) ratio is a commonly used method to score potato tubers for suitability for different markets and is relatively easy to measure, though labor intensive when done manually. L/W also does not adequately capture secondary growth and other tuber malformations that contribute to tuber shape. Tuber shape has a genetic component and is a prime target for early breeding selection. In the current study we developed an image analysis pipeline to extract tuber shape statistics from images taken using inexpensive, commercially available cameras. The image processing pipeline was used to evaluate greenhouse grown tubers from 32 unique crosses. Tubers from greenhouse grown plants were then grown in a field located in Vauxhall, AB, Canada, and evaluated for tuber shape. Randomly selected tuber images were also shown to industry agronomists and potato growers located in Southern Alberta and their shape scored for suitability for processing (French fry and chipping) markets. Based on measurements taken from greenhouse grown tubers we were able to classify whether mean tuber shape from field grown plants were within ideal shape parameters for processing markets with ~76–86% accuracy. Based on performance of progeny we identified parents which show higher breeding value for tuber shape.

Glaucoma detection is an important task, as this disease can affect the optic nerve, and this could lead to blindness. This can be prevented with early diagnosis, periodic controls, and treatment so that it can be stopped and prevent visual loss. Usually, the detection of glaucoma is carried out through various examinations such as tonometry, gonioscopy, pachymetry, etc. In this work, we carry out this detection by using images obtained through retinal cameras, in which we can observe the state of the optic nerve. This work addresses an accurate diagnostic methodology based on Convolutional Neural Networks (CNNs) to classify these optical images. Most works require a large number of images to train their CNN architectures, and most of them use the whole image to perform the classification. We will use a small dataset containing 366 examples to train the proposed CNN architecture and we will only focus on the analysis of the optic disc by extracting it from the full image, as this is the element that provides the most information about glaucoma. We experiment with different RGB channels and their combinations from the optic disc, and additionally, we extract depth information. We obtain accuracy values of 0.945, by using the GB and the full RGB combination, and 0.934 for the grayscale transformation. Depth information did not help, as it limited the best accuracy value to 0.934.

Simple Summary Animal behaviors are critical for survival, which is expressed over a long period of time. The emergence of computer vision and deep learning technologies creates new possibilities for understanding the biological basis of these behaviors and accurately quantifying behaviors, which contributes to attaining high production efficiency and precise management in precision farming. Here, we demonstrate that a dual-stream 3D convolutional neural network with RGB and optical flow video clips as input can be used to classify behavior states of fish schools. The FlowNet2 based on deep learning, combined with a 3D convolutional neural network, was first applied to identify fish behavior. Additionally, the results indicate that the proposed non-invasive recognition method can quickly, accurately, and automatically identify fish behaviors across hundreds of hours of video. Abstract The rapid and precise recognition of fish behavior is critical in perceiving health and welfare by allowing farmers to make informed management decisions on recirculating aquaculture systems while reducing labor. The conventional recognition methods are to obtain movement information by implanting sensors on the skin or in the body of the fish, which can affect the normal behavior and welfare of the fish. We present a novel nondestructive method with spatiotemporal and motion information based on deep learning for real-time recognition of fish schools’ behavior. In this work, a dual-stream 3D convolutional neural network (DSC3D) was proposed for the recognition of five behavior states of fish schools, including feeding, hypoxia, hypothermia, frightening and normal behavior. This DSC3D combines spatiotemporal features and motion features by using FlowNet2 and 3D convolutional neural networks and shows significant results suitable for industrial applications in automatic monitoring of fish behavior, with an average accuracy rate of 95.79%. The model evaluation results on the test dataset further demonstrated that our proposed method could be used as an effective tool for the intelligent perception of fish health status.

Despite recent successes in hand pose estimation from RGB images or depth maps, inherent challenges remain. RGB-based methods suffer from heavy self-occlusions and depth ambiguity. Depth sensors rely heavily on distance and can only be used indoors, thus there are many limitations to the practical application of depth-based methods. The aforementioned challenges have inspired us to combine the two modalities to offset the shortcomings of the other. In this paper, we propose a novel RGB and depth information fusion network to improve the accuracy of 3D hand pose estimation, which is called CrossFuNet. Specifically, the RGB image and the paired depth map are input into two different subnetworks, respectively. The feature maps are fused in the fusion module in which we propose a completely new approach to combine the information from the two modalities. Then, the common method is used to regress the 3D key-points by heatmaps. We validate our model on two public datasets and the results reveal that our model outperforms the state-of-the-art methods.

Microscale photobioreactors for microalgae growth represent an interesting technology for fast data production and biomass characterization, however, the small scale poses severe monitoring challenges, as traditional methods cannot be used. Non-invasive techniques are therefore needed to quantify biomass concentration and other culture properties, for example, pigment composition. To this purpose, a soft sensing approach based on multivariate image regression is proposed to exploit RGB images and/or PAM-imaging chlorophyll fluorescence. Different PLS (Partial Least Squares) regression models are used to estimate: (a) biomass concentration from the features extracted by RGB indices and/or PAM-imaging chlorophyll fluorescence measurements, and (b) Chlorophyll a content per cell from the features extracted by RGB indices and biomass concentration measurements. Every single model is aimed at characterizing the microalgae culture at different light intensities during batch growth. Results show that the proposed monitoring approach is as accurate as traditional measurement approaches and may represent a promising methodology for fast and inexpensive monitoring of microscale photobioreactors.

Precision agriculture has greatly benefited from advances in machine vision and image processing techniques. The use of feature descriptors and detectors allows to find distinctive keypoints in an image and the use of this approach for agronomical applications has become a widespread field of study. By combining near infrared (NIR) images, acquired with a modified Nikon D80 camera, and visible spectrum (VIS) images, acquired with a Nikon D300s, a proper crop identification could be obtained. Still, the use of different sensors brings an image matching challenge due to the difference between cameras and the possible distortions from each imaging technique. The aim of this paper is to compare the performance of several feature descriptors and detectors by comparing near infrared and visual spectral bands in rice crop images. Therefore, a group of 20 different scenes with different cameras and growth stages in a rice crop were evaluated. Thus, red, green, blue (RGB) and L, a, b (CIE L*a*b*) channels were extracted from VIS images in order to compare the matches obtained between each of them and the corresponding NIR image. The BRISK, SURF, SIFT, ORB, KAZE, and AKAZE methods were implemented, which act as descriptors and detectors. Additionally, a combination was made between the FAST algorithm for the detection of keypoints with the BRIEF, BRISK, and FREAK methods for features description. BF and FLANN matching methods were used. The algorithms were implemented in Python using OpenCV library. The green channel presented the highest number of correct matches in all methods. In turn, the method that presented the highest performance both in time and in the number of correct matches was the combination of the FAST feature detector and the BRISK descriptor.

Point cloud classification is a key technology for point cloud applications and point cloud feature extraction is a key step towards achieving point cloud classification. Although there are many point cloud feature extraction and classification methods, and the acquisition of colored point cloud data has become easier in recent years, most point cloud processing algorithms do not consider the color information associated with the point cloud or do not make full use of the color information. Therefore, we propose a voxel-based local feature descriptor according to the voxel-based local binary pattern (VLBP) and fuses point cloud RGB information and geometric structure features using a random forest classifier to build a color point cloud classification algorithm. The proposed algorithm voxelizes the point cloud; divides the neighborhood of the center point into cubes (i.e., multiple adjacent sub-voxels); compares the gray information of the voxel center and adjacent sub-voxels; performs voxel global thresholding to convert it into a binary code; and uses a local difference sign–magnitude transform (LDSMT) to decompose the local difference of an entire voxel into two complementary components of sign and magnitude. Then, the VLBP feature of each point is extracted. To obtain more structural information about the point cloud, the proposed method extracts the normal vector of each point and the corresponding fast point feature histogram (FPFH) based on the normal vector. Finally, the geometric mechanism features (normal vector and FPFH) and color features (RGB and VLBP features) of the point cloud are fused, and a random forest classifier is used to classify the color laser point cloud. The experimental results show that the proposed algorithm can achieve effective point cloud classification for point cloud data from different indoor and outdoor scenes, and the proposed VLBP features can improve the accuracy of point cloud classification.

[EN] The use of drones in agriculture is becoming a valuable tool for crop monitoring. There are some critical moments for crop success; the establishment is one of those. In this paper, we present an initial approximation of a methodology that uses RGB images gathered from drones to evaluate the establishment success in legumes based on matrixes operations. Our aim is to provide a method that can be implemented in low-cost nodes with relatively low computational capacity. An index (B1/B2) is used for estimating the percentage of green biomass to evaluate the establishment success. In the study, we include three zones with different establishment success (high, regular, and low) and two species (chickpea and lentils). We evaluate data usability after applying aggregation techniques, which reduces the picture's size to improve long-term storage. We test cell sizes from 1 to 10 pixels. This technique is tested with images gathered in production fields with intercropping at 4, 8, and 12 m relative height to find the optimal aggregation for each flying height. Our results indicate that images captured at 4 m with a cell size of 5, at 8 m with a cell size of 3, and 12 m without aggregation can be used to determine the establishment success. Comparing the storage requirements, the combination that minimises the data size while maintaining its usability is the image at 8 m with a cell size of 3. Finally, we show the use of generated information with an artificial neural network to classify the data. The dataset was split into a training dataset and a verification dataset. The classification of the verification dataset offered 83% of the cases as well classified. The proposed tool can be used in the future to compare the establishment success of different legume varieties or species., This research and the contract of S.Y. were funded by project PDR18-XEROCESPED, under the PDR-CM 2014-2020, by the EU (European Agricultural Fund for Rural Development, EAFRD), Spanish Ministry of Agriculture, Fisheries and Food (MAPA) and Comunidad de Madrid regional government through IMIDRA and the contract of L.P. was funded by Conselleria de Educacion, Cultura y Deporte with the Subvenciones para la contratacion de personal investigador en fase postdoctoral, APOSTD/2019/04.

The demand for the sensor-based detection of camouflage objects widely exists in biological research, remote sensing, and military applications. However, the performance of traditional object detection algorithms is limited, as they are incapable of extracting informative parts from low signal-to-noise ratio features. To address this problem, we propose Camouflaged Object Detection with Cascade and Feedback Fusion (CODCEF), a deep learning framework based on an RGB optical sensor that leverages a cascaded structure with Feedback Partial Decoders (FPD) instead of a traditional encoder–decoder structure. Through a selective fusion strategy and feedback loop, FPD reduces the loss of information and the interference of noises in the process of feature interweaving. Furthermore, we introduce Pixel Perception Fusion (PPF) loss, which aims to pay more attention to local pixels that might become the edges of an object. Experimental results on an edge device show that CODCEF achieved competitive results compared with 10 state-of-the-art methods.

Facial recognition is a method of identifying or authenticating the identity of people through their faces. Nowadays, facial recognition systems that use multispectral images achieve better results than those that use only visible spectral band images. In this work, a novel architecture for facial recognition that uses multiple deep convolutional neural networks and multispectral images is proposed. A domain-specific transfer-learning methodology applied to a deep neural network pre-trained in RGB images is shown to generalize well to the multispectral domain. We also propose a skin detector module for forgery detection. Several experiments were planned to assess the performance of our methods. First, we evaluate the performance of the forgery detection module using face masks and coverings of different materials. A second study was carried out with the objective of tuning the parameters of our domain-specific transfer-learning methodology, in particular which layers of the pre-trained network should be retrained to obtain good adaptation to multispectral images. A third study was conducted to evaluate the performance of support vector machines (SVM) and k-nearest neighbor classifiers using the embeddings obtained from the trained neural network. Finally, we compare the proposed method with other state-of-the-art approaches. The experimental results show performance improvements in the Tufts and CASIA NIR-VIS 2.0 multispectral databases, with a rank-1 score of 99.7% and 99.8%, respectively.

Made available in DSpace on 2022-04-29T08:30:21Z (GMT). No. of bitstreams: 0 Previous issue date: 2021-07-01 Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) Urban forests contribute to maintaining livability and increase the resilience of cities in the face of population growth and climate change. Information about the geographical distribution of individual trees is essential for the proper management of these systems. RGB high-resolution aerial images have emerged as a cheap and efficient source of data, although detecting and mapping single trees in an urban environment is a challenging task. Thus, we propose the evaluation of novel methods for single tree crown detection, as most of these methods have not been investigated in remote sensing applications. A total of 21 methods were investigated, including anchor-based (one and two-stage) and anchor-free state-of-the-art deep-learning methods. We used two orthoimages divided into 220 non-overlapping patches of 512 × 512 pixels with a ground sample distance (GSD) of 10 cm. The orthoimages were manually annotated, and 3382 single tree crowns were identified as the ground-truth. Our findings show that the anchor-free detectors achieved the best average performance with an AP50 of 0.686. We observed that the two-stage anchor-based and anchor-free methods showed better performance for this task, emphasizing the FSAF, Double Heads, CARAFE, ATSS, and FoveaBox models. RetinaNet, which is currently commonly applied in remote sensing, did not show satisfactory performance, and Faster R-CNN had lower results than the best methods but with no statistically significant difference. Our findings contribute to a better understanding of the performance of novel deep-learning methods in remote sensing applications and could be used as an indicator of the most suitable methods in such applications. Faculty of Engineering Architecture and Urbanism and Geography Federal University of Mato Grosso do Sul Faculty of Computer Science Federal University of Mato Grosso do Sul Department of Cartography São Paulo State University (UNESP) Computing Science and Mathematics Division Faculty of Natural Sciences University of Stirling Department of Cartography São Paulo State University (UNESP) CNPq: 303559/2019-5 CNPq: 304052/2019-1 CNPq: 433783/2018-4 CAPES: 88881.311850/2018-01

Grading is a vital process during the postharvest of horticultural products as it dramatically affects consumer preference and satisfaction when goods reach the market. Manual grading is time-consuming, uneconomical, and potentially destructive. A non-invasive automated system for export-quality banana tiers was developed, which utilized RGB, hyperspectral imaging, and deep learning techniques. A real dataset of pre-classified banana tiers based on quality and size (Class 1 for export quality bananas, Class 2 for the local market, and Class 3 for defective fruits) was utilized using international standards. The multi-input model achieved an excellent overall accuracy of 98.45% using only a minimal number of samples compared to other methods in the literature. The model was able to incorporate both external and internal properties of the fruit. The size of the banana was used as a feature for grade classification as well as other morphological features using RGB imaging, while reflectance values that offer valuable information and have shown a high correlation with the internal features of fruits were obtained through hyperspectral imaging. This study highlighted the combined strengths of RGB and hyperspectral imaging in grading bananas, and this can serve as a paradigm for grading other horticultural crops. The fast-processing time of the multi-input model developed can be advantageous when it comes to actual farm postharvest processes.

Unmanned aerial vehicles-collected (UAVs) digital red–green–blue (RGB) images provided a cost-effective method for precision agriculture applications regarding yield prediction. This study aims to fully explore the potential of UAV-collected RGB images in yield prediction of winter wheat by comparing it to multi-source observations, including thermal, structure, volumetric metrics, and ground-observed leaf area index (LAI) and chlorophyll content under the same level or across different levels of nitrogen fertilization. Color indices are vegetation indices calculated by the vegetation reflectance at visible bands (i.e., red, green, and blue) derived from RGB images. The results showed that some of the color indices collected at the jointing, flowering, and early maturity stages had high correlation (R2 = 0.76–0.93) with wheat grain yield. They gave the highest prediction power (R2 = 0.92–0.93) under four levels of nitrogen fertilization at the flowering stage. In contrast, the other measurements including canopy temperature, volumetric metrics, and ground-observed chlorophyll content showed lower correlation (R2 = 0.52–0.85) to grain yield. In addition, thermal information as well as volumetric metrics generally had little contribution to the improvement of grain yield prediction when combining them with color indices derived from digital images. Especially, LAI had inferior performance to color indices in grain yield prediction within the same level of nitrogen fertilization at the flowering stage (R2 = 0.00–0.40 and R2 = 0.55–0.68), and color indices provided slightly better prediction of yield than LAI at the flowering stage (R2 = 0.93, RMSE = 32.18 g/m2 and R2 = 0.89, RMSE = 39.82 g/m2) under all levels of nitrogen fertilization. This study highlights the capabilities of color indices in wheat yield prediction across genotypes, which also indicates the potential of precision agriculture application using many other flexible, affordable, and easy-to-handle devices such as mobile phones and near surface digital cameras in the future.

With the rapid development of road traffic, real-time vehicle counting is very important in the construction of intelligent transportation systems (ITSs). Compared with traditional technologies, the video-based method for vehicle counting shows great importance and huge advantages in its low cost, high efficiency, and flexibility. However, many methods find difficulty in balancing the accuracy and complexity of the algorithm. For example, compared with traditional and simple methods, deep learning methods may achieve higher precision, but they also greatly increase the complexity of the algorithm. In addition to that, most of the methods only work under one mode of color, which is a waste of available information. Considering the above, a multi-loop vehicle-counting method under gray mode and RGB mode was proposed in this paper. Under gray and RGB modes, the moving vehicle can be detected more completely, with the help of multiple loops, vehicle counting could better deal with different influencing factors, such as driving behavior, traffic environment, shooting angle, etc. The experimental results show that the proposed method is able to count vehicles with more than 98.5% accuracy while dealing with different road scenes.

Unmanned aerial systems (UASs) have recently become an affordable means to map forests at the species level, but research into the performance of different classification methodologies and sensors is necessary so users can make informed choices that maximize accuracy. This study investigated whether multi-temporal UAS data improved the classified accuracy of 14 species examined the optimal time-window for data collection, and compared the performance of a consumer-grade RGB sensor to that of a multispectral sensor. A time series of UAS data was collected from early spring to mid-summer and a sequence of mono-temporal and multi-temporal classifications were carried out. Kappa comparisons were conducted to ascertain whether the multi-temporal classifications significantly improved accuracy and whether there were significant differences between the RGB and multispectral classifications. The multi-temporal classification approach significantly improved accuracy; however, there was no significant benefit when more than three dates were used. Mid- to late spring imagery produced the highest accuracies, potentially due to high spectral heterogeneity between species and homogeneity within species during this time. The RGB sensor exhibited significantly higher accuracies, probably due to the blue band, which was found to be very important for classification accuracy and lacking in the multispectral sensor employed here.

Tassel counts provide valuable information related to flowering and yield prediction in maize, but are expensive and time-consuming to acquire via traditional manual approaches. High-resolution RGB imagery acquired by unmanned aerial vehicles (UAVs), coupled with advanced machine learning approaches, including deep learning (DL), provides a new capability for monitoring flowering. In this article, three state-of-the-art DL techniques, CenterNet based on point annotation, task-aware spatial disentanglement (TSD), and detecting objects with recursive feature pyramids and switchable atrous convolution (DetectoRS) based on bounding box annotation, are modified to improve their performance for this application and evaluated for tassel detection relative to Tasselnetv2+. The dataset for the experiments is comprised of RGB images of maize tassels from plant breeding experiments, which vary in size, complexity, and overlap. Results show that the point annotations are more accurate and simpler to acquire than the bounding boxes, and bounding box-based approaches are more sensitive to the size of the bounding boxes and background than point-based approaches. Overall, CenterNet has high accuracy in comparison to the other techniques, but DetectoRS can better detect early-stage tassels. The results for these experiments were more robust than Tasselnetv2+, which is sensitive to the number of tassels in the image.

This study used close-range multispectral imagery over cucumber plants inside a commercial greenhouse to detect powdery mildew due to Podosphaera xanthii. It was collected using a MicaSense® RedEdge camera at 1.5 m over the top of the plant. Image registration was performed using Speeded-Up Robust Features (SURF) with an affine geometric transformation. The image background was removed using a binary mask created with the aligned NIR band of each image, and the illumination was corrected using Cheng et al.’s algorithm. Different features were computed, including RGB, image reflectance values, and several vegetation indices. For each feature, a fine Gaussian Support Vector Machines algorithm was trained and validated to classify healthy and infected pixels. The data set to train and validate the SVM was composed of 1000 healthy and 1000 infected pixels, split 70–30% into training and validation datasets, respectively. The overall validation accuracy was 89, 73, 82, 51, and 48%, respectively, for blue, green, red, red-edge, and NIR band image. With the RGB images, we obtained an overall validation accuracy of 89%, while the best vegetation index image was the PMVI-2 image which produced an overall accuracy of 81%. Using the five bands together, overall accuracy dropped from 99% in the training to 57% in the validation dataset. While the results of this work are promising, further research should be considered to increase the number of images to achieve better training and validation datasets.

The rapid and accurate identification of sunflower lodging is important for the assessment of damage to sunflower crops. To develop a fast and accurate method of extraction of information on sunflower lodging, this study improves the inputs to SegNet and U-Net to render them suitable for multi-band image processing. Random forest and two improved deep learning methods are combined with RGB, RGB + NIR, RGB + red-edge, and RGB + NIR + red-edge bands of multi-spectral images captured by a UAV (unmanned aerial vehicle) to construct 12 models to extract information on sunflower lodging. These models are then combined with the method used to ignore edge-related information to predict sunflower lodging. The results of experiments show that the deep learning methods were superior to the random forest method in terms of the obtained lodging information and accuracy. The predictive accuracy of the model constructed by using a combination of SegNet and RGB + NIR had the highest overall accuracy of 88.23%. Adding NIR to RGB improved the accuracy of extraction of the lodging information whereas adding red-edge reduced it. An overlay analysis of the results for the lodging area shows that the extraction error was mainly caused by the failure of the model to recognize lodging in mixed areas and low-coverage areas. The predictive accuracy of information on sunflower lodging when edge-related information was ignored was about 2% higher than that obtained by using the direct splicing method.

The accuracy of photogrammetric reconstruction depends largely on the acquisition conditions and on the quality of input photographs. This paper proposes methods of improving raster images that increase photogrammetric reconstruction accuracy. These methods are based on modifying color image histograms. Special emphasis was placed on the selection of channels of the RGB and CIE L*a*b* color models for further improvement of the reconstruction process. A methodology was proposed for assessing the quality of reconstruction based on premade reference models using positional statistics. The analysis of the influence of image enhancement on reconstruction was carried out for various types of objects. The proposed methods can significantly improve the quality of reconstruction. The superiority of methods based on the luminance channel of the L*a*b* model was demonstrated. Our studies indicated high efficiency of the histogram equalization method (HE), although these results were not highly distinctive for all performed tests.

This work describes a new method for determining K+ concentration, [K+], in blood plasma using a smartphone with a custom-built optical attachment. The method is based on turbidity measurement of blood plasma solutions in the presence of sodium tetraphenylborate, a known potassium precipitating reagent. The images obtained by a smartphone camera are analyzed by a custom image-processing algorithm which enables the transformation of the image data from RGB to HSV color space and calculation of a mean value of the light-intensity component (V). Analysis of images of blood plasma containing different amounts of K+ reveal a correlation between V and [K+]. The accuracy of the method was confirmed by comparing the results with the results obtained using commercial ion-selective electrode device (ISE) and atomic absorption spectroscopy (AAS). The accuracy of the method was within ± 0.18 mM and precision ± 0.27 mM in the [K+] range of 1.5–7.5 mM when using treated blood plasma calibration. Spike tests on a fresh blood plasma show good correlation of the data obtained by the smartphone method with ISE and AAS. The advantage of the method is low cost and integration with a smartphone which offers possibility to measure [K+] on demand and in remote areas where access to hospitals is limited.

Cross-modality person re-identification is the study of images of people matching under different modalities (RGB modality, IR modality). Given one RGB image of a pedestrian collected under visible light in the daytime, cross-modality person re-identification aims to determine whether the same pedestrian appears in infrared images (IR images) collected by infrared cameras at night, and vice versa. Cross-modality person re-identification can solve the task of pedestrian recognition in low light or at night. This paper aims to improve the degree of similarity for the same pedestrian in two modalities by improving the feature expression ability of the network and designing appropriate loss functions. To implement our approach, we introduce a deep neural network structure combining heterogeneous center loss (HC loss) and a non-local mechanism. On the one hand, this can heighten the performance of feature representation of the feature learning module, and, on the other hand, it can improve the similarity of cross-modality within the class. Experimental data show that the network achieves excellent performance on SYSU-MM01 datasets.

Forests are the planet’s main CO2 filtering agent as well as important economical, environmental and social assets. Climate change is exerting an increased stress, resulting in a need for improved research methodologies to study their health, composition or evolution. Traditionally, information about forests has been collected using expensive and work-intensive field inventories, but in recent years unoccupied autonomous vehicles (UAVs) have become very popular as they represent a simple and inexpensive way to gather high resolution data of large forested areas. In addition to this trend, deep learning (DL) has also been gaining much attention in the field of forestry as a way to include the knowledge of forestry experts into automatic software pipelines tackling problems such as tree detection or tree health/species classification. Among the many sensors that UAVs can carry, RGB cameras are fast, cost-effective and allow for straightforward data interpretation. This has resulted in a large increase in the amount of UAV-acquired RGB data available for forest studies. In this review, we focus on studies that use DL and RGB images gathered by UAVs to solve practical forestry research problems. We summarize the existing studies, provide a detailed analysis of their strengths paired with a critical assessment on common methodological problems and include other information, such as available public data and code resources that we believe can be useful for researchers that want to start working in this area. We structure our discussion using three main families of forestry problems: (1) individual Tree Detection, (2) tree Species Classification, and (3) forest Anomaly Detection (forest fires and insect Infestation).