Search

Hand detection and tracking are key components in many computer vision applications, including hand pose estimation and gesture recognition for human–computer interaction systems, virtual reality, and augmented reality. Despite their importance, reliable hand detection in cluttered scenes remains a challenge. This study explores the use of deep learning techniques for fast and robust hand detection and tracking. A novel algorithm is proposed by combining the Kernelized Correlation Filter (KCF) tracker with the Single-Shot Detection (SSD) method. This integration enables the detection and tracking of hands in challenging environments, such as cluttered backgrounds and occlusions. The SSD algorithm helps reinitialize the KCF tracker when it fails or encounters drift issues due to sudden changes in hand gestures or fast movements. Testing in challenging scenes showed that the proposed tracker achieved a tracking rate of over 90% and a speed of 17 frames per second (FPS). Comparison with the KCF tracker on 17 video sequences revealed an average improvement of 13.31% in tracking detection rate (TRDR) and 27.04% in object detection error (OTE). Additional comparison with MediaPipe hand tracker on 10 hand gesture videos taken from the Intelligent Biometric Group Hand Tracking (IBGHT) dataset showed that the proposed method outperformed the MediaPipe hand tracker in terms of overall TRDR and tracking speed. The results demonstrate the promising potential of the proposed method for long-sequence tracking stability, reducing drift issues, and improving tracking performance during occlusions.

Hyperspectral Imaging (HSI) is one of the emerging techniques used in plant phenotyping as it carries abundant information and is non-invasive to plants. However, factors like illumination effect and high-dimensional spectral features need to be solved to attain higher accuracy of plant trait analysis. This research explored and analysed spectral normalisation and dimensionality reduction methods. The focus of this paper is twofold; the first objective was to explore the Standard Normal Variate (SNV), Least Absolute Deviations (L1) and Least Squares (L2) normalisation for spectral correction. The second objective was to explore the feasibility of Principal Component Analysis (PCA) and Analysis of Variance Fisher’s Test (ANOVA F-test) for spectral dimensionality reduction in spectral discriminative modelling. The analysis techniques were validated with HSI data of maise plants for early detection of water deficit stress response. Results showed that SNV performed the best among the three normalisation methods. Besides, ANOVA F-test outperformed PCA for the band selection method as it improved the trait assessment on the water deficit response of maise plants.

The unprecedented outbreak of novel coronavirus 2019 (COVID-19) globally has a huge impact to our daily life in numerous ways. To effectively minimize the spread of the virus, early symptom detection is crucial, especially in closed environment with high human traffic areas which post higher chances of human-to-human transmission. Body temperature measurement has been identified among the vital monitoring parameters. However, current available temperature monitoring mechanism is costly, limited to single individual and limited to locally without integrating to cloud and database. This led to difficulty in effective surveillance for suspicious COVID cases. Hence, the purpose of this paper is to introduce an end-to-end Internet of Things-enabled application for thermal monitoring as an early signal detection and screening method. This work integrates Raspberry Pi, thermal sensor, LCD display, buzzer, and LED light with Raspbian and Restful API for device-to-cloud communication. The system implemented is capable for user identification, body temperature remote monitoring and warning signal for fever symptoms. The result of this real-time system is capable to detect and screen the suspected contagious person in an organization effectively. Future works on integrating face recognition with machine learning and artificial intelligent enhancement.

Defect inspection is a crucial part of industrial manufacturing. However, it relies heavily on human effort on manual visual inspection. Various machine vision techniques have been introduced to replace human labour and to improve inspection quality and efficiency. The limitation of these techniques is that the algorithms need to be engineered again with each different use case. In this work, a Convolutional Neural Network (CNN) is used to classify the defects of the Chemical Mechanical Planarization (CMP) ring. The trained CNN model achieved an accuracy of 91% and the time taken for each inference process is around 1800 msec. To achieve computational efficiency, the CNN model is performed on the embedded device. The previous implementation of embedded CNN deploys OpenCL-based high-level synthesis accelerator on a high-end Field Programmable Gate Array (FPGA) board. In this work, the model inference is accelerated by PipeCNN FPGA implementation on Cyclone-VSE DE1-SoC, a low-end embedded FPGA board. Several configurations of hardware parameters are tested to search for the best setup of the FPGA resources. The hardware implementation has improved approximately seven times faster, as the inference time for each classification has improved from 1800 msec to 250 msec. However, the model implemented using the hardware is observed to produce lower inference accuracy as the accuracy drops from 91% to 81%.

The optical signature of a plant is an essential tool in predicting vegetation water content for quantitative assessment of plant status under drought stress. Plant responses to water stress may involve optically-complex reactions, thus a more sophisticated learning algorithm is needed for accurate prediction. This study proposes Kernel Ridge Regression (KRR) algorithm, a simple yet effective nonlinear learning method to uncover the complex relationship between the response variable and input spectra. A prediction model was developed by calibrating the normalized spectral in Short-Wave-Infrared (SWIR) with the leaf Relative Water Content (RWC) values. The predicted model was applied to a time-series of Hyperspectral images (HSI) of maize plants for early detection of drought stress. RWC was estimated for every plant pixel, and the histogram representation was constructed to characterize the whole plant. Discrimination between healthy and stressed plants was achieved by means of the histogram similarity measure. Further, a OneWay Analysis of Variance (ANOVA) was applied to test the significance of the discrimination between healthy and stressed plants. The proposed method successfully detected drought stress from the fifth day of drought induction, confirming the potential of HSI for drought stress detection studies.

The study of physiological processes resulting from water-limited conditions in crops is essential for the selection of drought-tolerant genotypes and the functional analysis of related genes. A promising, non-invasive technique for plant trait analysis is close-range hyperspectral imaging (HSI), which has great potential for the early detection of plant responses to water deficit stress. In this work, a data analysis method is described that, unlike vegetation indices, the present method applies spectral similarity on selected bands with high discriminative information, while requiring a careful treatment of uninformative illumination effects. The latter issue is solved by a standard normal variate (SNV) normalization that removes linear effects and a supervised clustering approach to remove pixels that exhibit nonlinear multiple scattering effects. On the remaining pixels, the stress-related dynamics is quantified by a spectral analysis procedure that involves a supervised band selection procedure and a spectral similarity measure against well-watered control plants. The proposed method was validated by a large-scale study of water-stress and recovery of maize plants in a high-throughput plant phenotyping platform. The results showed that the analysis method allows for an early detection of drought stress responses and of recovery effects shortly after re-watering.

The complexity of an embedded system is directly proportional to the requirements of industrial applications. Various embedded operating system (OS) approaches had been built to fulfil the requirements. This review aims to systematically address the similarities and differences of the embedded OS solutions and analyse the factors that will influence decision-making when choosing what solution to use in the applications. This paper reviews three standard solutions; super loop, cooperative, and real-time operating system (RTOS). These are commonly used in industrial applications. By grouping the tasks in the foreground and background execution region, the concept and working principle of each of them are reviewed. The unique feature of RTOS in the context of task switching was used to define the deterministic characteristic of meeting the deadlines. The importance and performance of this characteristic is addressed and compared among various solutions in this paper. Subsequently, this paper reviewed the internet of things (IoT) requirements, automotive, medical and consumer electronics industry. The influential factors on choosing the right embedded OS to be used are extracted based on the requirements. They are reviewed in the perspective of memory footprint, regulated standards, cost-effectiveness, time effectiveness, and scalability.

Hyperspectral imaging (HSI) can acquire data in two modes: imaging and spectroscopy, revealing the spatially-resolved spectral properties of materials. Traditional HSI processing in the close-range domain primarily focuses on the spectral information with minimal utilisation of the spatial information present in the data. The present work describes a methodology for utilising the spatial information present in HSI data to improve classification modelling over that achievable with spectral information alone. The methodology has been evaluated using near infrared (NIR) HSI data of sixteen green tea products from seven different countries. The methodology involves selecting and sharpening an image plane to enhance the textural details. The textural information is then extracted from the statistical properties of the grey level co-occurrence matrix (GLCM) of the sharpened image plane using a moving window operation. Finally, the textural properties are combined with the spectral information using one of the three different levels of data fusion, i.e. raw data level, feature level and decision level. Raw data-level fusion involved concatenating the spectral and textural data before performing the classification task. The feature-level fusion involved performing principal component analysis (PCA) on spectral and textural information and combining the PC scores obtained prior to performing classification. Decision-level fusion involved a majority voting scheme to enhance the final classification maps. All the classification tasks were performed using multi-class support vector machine (SVM) models. The results showed that combining the textural and spectral information during modelling resulted in improved classification of the sixteen green tea products compared to models built using spectral or textural information alone.

In this study, a new geometric feature representation for a single finger geometry recognition based on infrared image is presented. The geometric representation is expressed based on the fingertip angles (FTA) measured from the right and the left finger edges. The extracted FTA feature is transformed into the frequency domain to form Fourier descriptor (FD) vector using discrete Fourier transform. The domain transformation is intended to make feature representation robust to the shifting, rotation and scaling variations. FD vectors from both right and the left contours are fused together to form a single row vector and principal component analysis is adopted to enhance the orthogonality between the FD components. The authors’ experimental results demonstrate the feasibility and the effectiveness of the proposed method.

The detection and characterization of physiological processes in crop plants under water-limited conditions is essential for the selection of drought-tolerant genotypes and the functional analysis of related genes. Close-range hyperspectral imaging (HSI) is a promising, non-invasive technique for sensing of plant traits, and has the potential to detect plant responses to water deficit stress at an early stage. The present study describes a data analysis method to realize this potential. Reflectance spectra of plants in close-range imaging are highly influenced by illumination effects. Standard normal variate (SNV) was applied to reduce linear illumination effects, while non-linear effects were filtered by discarding the affected pixels through a clustering procedure. Once the illumination effects were eliminated, the remaining differences in plant spectra were assumed to be related to changes in plant traits. To quantify stress-related spectral dynamics, a spectral analysis procedure was developed based on a supervised band selection and a direct calculation of a spectral similarity measure against a reference. The proposed method was tested on HSI data of maize plants acquired in a high-throughput plant phenotyping platform for assessment of drought stress responses and recovery after re-watering events. Results show that the spectral analysis method successfully detected the drought stress responses at an early stage and consistently revealed the recovery effects shortly after the re-watering period.

The potential of close-range hyperspectral imaging (HSI) as a tool for detecting early drought stress responses in plants grown in a high-throughput plant phenotyping platform (HTPPP) was explored. Reflectance spectra from leaves in close-range imaging are highly influenced by plant geometry and its specific alignment towards the imaging system. This induces high uninformative variability in the recorded signals, whereas the spectral signature informing on plant biological traits remains undisclosed. A linear reflectance model that describes the effect of the distance and orientation of each pixel of a plant with respect to the imaging system was applied. By solving this model for the linear coefficients, the spectra were corrected for the uninformative illumination effects. This approach, however, was constrained by the requirement of a reference spectrum, which was difficult to obtain. As an alternative, the standard normal variate (SNV) normalisation method was applied to reduce this uninformative variability. Once the envisioned illumination effects were eliminated, the remaining differences in plant spectra were assumed to be related to changes in plant traits. To distinguish the stress-related phenomena from regular growth dynamics, a spectral analysis procedure was developed based on clustering, a supervised band selection, and a direct calculation of a spectral similarity measure against a reference. To test the significance of the discrimination between healthy and stressed plants, a statistical test was conducted using a one-way analysis of variance (ANOVA) technique. The proposed analysis techniques was validated with HSI data of maize plants (Zea mays L.) acquired in a HTPPP for early detection of drought stress in maize plant. Results showed that the pre-processing of reflectance spectra with the SNV effectively reduces the variability due to the expected illumination effects. The proposed spectral analysis method on the normalized spectra successfully detected drought stress from the third day of drought induction, confirming the potential of HSI for drought stress detection studies and further supporting its adoption in HTPPP. (C) 2018 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rights reserved.

International audience; The increasing need to develop a rapid understanding of plant functional dynamics has led to the employment of sensor technology for non-destructive assessment of plants. Hyperspectral Imaging (HSI) being an integration of two modalities, imaging and point spectroscopy, is nowadays emerging as a potential tool for rapid, non-destructive and automated close range assessment of plants functional dynamics both in terms of structure and physiology. Firstly, this paper presents an overview of some basic concepts of close range HSI on plants, concerning the plant'light interaction, instrumental setup, and spectral data analysis. Furthermore, the work reviews recent advances of HSI for plant related studies under controlled experimental conditions as well as in natural agricultural settings. Applications are discussed on foliar content estimation, variety identification, growth monitoring, stress and disease-related studies, phenotyping and adoption of HSI in high-throughput phenotyping platforms (HTPPs). Close range HSI is a challenging task and suffers from technical complexities related to external factors (e.g. illumination effects) as well as plant-related factors (e.g. complex plant geometry). The paper finally discusses some of the technical challenges related to the implementation of HSI in the close range assessment of plant traits.

Hand tracking is one of the essential elements in vision-based hand gesture recognition systems. It has a great potential as a tool for better human-computer interaction (HCI) by means of communicating in natural and articulate ways. This has motivated an active research concerning with the interpretation of hand tracking for gesture recognition system. However, due to the nature of hand motion which is flexible, erratic and always varies in its appearance, the tracking of human hand using vision-based remains a complex problem. In this paper, we present an efficient method to overcome such difficulties using integration of Adaptive Kalman Filter (AKF) and Eigenhand method. In the proposed method, the tracking task is first carried out by running the Region of Interest (ROI) based tracker. Here, by fusing the skin and motion cues as the main tracking features, the actual hand position in the current frame is measured. This initial measurement is rather inconsistent due to the working principle of ROI based tracker which greatly depends on how effectively the extracted tracking features are. To reduce this inconsistency, the measurement error is minimized by employing the AKF prediction. After the hand position is effectively estimated by the AKF tracker, a low dimensional eigenspace representation, i.e., the Eigenhand, is employed to further improve the tracking performance. This representation is necessary as the AKF tracker only treats the hand image as a set of moving pixel which consequently disregards the detail appearance of the target. Therefore, the incorporated eigenspace representation provides a compact description of the internal target appearance for better object recognition. This eigenspace adaptively learns the current state to reflect the appearance changes of the target image in each frame. The experimental results demonstrate the effectiveness of the proposed tracking algorithm in indoor and outdoor environments where the target objects undergo large pose changes, lighting variation, fast motion and partial occlusion with average detection rate above 97 % at the speed of 35 frame/second (fps).

A new finger vein recognition algorithm based on Band Limited Phase Only Correlation.Finger width and Centroid Contour Distance for finger geometry recognition.The fusion of vein and geometry for a finger based bimodal biometrics system.A new infrared finger image database is made publicly available on the web. In this paper, a new approach of multimodal finger biometrics based on the fusion of finger vein and finger geometry recognition is presented. In the proposed method, Band Limited Phase Only Correlation (BLPOC) is utilized to measure the similarity of finger vein images. Unlike previous methods, BLPOC is resilient to noise, occlusions and rescaling factors; thus can enhance the performance of finger vein recognition. As for finger geometry recognition, a new type of geometrical features called Width-Centroid Contour Distance (WCCD) is proposed. This WCCD combines the finger width with Centroid Contour Distance (CCD). As compared with the single type of feature, the fusion of W and CCD can improve the accuracy of finger geometry recognition. Finally, we integrate the finger vein and finger geometry recognitions by a score-level fusion method based on the weighted SUM rule. Experimental evaluation using our own database which was collected from 123 volunteers resulted in an efficient recognition performance where the equal error rate (EER) was 1.78% with a total processing time of 24.22ms.

Hand tracking is one of the essential elements in vision based hand gesture recognition sys- tem. The tracked hand image can provide a meaningful gesture for more natural ways of Human Computer Interaction (HCI) systems. In this paper, we present a fast hand tracking method based on the fusion of skin and motion features incorporated covariance matrix. First, hand region is detected using a fusion of skin and motion cues, and a region of interest (ROI) is created around the detected region. During the tracking, skin and motion features are extracted around top, left and right corners of the ROI and hand displacement is measured using ROI based tracker. To increase the robustness, we incorporate a covariance matrix of the ROI window as a region descriptor to represent the target object. In the consecutive frames, we measure the distance descriptor covariance matrix (DDCM) between the target object and the covariance matrix extracted from new ROI position. When DDCM is not satisfying a certain acceptable threshold, ROI position is adjusted by shifting the ROI window around nearest neighbor to obtain a set of candidate regions. We assign a candidate region which has the smallest DDCM as the correct estimated ROI position. The experimental result shows that our approach can track the hand gesture under several real-live scenarios with a detection rate above 95% with the tracking speed at 42fps on average.

With the increase of innovations in vision-based hand gesture interaction system, new techniques and algorithms are being developed by researchers. However, less attention has been paid on the scope of dismantling hand tracking problems. There is also limited publicly available database developed as benchmark data to standardize the research on hand tracking area. For this purpose, we develop a versatile hand gesture tracking database. This database consists of 60 video sequences containing a total of 15,554 RGB color images. The tracking sequences are captured in different situations ranging from an easy indoor scene to extremely high challenging outdoor scenes. Complete with annotated ground truth data, this database is made available on the web for the sake of assisting other researchers in the related fields to test and evaluate their algorithms based on standard benchmark data.

This paper introduces a hand tracking system in unconstrained environment. The system consists of two main stages which are initialization and tracking. In initialization, hand region is first detected by combining motion and skin color pixels. A region of interest (ROI) is then created around the detected hand region. In tracking stage, skin and motion pixels are scanned around top, left and right corners of the ROI to detect the moving hand in consecutive video frames. These pixels are used to actually measure the ROI position and fed into measurement update of Adaptive Kalman Filter (AKF) operation. The process noise covariance and measurement noise covariance of AKF are adjusted adaptively by applying weighting factor based on acceleration threshold value. The experimental result shows the proposed method has the robust ability to track the moving hand under real life scenarios at speed 45fps with average 97.83% tracking rate.