Search

HighlightsWhile freshness is a critical value of food quality, its assessment requires complex methods, which are costly and time-consuming.In this work, it is demonstrated that spectral responses obtained from a portable VIS/NIR imaging spectrometer can be used to predict food freshness using a CNN-based machine learning algorithm.In the food industry, the method can assess real-time food freshness nondestructively and cost-effectively.Abstract. There has been strong demand for the development of accurate but simple methods to assess the freshness of foods. In this study, a system is proposed to determine the freshness of fish by analyzing the spectral response with a portable visible/near-infrared (VIS/NIR) imaging spectrometer and a convolution neural network (CNN) machine learning algorithm. Spectral response data from salmon and tuna, which were incubated at 25°C, were obtained every minute for 30 h and were categorized into three stages (fresh, likely spoiled, or spoiled) based on the time and pH. Using the obtained spectral data, a CNN-based machine learning algorithm was built to evaluate the freshness of the experimental samples. The accuracy of the spectral data in predicting the freshness was ~84% for salmon and ~88% for tuna. Keywords: CNN, Fish, Freshness, pH, Spectral data, VIS/NIR.

Highlights The hidden internal damage of falling impact on tomatoes will reduce the quality of products. Hyperspectral imaging and VIS/NIR spectrum analysis, including wavelength selection and classification model construction, have the possibility as a non-destructive and fast method to predict the effect of drop impact grades on tomato bruising damage. Abstract . Mechanical damage usually causes hidden internal damage to tomatoes (Solanum lycopersicum L.), which can reduce the product quality and can cause economic losses to farmers. The visible and near-infrared (VIS/NIR) spectra of tomato fruits were analyzed by using the wavelength selection algorithm (the combination of ant colony optimization and variable importance in projection), and the influence of impact grades of simulated transport on tomato fruit bruising was evaluated. A VIS/NIR hyperspectral imaging system was developed to capture hyperspectral images of tomatoes from 392–1034 nm spectral region and the part used in actual data analysis was 442-984 nm. Multivariate analysis classifier models (partial least squares discrimination analysis and ANN) were set up based on the original spectral dataset. On the basis of selected wavelength intervals, multivariate analysis classifier models were re-established. The overall classification accuracies of all models in the validation set are good, ranging from 64.29% to 100%. Especially in the two types of classification (bruising and normal), the range of correct accuracy is 89.29% to 100%, which shows very high predicted performance. The prediction performance of the model based on the selected wavelengths decreases slightly, but the prediction time is shortened by more than 70%. The results demonstrated that hyperspectral imaging and VIS/NIR spectrum analysis, including wavelength selection and classification model construction, have the possibility as a non-destructive and fast method to predict the effect of drop impact grades on tomato bruising damage. Keywords: ANN, Ant colony optimization (ACO), Partial least squares discrimination analysis (PLS-DA), Variable importance in projection (VIP), VIS/NIR hyperspectral imaging system, wavelength interval selection.

Conventional deep learning based methods for object detection require a large amount of bounding box annotations for training, which is expensive to obtain such high quality annotated data. Few-shot object detection, which learns to adapt to novel classes with only a few annotated examples, is very challenging since the fine-grained feature of novel object can be easily overlooked with only a few data available. In this work, aiming to fully exploit features of annotated novel object and capture fine-grained features of query object, we propose Dense Relation Distillation with Context-aware Aggregation (DCNet) to tackle the few-shot detection problem. Built on the meta-learning based framework, Dense Relation Distillation module targets at fully exploiting support features, where support features and query feature are densely matched, covering all spatial locations in a feed-forward fashion. The abundant usage of the guidance information endows model the capability to handle common challenges such as appearance changes and occlusions. Moreover, to better capture scale-aware features, Context-aware Aggregation module adaptively harnesses features from different scales for a more comprehensive feature representation. Extensive experiments illustrate that our proposed approach achieves state-of-the-art results on PASCAL VOC and MS COCO datasets. Code will be made available at https://github.com/hzhupku/DCNet., Accepted by CVPR2021

Recent works have made great progress in semantic segmentation by exploiting contextual information in a local or global manner with dilated convolutions, pyramid pooling or self-attention mechanism. However, few works have focused on harvesting boundary information to improve the segmentation performance. In order to enhance the feature similarity within the object and keep discrimination from other objects, we propose a boundary-aware graph convolution (BGC) module to propagate features within the object. The graph reasoning is performed among pixels of the same object apart from the boundary pixels. Based on the proposed BGC module, we further introduce the Boundary-aware Graph Convolution N et-work(BGCN et), which consists of two main components including a basic segmentation network and the BGC module, forming a coarse-to-fine paradigm. Specifically, the BGC module takes the coarse segmentation feature map as node features and boundary prediction to guide graph construction. After graph convolution, the reasoned feature and the input feature are fused together to get the refined feature, producing the refined segmentation result. We conduct extensive experiments on three popular semantic segmentation benchmarks including Cityscapes, PASCAL VOC 2012 and COCO Stuff, and achieve state-of-the-art performance on all three benchmarks.

Attachment is a deep and enduring emotional bond that connects one person to another across time and space. Our early attachment styles are established in childhood through the interaction between infants and caregivers. There are two attachment types, secure and insecure. The attachment experience affects personality development, particularly a sense of security, and research shows that it influences the ability to form stable relationships throughout life. It is also an important aspect of assessing the quality of parenting. Therefore, attachment has been widely studied in psychology research. It’s usually acquired by Ainsworth’s Strange Situation Assessment (SSA) through tedious observation. As far as we know, there is no computational method to predict infant attachment type. We try to use the Still-Face Paradigm (SFP) video and audio as input to predict attachment types through machine learning methods. In the present work, we recruited 64 infant-mother participants, collected videos of SFP when babies are 5–8 months of age and identified their attachment types including secure and insecure by SSA when those infants are almost 2 years old. For the visual part, we extract motion features and apply a RNN network with LSTM units model for classification. For the audio part, speech enhancement is conducted as data pre-processing, pitch frequency, short-time energy and Mel Frequency Cepstral Coefficient feature sequences are extracted. Then SVM is deployed to explore the patterns in it. The experiments show that our method is able to discriminate between the 2 classes of subjects with a good accuracy.

Fresh fruit bruising caused by mechanical impact is the most problematic effect of mechanical postharvest processing of tomatoes. The conventional postharvest process of tomato fruit (Solanum lycopersicum L.) handling was surveyed from harvest to shipping preparation at a farm in Toechon in the Republic of Korea. Acceleration sensors located on the harvest boxes and fruit skin showed that the most severe impact forces (>10 g) occurred while fruit was moved in and out of the harvest box. Next, multivariate analysis was used to evaluate nondestructively the susceptibility of tomatoes to bruising by developing five estimation models incorporating quality factors and the presence of bruising resulting from peak contact forces. Linear regression models, artificial neural network (ANN) regression models, and a logistic regression model were built; the primary dependent variables were the rates of weight loss and firmness loss, and impact-induced bruising. The impact force was controlled by using a pendulum, which produced four levels of impact force on tomato fruit. Increasing the relative humidity (RH) and firmness, and decreasing the temperature and degree of weight loss decreased tomato fruit bruise damage. In addition, the proposed drop–impact measurement technique, with multivariate analysis, can be used to evaluate tomato quality nondestructively.

Energy harvesting, which converts ambient, otherwise wasted, energy sources into usable electricity, is expected to contribute to the formulation of a self-generating power grid. This type of grid can enable sustainable operation of wireless sensor networks as the “Smart City” vision becomes reality. Human walking is a plentiful mechanical energy source wasted during daily activities. This study aims to develop an omnidirectional biomechanical energy harvesting (OBEH) sidewalk block that is able to generate electricity from human walking. Here, a systematic design framework for the OBEH sidewalk block is presented; it consists of three important ingredients, specifically: (1) extraction of a footstep loading profile from human gait analysis; (2) electroelastically coupled finite element modeling to estimate the transient output responses under the footstep loading profile; and (3) reliability-based design optimization of the OBEH sidewalk block. This study considers two kinds of the inherent randomness, including (1) variability in the material properties and geometry; and (2) uncertainty in the position and direction of the footsteps. It can be concluded from the results that the optimum design of the proposed OBEH sidewalk block enables useful power generation while satisfying the target reliability of fatigue failure in the presence of the inherent randomness.

This study was conducted to investigate the weight loss, firmness, external color and vitamin C (VC) content of tomatoes (Lycopersicon esculentum) using non-destructive method to measure identical tomato samples during 15 days storage at low temperature and high humidity. Tomatoes were harvested at the light red stage, sorted, box packed and then stored in thermo-hygrostat (10±1°C, 90±10%RH). The quality changes in weight loss, firmness and external color were measured every 3 day interval. Weight loss was increased by 1.13±0.15%, but it may not be considered to affect quality. Surface color of fruit was changed, especially in lightness and hue angle value. The color values were analyzed by analysis of variance (ANOVA), and the results were significant (p

This paper proposes a system for recognition of infants' gaze behaviors and emotions from videos. In the current work, researchers believed that the information of eye region is crucial for gaze behavior recognition, and emotion recognition mostly depends on the appearance of face. However, because the differentiation of all parts of infant's body has not finished yet, we found that infants always express their intentions and emotions using their whole body, especially moving their heads. Therefore, we incorporate the head pose information as features into the gaze behavior recognition, and we extract the gaze features to improve the recognition of infants' emotions. In addition, we combine several Deep Neural Networks, which can not only capture the details of the images very well, but also make full use of the temporal features. In order to recognize infants' gaze behaviors, we design a feature-extraction convolutional neural network which can obtain the features of infants' gaze direction, then we feed these features with head pose into the next gaze behavior recurrent neural network. Moreover, we combine the features of facial express and gaze behavior to characterize infants' emotions, and expand this system with an emotion recurrent neural network. In the end, we achieve the recognition accuracy 98.31% and 94.71% respectively on our data set.

To predict and assess visual attention, saliencybased visual attention modeling is a popular approach. However, state-of-the-art models are developed for adults, in which children are not considered. Additionally, these models consider neither social cues like face, nor attention learning cues. The face is a vital part of visual attention. Psychological studies reveal that sub-facial areas are different in visual attention. Some models highlight faces in social scenes, but sub-facial areas are not taken into account. Attention learning reveals internal processing of visual attention. By learning how the cognitive system deals with visual stimuli, it is possible to predict visual attention behavior. In this paper, we propose a multilevel visual attention model to predict fixations of children when watching a talking face. Based on traditional saliency maps, the proposed model includes both separate facial areas and attention shift cue. An eye-tracking experiment is conducted to evaluate the model. Results show that the proposed model significantly outperforms conventional models in talking face scenes.

This study proposes a multi-level trajectory analysis method for modeling traffic behavior from an ego-centric view, where on-road vehicle trajectories are collected based on the authors' previous studies of an on-board system consisting of multiple 2D lidar sensors. From an input set of trajectories, a set of hot regions (topics) that trajectory points most frequently present are first discovered using a sticky HDP-HMM; then, the major paths of the trajectories' transitions across different hot regions are extracted by recursively mining frequent subsequences of topics; and finally, paths are modeled using a hierarchical hidden Markov model (HHMM), where the intra-path dynamics is represented using an HMM, in which each state corresponds to a hot region, while the inter-path transition is assumed to be Markovian. The model could be used for behavior prediction, i.e. whenever a vehicle is detected in a scene, predicting which route it will probably follow and how its trajectory will probably develop over time, which is essential to interpreting the potential risks for longer time horizons. Experiments are conducted using a large set of vehicle trajectories collected from motorways in Beijing, and promising results are presented.

Discoloration and dehydration are crucial problems associated with quality of bell pepper (Capsicum annuum L.). Color and weight loss of the bell pepper were measured at different vibration levels, storage environments and storage periods. Vibration tests were carried out at different ‘g’ levels from normal to the worst-case for 180 sec on the initial day. The effect of two different storage environments (10°C, 90 % RH and 18°C, 40 % RH) for four time periods (1, 4, 7 and 10 days) were observed. At 18°C, 40 % RH storage environment, more weight loss and duller color were observed. In the 10°C, 90 % RH storage environment, the group of bell pepper sample having 12 g level vibration had significantly much more weight loss compared to other groups. In this storage environment, most of bell pepper samples maintained vivid color. Storage life and fruit quality of bell pepper were affected by storage temperatures, humidity and duration. The results showed that high g level vibration, high temperature, and low humidity environment during harvesting and storage process of the bell pepper should be avoided.

Near-infrared spectroscopy and multivariate analysis techniques were employed to nondestructively evaluate the rancidity of perilla seed oil by developing prediction models for the acid and peroxide values. The acid, peroxide value, and transmittance spectra of perilla seed oil stored in two different environments for 96 and 144 h were obtained and used to develop prediction models for different storage conditions and time periods. Preprocessing methods were applied to the transmittance spectra of perilla seed oil, and multivariate analysis techniques, such as principal component regression (PCR), partial least squares regression (PLSR), and artificial neural network (ANN) modeling, were employed to develop the models. Titration analysis shows that the free fatty acids in an oil oxidation process were more affected by relative humidity than temperature, whereas peroxides in an oil oxidation process were more significantly affected by temperature than relative humidity for the two different environments in this study. Also, the prediction results of ANN models for both acid and peroxide values were the highest among the developed models. These results suggest that the proposed near-infrared spectroscopy technique with multivariate analysis can be used for the nondestructive evaluation of the rancidity of perilla seed oil, especially the acid and peroxide values.

Effective video-based detection methods are of great importance to intelligent transportation systems (ITS), and here we propose a method to localize and label objects. The method is able to detect pedestrians and bicycle riders in a complex scene. Our method is inspired by the common fate principle, which is a mechanism of visual perception in human beings, and which states tokens moving or functioning in a similar manner tend to be perceived as one unit. Our method embeds the principle in an Implicit Shape Model (ISM). In our method, keypoint-based object parts are firstly detected and then grouped by their motion patterns. Based on the grouping results, when the object parts vote for object centers and labels, each vote belonging to the same object part is assigned a weight according to its consistency with the votes of other object parts in the same motion group. Afterwards, the peaks, which correspond to detection hypotheses on the Hough image formed by summing up all weighted votes, become easier to find. Thus our method performs better in both position and label estimations. Experiments show the effectiveness of our method in terms of detection accuracy.

Successful multi-target tracking requires locating the targets and labeling their identities. For a laser-based tracking system, this mission becomes significantly challenging when the targets are in close proximity or frequently interact with one another. This paper presents a novel on-line learning-based method for laser-based multi-target tracking. When the targets do not interact with one another, multiple independent trackers are employed to train a classifier for each target. When the targets are in close proximity, the learned classifiers are used to assist in tracking. The tracking and learning supplement each other in the proposed method, which is helpful in dealing with difficult problems encountered in laser-based multi-target tracking; moreover, it ensures that the entire process can be completely automatic and available on-line. Various evaluations have demonstrated that this method performs better than previous methods when interactions occur, and it can maintain the correct tracking under various complex tracking situations.

Tracking generic human motion is highly challenging due to its high-dimensional state space and the various motion types involved. In order to deal with these challenges, a fusion formulation which integrates low- and high-dimensional tracking approaches into one framework is proposed. The low-dimensional approach successfully overcomes the high-dimensional problem of tracking the motions with available training data by learning motion models, but it only works with specific motion types. On the other hand, although the high-dimensional approach may recover the motions without learned models by sampling directly in the pose space, it lacks robustness and efficiency. Within the framework, the two parallel approaches, low- and high-dimensional, are fused via a probabilistic approach at each time step. This probabilistic fusion approach ensures that the overall performance of the system is improved by concentrating on the respective advantages of the two approaches and resolving their weak points. The experimental results, after qualitative and quantitative comparisons, demonstrate the effectiveness of the proposed approach in tracking generic human motion.

For reasons of public security, an intelligent surveillance system that can cover a large, crowded public area has become an urgent need. In this article, we propose a novel laser-based system that can simultaneously perform tracking, semantic scene learning, and abnormality detection in a fully online and unsupervised way. Furthermore, these three tasks cooperate with each other in one framework to improve their respective performances. The proposed system has the following key advantages over previous ones: (1) It can cover quite a large area (more than 60×35m), and simultaneously perform robust tracking, semantic scene learning, and abnormality detection in a high-density situation. (2) The overall system can vary with time, incrementally learn the structure of the scene, and perform fully online abnormal activity detection and tracking. This feature makes our system suitable for real-time applications. (3) The surveillance tasks are carried out in a fully unsupervised manner, so that there is no need for manual labeling and the construction of huge training datasets. We successfully apply the proposed system to the JR subway station in Tokyo, and demonstrate that it can cover an area of 60×35m, robustly track more than 150 targets at the same time, and simultaneously perform online semantic scene learning and abnormality detection with no human intervention.

Multitarget tracking becomes significantly more challenging when the targets are in close proximity or frequently interact with each other. This article presents a promising online system to deal with these problems. The novelty of this system is that laser and vision are integrated with tracking and online learning to complement each other in one framework: when the targets do not interact with each other, the laser-based independent trackers are employed and the visual information is extracted simultaneously to train some classifiers online for “possible interacting targets”. When the targets are in close proximity, the classifiers learned online are used alongside visual information to assist in tracking. Therefore, this mode of cooperation not only deals with various tough problems encountered in tracking, but also ensures that the entire process can be completely online and automatic. Experimental results demonstrate that laser and vision fully display their respective advantages in our system, and it is easy for us to obtain a good trade-off between tracking accuracy and the time-cost factor.

In our previous work, we reported a system that monitors an intersection using a network of horizontal laser scanners. This paper focuses on an algorithm for moving-object detection and tracking, given a sequence of distributed laser scan data of an intersection. The goal is to detect each moving object that enters the intersection; estimate state parameters such as size; and track its location, speed, and direction while it passes through the intersection. This work is unique, to the best of the authors' knowledge, in that the data is novel, which provides new possibilities but with great challenges; the algorithm is the first proposal that uses such data in detecting and tracking all moving objects that pass through a large crowded intersection with focus on achieving robustness to partial observations, some of which result from occlusions, and on performing correct data associations in crowded situations. Promising results are demonstrated using experimental data from real intersections, whereby, for 1063 objects moving through an intersection over 20 min, 988 are perfectly tracked from entrance to exit with an excellent tracking ratio of 92.9%. System advantages, limitations, and future work are discussed.

This work is motivated by the following two potential applications: 1) enhancing driving safety and 2) collecting traffic data in a large dynamic urban environment. A laser-scanner-based approach is proposed. The problem is formulated as a simultaneous localization and mapping (SLAM) with object tracking and classification, where the focus is on managing a mixture of data from both dynamic and static objects in a highly dynamic environment. A trajectory-oriented closure is also proposed using the sporadically available global positioning system (GPS) measurements in urban areas to assist for global accuracy, particularly when the vehicle makes a noncyclical measurement in a large outdoor environment. Experiments are conducted using the data that were collected along a course near 4.5 km in a highly dynamic environment. Possibilities of the approaches toward the two potential applications are demonstrated, and avenues for future works are discussed.

In this research, a novel system for monitoring an intersection using a network of single-row laser range scanners (subsequently abbreviated as "laser scanner") and video cameras is proposed. Laser scanners are set on the road side to profile an intersection horizontally from different viewpoints. The contour points of moving objects are captured at a certain horizontal plane with a high scanning rate (e.g., 37 Hz). A laser-based processing algorithm is developed, thus the moving objects entered the intersection are detected and tracked to estimate their state parameters, such as: location, speed, and direction at each time instance. In addition, laser data and processing results are forwarded to an associated video camera, so that a visualization as well as fusion-based processing can be achieved. An experiment in central Beijing is presented, demonstrating that a large quantity of physical dimension and detailed traffic data can be obtained through such a system.

Inspite extensive research on visual tracking of multiple people in computer vision area, the robustness and usability of visual trackers are still discouraging. Recently, a few laser-based detection and tracking methods have been developed in robotics area. However, poor features provided by laser data make the tracker fail in many situations. In this paper, we present a novel method that aims at reliably detecting and tracking multiple people in an open area. Multiple laser scanners and one camera are used as input sensors. In detection stage, laser-based detection algorithm captures newly appeared people and initializes the mean-shift-based visual tracker. In tracking stage, laser-based feet trajectory tracking result and visual body region tracking result are combined with a decision-level Bayesian fusion method. The experimental results demonstrate reliable and real-time performance of the method.

Laser-based people tracking systems have been developed for mobile robotic, and intelligent surveillance areas. Existing systems rely on laser point clustering method to extract object locations. However, for dense crowd tracking, laser points of different objects are often interlaced and undistinguishable due to measurement noise and they can not provide reliable features. It causes current systems quite fragile and unreliable. This paper presents a novel and robust laser-based dense crowd tracking method. Firstly, we introduce a stable feature extraction method based on accumulated distribution of successive laser frames. With this method, the noise that generates split and merged measurements is smoothed away and the pattern of rhythmic swing legs is utilized to extract each leg of persons. And then, a region coherency property is introduced to construct an efficient measurement likelihood model. The final tracker is based on the combination of independent Kalman filter and Rao-Blackwellized Monte Carlo data association filter (RBMC-DAF). In real experiments, we obtain raw data from multiple registered laser scanners, which measure two legs for each people on the height of 16cm from horizontal ground. Evaluation with real data shows that the proposed method is robust and effective. It achieves a significant improvement compared with existing laser-based trackers. In addition, the proposed method is much faster than previous works, and can overcome tracking errors resulted from mixed data of two closely situated persons.

Visual hand gesture based interfaces have been widely used for navigation of virtual environments and control of a robot arm in robotic systems. In fact, hand movement has abundant powers of expression with complex finger joint motions. In this paper, the full degree-of-freedom hand motion is tracked in a vision-based mode towards natural and intuitive control of a dexterous robotic hand. The main obstacle to handle human hand posture tracking is the high dimensionality. A novel three-dimensional model-based approach is proposed. It utilizes a genetic algorithm-based global optimization procedure after each particle-filtering step, which deals with the tracking in high-dimensional and multi-modal state space reasonably well. Experimental results show that present approach improves performance of hand posture tracking significantly and provides a feasible solution for guiding a dexterous hand.

International audience; Head pose is an important indicator of a person's visual focus of attention (VFoA). A traditional way to recognize VFoA is to consider accurate head pose or gaze estimations. However, these estimations usually degrade drastically in middle or low resolution video data. In this paper, a joint estimation of head pose and VFoA is proposed to address this issue; both head pose and VFoA are iteratively refined until convergence. This approach is evaluated in a specific scenario involving children around a table playing together with toys. Datasets are acquired and annotated by psychologists in Peking university. The experimental results demonstrate the usefulness of the join estimation process to recognize visual focus of attention in middle resolution video sequences.

International audience; Precise localization is an essential issue for autonomous driving applications, where GPS-based systems are challenged to meet requirements such as lane-level accuracy. This paper introduces a new visual-based localization approach in dynamic traffic environments, focusing on and exploiting properties of structured roads like straight roads or intersections. Such environments show several line segments on lane markings, curbs, poles, building edges, etc., which demonstrate the road's longitude, latitude and vertical directions. Based on this observation, we define a Road Structural Feature (RSF) as sets of segments along three perpendicular axes together with feature points. At each video frame, the proper road structure (or multiple road structures in case of an intersection) is predicted based on the geometric information given by a 2D map. The RSF is then detected from line segments and points extracted from the image, and used to estimate the pose of the vehicle. Experiments are conducted using video streams collected on major roads in downtown Beijing, which are structured and with intense dynamic traffic. GPS/IMU data have been collected and synchronized with the video streams as a reference in validation. The results show good performance compared with that of a more traditional visual odometry method. Future work will be addressed on using visual approach to improve GPS localization accuracy.

Many robotic and mobile mapping systems have been developed using multiple 2D LIDARs (briefly multi- LIDAR system) to sense environment. In such systems, extrinsic calibration of all LIDARs is essential for making collaborative use of the data from different sensors. This research aims at developing a calibration method for multi-LIDAR systems at the general scene, such as an outdoor place or an underground parking-lot, without modification to environment by putting calibration targets. In this paper, the calibration method is proposed by aligning the 3D data of different LIDARs. They are concerned at two-levels: 1) reference calibration, i.e. finding the transformation from a reference LIDAR to the platform frame; 2) multi-LIDAR calibration, i.e. finding the LIDARs' relative geometries by referring to the reference one. The method is examined in calibrating the multiple 2D LIDARs on an intelligent vehicle platform POSS-V, where the data collected through a driving in an underground parking-lot are registered to find sensors' geometry. Calibration accuracy is examined by comparing with a CAD model of the scene, which was measured by using a total station.

Labeling children's play behavior is an important process in children's social behavior analysis which is traditionally done by experienced coders. With the growing volume of data, automatic methods for labeling are increasingly required. This paper presents a novel method to label children's social behavior automatically in peer-play scenarios based on visual attention and mutual interaction computation. In this method, the discrete distribution of children's visual attention is computed based on face pose estimation. Then, the mutual interaction among children is calculated by "Attention Processes", which are continuous periods of time during which a certain child pays attention to the same target. After that, "Solitary Feature" and "Group Feature" are extracted by the mutual interaction of children and children's play behaviors will be classified into 3 types ("Solitary Play", "Parallel Play" and "Group Play") by these two kinds of features. Finally, this method is evaluated by a dataset of children's peer-play scenarios and the results show this method has a good performance in our dataset.

Gaze is a powerful cue for children's social behavior analysis. In this paper, a novel method is proposed to estimate children's gaze orientation in the experimental data of developmental psychology based on head pose estimation. In consideration of the possible errors of head pose estimation results, temporal information and potential targets are both introduced to improve the results of gaze estimation. At last, this method is evaluated by a dataset of children's peer-play scenarios and the results show that this method has a good performance. According to the experimental valuation and analysis, in a certain peer-play scenario, potential targets are powerful spatial cues for children's gaze estimation and temporal information also provides some cues to improve the estimation results.

It has become a well-known technology that 3D measurement of a large environment could be achieved by using a number of 2D LIDARs on a mobile platform. In such a system, calibration is essential for making collaborative use of different LIDAR data, while existing methods usually require modifications to the environments, such as putting calibration targets, or rely on special facilities, which is labor intensive and put many restrictions to potential applications. This research aims at developing a calibration method for multiple 2D LIDAR sensing systems, which could be conducted in a general outdoor environment using the features of a nature scene. Special focus is cast on solving the noisy sensing in a complex environment and the occlusions caused by largely different sensor viewpoints. A multi-type geometric feature based calibration algorithm is proposed, which extracts the features such as points, lines, planes and quadrics from the 3D points of each LIDAR sensing. Transformation parameters from each sensor to the frame on a moving platform is estimated by matching the multi-type features. Experiments are conducted using the data sets of an intelligent vehicle platform (POSS-V) through a driving in the campus of Peking University. Results of calibrating two LIDAR sensors with largely different viewpoints are presented, and the accuracy and robustness concerning noisy feature extractions are examined intensively.

Intention prediction is an active research topic, however, few work make use of the head pose. This research investigates the usefulness of head pose and pose change features into a Discrete Dynamic Bayesian Network to predict pedestrian's intention. In particular, we first focus on scenarios within a shopping center (using the standard CAVIAR database), to predict whether a pedestrian will enter a shop or not. We assume that different head pose patterns reflect different intentions. Thus, a hierarchical clustering method is used to process the histogram of head pose and the change of head pose. Then the clustering results are used as features feeding into a dynamic bayesian network. Experiments have been conducted to demonstrate the importance of using head pose in intention prediction and it shows a fairly good result which use only the information derived from pedestrian themselves regardless of the complex environment. The proposed approach is simple to use and makes it easy to integrate other features.

In order to study the driving behaviors, such as lane change and overtaking, which concerns the relationship between ego and all-surrounding vehicles, it is of great demand in developing an automated system to collect the synchronized motion trajectories that characterize the full course of driving maneuvers in real-world traffic scene. This research proposes a measurement and data processing system, where multiple 2D-Lidars are mounted on a vehicle platform to generate an omni-directional horizontal coverage to the ego-vehicle’s surrounding; focusing the driving scenario on motorway, two processing approaches in online and offline procedures are studied for vehicle trajectory extraction by fusing the multi-Lidar data that are acquired during on-road driving. A case study is conducted using a data set collected during 10 min’ driving and lasted for 4.1 km long. The performance of trajectory extraction in online and offline procedures is comparatively examined. In addition, a reference vehicle participated in data collection too. The trajectory is analyzed to study its potential in characterizing the situations during vehicle maneuvers.