Search

Deep learning has been proved effective in multiple object tracking, which confronts the difficulties of frequent occlusions, confusing appearance, in‐and‐out objects, and lack of enough labelled data. Recently, deep learning based multi‐object tracking methods make a rapid progress from representation learning to network modelling due to the development of deep learning theory and benchmark setup. In this study, the authors summarise and analyse deep learning based multi‐object tracking methods which are top‐ranked in the public benchmark test. First, they investigate functionality of deep networks in these methods, and classify the methods into three categories as description enhancement using deep features, deep network embedding, and end‐to‐end deep network construction. Second, they review deep network structures in these methods, and detail the usage and training of these networks for multi‐object tracking problem. Through experimental comparison of tracking results in the benchmarks in total and by group, they finally show the effectiveness of deep networks for tracking employed in different manners, and compare the advantages of these networks and their robustness under different tracking conditions. Moreover, they analyse the limitations of current methods, and draw some useful conclusions to facilitate the exploration of new directions for multi‐object tracking.

The goal of multiple object tracking (MOT) is to estimate the locations of objects and maintain their identities consistently to yield their individual trajectories. MOT has been developed enormously, but it is still a challenging work due to similar appearances of different objects and occlusion by other objects or background in a complex scene. In this study, the authors propose confidence score‐based appearance model learning and hierarchical data association for MOT. First, the confidence score is used to divide associated tracklet‐detection in the first stage data association into confident and unconfident results, and in the second stage, data association is applied to unconfident tracklet‐detection to improve the performance. Furthermore, it can be employed to enhance the robustness of the appearance model and due to the fast confidence score calculation, it can balance the accuracy and processing time. The experimental results with challenging public datasets show distinct performance improvement over other state‐of‐the‐art methods and demonstrate the effect of the authors’ method for online MOT.

The recent advancement in autonomous robotics is directed toward designing a reliable system that can detect and track multiple objects in the surrounding environment for navigation and guidance purposes. This paper aims to survey the recent development in this area and present the latest trends that tackle the challenges of multiple object tracking, such as heavy occlusion, dynamic background, and illumination changes. Our research includes Multiple Object Tracking (MOT) methods incorporating the multiple inputs that can be perceived from sensors such as cameras and Light Detection and Ranging (LIDAR). In addition, a summary of the tracking techniques, such as data association and occlusion handling, is detailed to define the general framework that the literature employs. We also provide an overview of the metrics and the most common benchmark datasets, including Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI), MOTChallenges, and University at Albany DEtection and TRACking (UA-DETRAC), that are used to train and evaluate the performance of MOT. At the end of this paper, we discuss the results gathered from the articles that introduced the methods. Based on our analysis, deep learning has introduced significant value to the MOT techniques in recent research, resulting in high accuracy while maintaining real-time processing.

Studies examining individual differences in interruption recovery have shown that higher working memory capacity (WMC) attenuated the negative impact of interruption length on resumption, at least in static contexts. In continuously evolving (or dynamic) situations, however, working memory may not be as central to the effective resumption of a task, especially in the case of long interruptions. One of the main theories of task interruption suggests that dynamic task resumption could depend on a reconstruction of the primary task context, that is, a visual examination of the post-interruption environment. To better define the role of working memory and reconstruction processes in interruption recovery, the current study examined the association between (1) dynamic task resumption following interruptions of various lengths and (2) two cognitive abilities chosen to operationalize the processes under study, namely, WMC and visual search capacity. Participants performed a multiple object tracking task which could be uninterrupted or interrupted for 5, 15, or 30 s while the hidden stimuli continued their trajectory. They also completed tasks measuring the two cognitive abilities of interest. The results revealed that WMC contributed to post-interruption accuracy regardless of interruption duration. On the contrary, visual search capacity was related to faster resumption in the 15-s and 30-s interruption conditions only. Those results show that working memory plays a preponderant role in resumption not only in static, but also in dynamic contexts. However, our study suggests that this mechanism must share the limelight with reconstruction following lengthy interruptions in dynamic settings.

Objective: The present research applied a well-established measure of cognitive workload in driving literature to an in-lab paradigm. We then extended this by comparing the in-lab version of the task to an online version. Background: The accurate and objective measurement of cognitive workload is important in many aspects of psychological research. The detection response task (DRT) is a well-validated method for measuring cognitive workload that has been used extensively in applied tasks, for example, to investigate the effects of phone usage or passenger conversation on driving, but has been used sparingly outside of this field. Method: The study investigated whether the DRT could be used to measure cognitive workload in tasks more commonly used in experimental cognitive psychology and whether this application could be extended to online environments. We had participants perform a multiple object tracking (MOT) task while simultaneously performing a DRT. We manipulated the cognitive load of the MOT task by changing the number of dots to be tracked. Results: Measurements from the DRT were sensitive to changes in the cognitive load, establishing the efficacy of the DRT for experimental cognitive tasks in lab-based situations. This sensitivity continued when applied to an online environment (our code for the online DRT implementation is freely available at https://osf.io/dc39s/), though to a reduced extent compared to the in-lab situation. Conclusion: The MOT task provides an effective manipulation of cognitive workload. The DRT is sensitive to changes in workload across a range of settings and is suitable to use outside of driving scenarios, as well as via online delivery. Application: Methodology shows how the DRT could be used to measure sources of cognitive workload in a range of human factors contexts. [ABSTRACT FROM AUTHOR]

In this paper, we propose a generic boosting framework for multiple object tracking (MOT). Unlike other works tracking objects from zero, our framework uses their results (tracklets) and makes further optimizations. The motivation of us derives from the observation that most modern MOT trackers have been acceptable performance and can yield relatively reliable tracklets; accordingly, we straight focus on the tracklet-level re-identification, which is the most challenging issue in this case. To achieve that goal, we simultaneously utilize the techniques of single object tracking, tracking fragment (tracklets) and re-identification mechanism through casting them into a multi-label energy optimization and then innovatively solving it using the α - expansion with label costs algorithm. All these techniques inspire recent MOT a lot to mitigate the occlusion problem, but to our knowledge, by far few works explore to reasonably combine them all like us. Furthermore, we introduce a spatial attention to improve the appearance model and a hierarchical clustering as post-process to progressively improve the tracking consistency. Finally, testing results on the most used benchmarks demonstrate the significant effectiveness and generality of our framework, and the importance of each contribution is also verified through ablative studies. [ABSTRACT FROM AUTHOR]

Background: Collegiate athletes with prior sports-related concussion (SRC) are at increased risk for lower extremity (LE) injuries; however, the biomechanical and cognitive mechanisms underlying the SRC-LE injury relationship are not well understood. Purpose: To examine the association between cognitive performance and LE land-and-cut biomechanics among collegiate athletes with and without a history of SRC and to determine the association among multiple cognitive testing batteries in the same athlete cohort. Study Design: Controlled laboratory study. Methods: A cohort of 20 collegiate athletes with prior SRC (9 men, 11 women; mean ± standard deviation [SD] age, 20.5 ± 1.3 years; mean ± SD time since last SRC, 461 ± 263 days) and 20 matched controls (9 men, 11 women; mean ± SD age, 19.8 ± 1.3 years) completed land-and-cut tasks using the dominant and nondominant limbs. LE biomechanical variables and a functional visuomotor reaction time (FVMRT) were collected during each trial. Athletes also completed the Immediate Post-Concussion Assessment and Cognitive Test (ImPACT) and Senaptec Sensory Station assessments. Results: In the SRC cohort, Pearson correlation coefficients indicated slower FVMRT was moderately correlated with decreased dominant limb (r = –0.512) and nondominant limb (r = –0.500) knee flexion, while increased dominant limb knee abduction moment was moderately correlated with decreased ImPACT Visual Memory score (r = –0.539) and slower ImPACT Reaction Time (r = 0.515). Most computerized cognitive measures were not associated with FVMRT in either cohort (P >.05). Conclusion: Decreased reaction time and working memory performance were moderately correlated with decreased sagittal plane knee motion and increased frontal plane knee loading in collegiate athletes with a history of SRC. The present findings suggest a potential unique relationship between cognitive performance and LE neuromuscular control in athletes with a history of SRC injury. Last, we determined that computerized measures of cognitive performance often utilized for SRC management are dissimilar to sport-specific cognitive processes. Clinical Relevance: Understanding the relationship between cognitive performance and LE biomechanics in athletes with prior SRC may inform future clinical management strategies. Future research should prospectively assess cognitive and biomechanical measures, along with LE injury incidence, to identify mechanisms underlying the SRC-LE injury relationship. [ABSTRACT FROM AUTHOR]

Most automatic license-plate recognition (ALPR) systems use still images and ignore the temporal information in videos. Videos provide rich temporal and motion information that should be considered during training and testing. This study focuses on creating an ALPR system that uses videos. The proposed system is comprised of detection, tracking, and recognition modules. The system achieved accuracies of 81.473% and 84.237% for license-plate detection and classification, respectively. [ABSTRACT FROM AUTHOR]

Background: Body movement-controlled video games involving physical motion and visual attention may have the potential to train both abilities simultaneously. Our purpose was to determine the associations between performance in these games and visual attention, balance and mobility in a group of older adults. The long-term goal is to identify the optimal type of interactive games with regards to training potential.Methods: Fifty healthy adults aged 65+ years participated in this cross-sectional study. Visual attention was measured with static and dynamic versions of a useful field of view (UFV) and a multiple object tracking (MOT) test. Balance was measured with a force plate in bi-pedal quiet stance test (QST) and one-legged stance (OLST). Gait variability and walking speed were assessed with the Five Meter Walk Test (5MWT). Four Microsoft™ Xbox® 360 Kinect™ interactive video games were chosen based on the apparent levels of visual attention demand.Results: Visual attention (UFV and MOT) was significantly associated with performance in Xbox® Kinect™ games that appeared to have a high visual attention demand (p < 0.05), while there was minimal or no significant association with games with apparent low visual attention demand. Balance and mobility show correlations with visual attention, and with Xbox games.Conclusion: The results suggest that there are relationships between visual attention, balance, mobility and Xbox® Kinect™ game performance. Since different Xbox® games were associated with different balance, mobility and visual attention scores, a variety of such games, rather than a single game, may be most effective for training for falls prevention. [ABSTRACT FROM AUTHOR]

Studies examining individual differences in interruption recovery have shown that higher working memory capacity (WMC) attenuated the negative impact of interruption length on resumption, at least in static contexts. In continuously evolving (or dynamic) situations, however, working memory may not be as central to the effective resumption of a task, especially in the case of long interruptions. One of the main theories of task interruption suggests that dynamic task resumption could depend on a reconstruction of the primary task context, that is, a visual examination of the post-interruption environment. To better define the role of working memory and reconstruction processes in interruption recovery, the current study examined the association between (1) dynamic task resumption following interruptions of various lengths and (2) two cognitive abilities chosen to operationalize the processes under study, namely, WMC and visual search capacity. Participants performed a multiple object tracking task which could be uninterrupted or interrupted for 5, 15, or 30 s while the hidden stimuli continued their trajectory. They also completed tasks measuring the two cognitive abilities of interest. The results revealed that WMC contributed to post-interruption accuracy regardless of interruption duration. On the contrary, visual search capacity was related to faster resumption in the 15-s and 30-s interruption conditions only. Those results show that working memory plays a preponderant role in resumption not only in static, but also in dynamic contexts. However, our study suggests that this mechanism must share the limelight with reconstruction following lengthy interruptions in dynamic settings. [ABSTRACT FROM AUTHOR]

Flexible resource models suggest that attentional deployment in Multiple Object Tracking either shows in target enhancement or distractor inhibition or both. In order to gauge the influence of these processes we manipulated distinctiveness of targets and distractors along a single dimension by finely grading gray values of objects. We established that a quantitative increase in distinctiveness results in a quantitative increase in tracking performance thus demonstrating a new finding. Further increases of distinctiveness beyond a certain degree produced no further improvements in tracking. This effect, however, was moderated by levels of task difficulty thus providing evidence for the relativity of this effect. With higher task difficulty higher degrees of distinctiveness still resulted in a significant performance gain (Experiment 1). A follow up experiment generalized this finding. By manipulating speed of objects, a situation with a yet higher task difficulty could be established showing that an increase of an already striking distinctiveness can produce an even further performance gain - provided higher task difficulty (Experiment 2). Finally we tested the hypothesis that target enhancement and distractor inhibition are influenced by feature distinctiveness. Although we found possible indications for distractor inhibition, we could not demonstrate an effect of feature distinctiveness on distractor inhibition. Target enhancement, however, was substantially influenced by variation of feature distinctiveness (Experiment 3). [ABSTRACT FROM AUTHOR]

Humans often perform visual tasks together, and when doing so, they tend to devise division of labor strategies to share the load. Implementing such strategies, however, is effortful as co-actors need to coordinate their actions. We tested if pupil size – a physiological correlate of mental effort – can detect such a coordination effort in a multiple object tracking task (MOT). Participants performed the MOT task jointly with a computer partner and either devised a division of labor strategy (main experiment) or the labor division was already pre-determined (control experiment). We observed that pupil sizes increase relative to performing the MOT task alone in the main experiment while this is not the case in the control experiment. These findings suggest that pupil size can detect a rise in coordination effort, extending the view that pupil size indexes mental effort across a wide range of cognitively demanding tasks.

Multiple object tracking (MOT) is a fundamental issue and has attracted considerable attention in the autonomous driving community. This paper presents a novel MOT framework for autonomous driving. The framework consists of two stages of object representation and data association. In the stage of object representation, we employ appearance, motion, and position features to characterize objects. We design a spatial pyramidal pooling hash network (SPPHNet) to generate the appearance features. Multiple-level representative features in the SPPHNet are mapped into a similarity-preserving binary space, called hash features. The hash features retain the visual discriminability of high-dimensional features and are beneficial for computational efficiency. For data association, a two-tier data association scheme is designed to address the occlusion issue, consisting of an affinity cost model and a hash-based image retrieval model. The affinity cost model accommodates the hash features, disparity, and optical flow as the first tier of data association. The hash-based image retrieval model exploits the hash features and adopts image retrieval technology to handle reappearing objects as the second tier of data association. Experiments on the KITTI public benchmark dataset and our campus scenario sequences show that our method has superior tracking performance to the state-of-the-art vision-based MOT methods.

Tracking construction machines in videos is a fundamental step in the automated surveillance of construction productivity, safety, and project progress. However, existing vision-based tracking methods are not able to achieve high tracking precision, robustness, and practical processing speed simultaneously. Occlusions and illumination variations on construction sites also prevent vision-based tracking methods from obtaining optimal tracking performance. To address these challenges, this research proposes a vision-based method, called construction machine tracker (CMT), to track multiple construction machines in videos. CMT consists of three main modules: detection, association, and assignment. The detection module detects construction machines using the deep learning algorithm YOLOv3 in each frame. Then the association module relates the detection results of two consecutive frames, and the assignment module produces the tracking results. In testing, CMT achieved 93.2% in multiple object tracking accuracy (MOTA) and 86.5% in multiple object tracking precision (MOTP) with a processing speed of 20.8 frames per second when tested on four construction videos. The proposed CMT was integrated into a framework of analyzing excavator productivity in earthmoving cycles and achieved 96.9% accuracy. [ABSTRACT FROM AUTHOR]

In the near future humans will increasingly be required to cooperate and share task load with artificial agents in joint tasks as they will be able to greatly assist humans in various types of tasks and contexts. In the present study, we investigated humans' willingness to share task load with a computer partner in a joint visuospatial task. The partner was described as either behaving in a human-like or machine-like way and followed a pre-defined behaviour that was either human-like or non-human-like. We found that participants successfully shared task load when the partner behaved in a human-like way. Critically, the successful collaboration was sustained throughout the experiment only when the partner was also described as behaving in a human-like way beforehand. These findings suggest that not only the behaviour of a computer partner but also the prior description of the partner is a critical factor influencing humans' willingness to share task load.

In dense pedestrian tracking, frequent object occlusions and close distances between objects cause difficulty when accurately estimating object trajectories. In this study, a conditional random field tracking model is established by using a visual long short term memory network in the three dimensional space and the motion estimations jointly performed on object trajectory segments. Object visual field information is added to the long short term memory network to improve the accuracy of the motion related object pair selection and motion estimation. To address the uncertainty of the length and interval of trajectory segments, a multimode long short term memory network is proposed for the object motion estimation. The tracking performance is evaluated using the PETS2009 dataset. The experimental results show that the proposed method achieves better performance than the tracking methods based on independent motion estimation.

Visual attention and visual working memory tasks recruit a common network of lateral frontal cortical (LFC) and posterior parietal cortical (PPC) regions. Here, we examine finer-scale organization of this frontoparietal network. Three LFC regions recruited by visual cognition tasks, superior precentral sulcus (sPCS), inferior precentral sulcus (iPCS), and mid inferior frontal sulcus (midIFS) exhibit differential patterns of resting-state functional connectivity to PPC. A broad dorsomedial to ventrolateral gradient is observed, with sPCS connectivity dominating in the dorsomedial PPC band, iPCS dominating in the middle band, and midIFS dominating in the ventrolateral band. These connectivity-defined subregions of PPC capture differential task activation between a pair of visual attention and working memory tasks. The relative functional connectivity of sPCS and iPCS also varies along the rostral-caudal axis of the retinotopic regions of PPC. iPCS connectivity is relatively stronger near the IPS0/IPS1 and IPS2/IPS3 borders, especially on the lateral portions of these borders, which each preferentially encode central visual field representations. In contrast, sPCS connectivity is relatively stronger elsewhere in retinotopic IPS regions which preferentially encode peripheral visual field representations. These findings reveal fine-scale gradients in functional connectivity within the frontoparietal visual network that capture a high-degree of specificity in PPC functional organization.

Background Dynamic visual attention is important in basketball because it may affect the performance of players and thus the match outcome. The goals of this study were to investigate the difference in dynamic visual attention characteristics between highly skilled basketball players and nonathletic college students and to explore the relationship between visual attention and game-related performance among the basketball players. Methods In total, 24 highly skilled basketball players and 24 nonathletic college students participated in a multiple object tracking task. The task was conducted so that either the number of targets that were visually tracked or the speed at which a given number of tracked targets moved was altered to examine the difference in dynamic visual attention characteristics between the basketball players and nonathletic college students. The relationship between visual tracking speed (VTS) and game-related statistics, including assists, steals, mistakes, fouls and points scored recorded for every match during the season, was assessed among the basketball players by using Pearson correlations. Results A significant main effect of target tracking load was observed (P < 0.001), with visual tracking performance significantly decreased as target number increased. In addition, the speed at which the targets moved had a significant effect on visual tracking performance (P < 0.001), with tracking performance significantly decreased as target speed increased. However, no significant difference was observed in the abilities of basketball players and nonathletic college students to simultaneously track up to six targets. By contrast, a significant interaction between group and target speed was found (P < 0.001), with the visual tracking accuracy of basketball players significantly greater than that of college students at the higher target speeds examined (P < 0.001). Among basketball players, there were positive, large, and statistically significant correlations in the accuracy in VTS trials and the number of assists (P < 0.001) and between the accuracy in VTS trials and the number of steals (P < 0.001). Conclusion The advantage of skilled basketball players to handle dynamic visual information in a multiple object tracking task was not attributable to the target number but to the target speed. Those athletes with greater dynamic visual attention were more likely to successfully assist or to steal the ball, enhancing performance of the athlete as well as contributing to a more successful team match. These findings may inform basketball training programs to improve player and team performances during matches.

In real-world scenarios, objects’ surface features sometimes change as they move, impairing the continuity of objects. However, it is still unknown how our visual system adapts to this dynamic change. Hence, the present study investigated the role of feature changes in attentive tracking through a modified multiple object tracking (MOT) task. The feature heterogeneity and feature stability were manipulated in two experiments. The results from Experiment 1 showed that the tracking performance under feature-changed condition was lower than that under the feature-fixed condition only when the objects were four colors grouped or all unique, suggesting that the performance decrease was moderated by the feature heterogeneity. In Experiment 2, we further examined this effect by manipulating the frequency of feature change. The results showed that when the target set was one color or two colors grouped (the color grouping for the distractor set corresponded with it), the tracking performance decreased significantly as the feature-change frequency increased. However, this trend was not the case when the objects were of the same color or eight unique colors. In addition, a relatively consistent effect appeared both in Experiments 1 and 2. When objects have unique features, the tracking performance decreased significantly as the increase of feature heterogeneity in each frequency of feature changes. Taken together, we concluded that unstable features could be utilized in attentive tracking, and the extent to which the observers relied on surface feature information to assist tracking depended on the level of feature heterogeneity and the frequency of feature change.

One of the most important tasks for the visual system is to construct an internal representation of the spatial properties of objects, including their size. Size perception includes a combination of bottom-up (retinal inputs) and top-down (e.g., expectations) information, which makes the estimates of object size malleable and susceptible to numerous contextual cues. For example, it has been shown that size perception is prone to adaptation: brief previous presentations of larger or smaller adapting stimuli at the same region of space changes the perceived size of a subsequent test stimulus. Large adapting stimuli cause the test to appear smaller than its veridical size and vice versa. Here, we investigated whether size adaptation is susceptible to attentional modulation. First, we measured the magnitude of adaptation aftereffects for a size discrimination task. Then, we compared these aftereffects (on average 15–20%) with those measured while participants were engaged, during the adaptation phase, in one of the two highly demanding central visual tasks: Multiple Object Tracking (MOT) or Rapid Serial Visual Presentation (RSVP). Our results indicate that deploying visual attention away from the adapters did not significantly affect the distortions of perceived size induced by adaptation, with accuracy and precision in the discrimination task being almost identical in all experimental conditions. Taken together, these results suggest that visual attention does not play a key role in size adaptation, in line with the idea that this phenomenon can be accounted for by local gain control mechanisms within area V1.

Remarkable progress has been made in the field of multi-object tracking. Although tracking-by-detection has recently became one of the most popular frameworks, it still has one main drawback: this approach relies heavily on the quality of detection. Thus, the missing detections caused by partial occlusion usually lead to fragment problem. To address this problem, this paper introduces supervoxels to represent objects with partial occlusion, even for missing detections. We first extract superpixels of the foreground, and then our proposed supervoxel consists of spatial-temporal sequences of superpixels. The supervoxels represent tracklets at the image level, so it is robust for initial detection. Then, we incorporate supervoxels into multiple hypotheses tracking by considering the enhanced association with supervoxels (EAS). Moreover, we propose a detection refinement method based on EAS. As our approach allows us to handle partial occlusion problems, we achieve remarkable results in crowded scenes. Finally, our experiments on both MOT15 and MOT16 benchmarks show that our EAS is competitive with the state-of-the-art trackers.

Camera calibration and radial distortion correction are the crucial prerequisites for many applications in image big data and computer vision. Many existing works rely on the Manhattan world assumption to estimate the camera parameters automatically; however, they may perform poorly when there was lack of man-made structure in the scene. As walking humans are the common objects in video surveillance, they have also been used for camera self-calibration, but the main challenges include the noise reduction for the estimation of vanishing points, the relaxation of assumptions on unknown camera parameters, and the radial distortion correction. In this paper, we present a novel framework for camera self-calibration and automatic radial distortion correction. Our approach starts with the reliable human body segmentation that is facilitated by robust object tracking. Mean shift clustering and Laplace linear regression are, respectively, introduced in the estimation of the vertical vanishing point and the horizon line. The estimation of distribution algorithm, an evolutionary optimization scheme, is then utilized to optimize the camera parameters and the distortion coefficients, in which all the unknowns in camera projection can be fine-tuned simultaneously. Experiments on the three public benchmarks and our own captured dataset demonstrate the robustness of the proposed method. The superiority of this algorithm is also verified by the capability of reliably converting 2D object tracking into 3D space.

Cooperative Vehicle and Infrastructure System (CVIS) and Autonomous Vehicle (AV) are two mainstream technologies to improve urban traffic efficiency and vehicle safety in the Intelligent Transportation System (ITS). However, there remain significant obstacles that must be overcome before fully unmanned applications are ready for widespread adoption in a transportation system. To achieve fully driverless driving, the perception ability of vehicle should be accurate, fast, continuous, and wide-ranging. In this paper, an interactive perception framework is proposed, which combines the visual perception of AV and information interaction of CVIS. Based on the framework, an interactive perception-based multiple object tracking (IP-MOT) method is presented. IP-MOT can be divided into two parts. First, a Lidar-only multiple object tracking (L-MOT) method obtains the status of surroundings using the voxel cluster algorithm. Second, the preliminary tracking result is fused with the interactive information to generate the trajectories of target vehicles. Two simulation platforms are established to verify the proposed methods: CVIS simulation platform and Virtual Reality (VR) test platform. The L-MOT algorithm is tested on a public dataset and the IP-MOT algorithm is tested on our simulation platform. The results show that the IP-MOT algorithm can improve the accuracy of object tracking as well as expand the vehicle perception range via combination of CVIS and AV.

Objective of multiple object tracking (MOT) is to assign a unique track identity for all the objects of interest in a video, across the whole sequence. Tracking-by-detection is the most common approach used in addressing MOT problem. In this work, we propose a method to address MOT by defining a dissimilarity measure based on object motion, appearance, structure, and size. We calculate the appearance and structure-based dissimilarity measure by matching histograms following a grid architecture. Motion and size for each track are predicted using the information from track's history. These dissimilarity values are then used in the Hungarian algorithm, in the data association step for track identity assignment. In addition, we introduce a method to address any false detection in stable tracks. The proposed method runs in real time following an online approach. We evaluate our method in both MOT17 benchmark data-set for pedestrian tracking and KITTI benchmark data-set for vehicle tracking using the same system parameters to verify the robustness of the proposed method. The method can achieve state-of-the-art results in both benchmarks.

In this paper, we propose an efficient online multi-object tracking method based on the Gaussian mixture probability hypothesis density (GMPHD) filter and occlusion group management scheme where a hierarchical data association is utilized for the GMPHD filter to reduce the false negatives caused by missed detection. The hierarchical data association consisting of two modules, detection-to-track and track-to-track associations, can recover the lost tracks and their switched IDs. In addition, the proposed grouping management scheme handles occlusion problems with two main parts. The first part, “track merging” can merge the false positive tracks caused by false positive detections from occlusions. The occlusion of the false positive tracks is usually measured with some metric. In this research, we define the occlusion measure between visual objects, as sum-of-intersection-over-each-area (SIOA) instead of the commonly used intersection-over-union (IOU). The second part, “occlusion group energy minimization (OGEM)” prevents the occluded true positive tracks from false “track merging”. Each group of the occluded objects is expressed with an energy function and an optimal hypothesis will be obtained by minimizing the energy. We evaluate the proposed tracker in benchmarks such as MOT15 and MOT17 which are public datasets for multi-person tracking. An ablation study in training dataset reveals not only that “track merging” and “OGEM” complement each other, but also that the proposed tracking method shows more robust performance and less sensitiveness than baseline methods. Also, the tracking performance with SIOA is better than that with IOU for various sizes of false positives. Experimental results show that the proposed tracker efficiently handles occlusion situations and achieves competitive performance compared to the state-of-the-art methods. In fact, our method shows the best multi-object tracking accuracy among the online and real-time executable methods.

Multiple object tracking has been a challenging field, mainly due to noisy detection sets an identity switch caused by occlusion and similar appearance among nearby targets. Previous works rely on appearance models that are built on an individual or several selected frames for the comparison of features, but they cannot encode the long-term appearance changes caused by pose, viewing angle, and lighting conditions. In this paper, we propose an adaptive model that learns online a relatively long-term appearance change of each target. The proposed model is compatible with any feature of fixed dimension or their combination, whose learning rates are dynamically controlled by the adaptive update and spatial weighting schemes. To handle occlusion and nearby objects that are sharing a similar appearance, we also design the cross-matching and re-identification schemes based on the application of the proposed adaptive appearance models. In addition, the 3D geometry information is effectively incorporated in our formulation for data association. The proposed method outperforms all the state of the art on the MOTChallenge 3D benchmark and achieves real-time computation with only a standard desktop CPU. It has also shown superior performance over the state of the art on the 2D benchmark of MOTChallenge.

Surface features are an important component in attentive tracking. However, the neural mechanisms underlying how features affect attentive tracking remain unknown. The present fMRI study addressed this issue by manipulating the intragroup feature complexity and intergroup feature similarity. In particular, this study distinguished the different neural mechanisms of intragroup feature complexity and intergroup feature similarity by investigating the roles of attentional control and working memory in dynamic feature‐based attentive tracking. Behavioral and neuroimaging evidence showed that when targets are distinct from distractors, the intragroup feature complexity of the targets, rather than that of the distractors, mainly increases the visual working memory load and significantly activates the frontoparietal cortical circuit. Thus, the involvement of working memory in feature‐based attentive tracking is modulated by goal‐directed attention control. In addition, when targets are similar to distractors, the intergroup feature similarity (i.e., target‐distractor similarity) mainly affects the allocation of attention. Specifically, target‐distractor similarity affects the goal‐directed attention toward the targets in a stimulus‐driven way and induces an interaction between the ventral and dorsal attention networks. Our fMRI study first separates the neural mechanisms of working memory and attentional control in the process of feature influence on attentive tracking. Intragroup feature complexity of the targets, rather than that of the distractors, increases working memory load and activates the frontoparietal cortical circuit. Intergroup feature similarity affects the goal‐directed attentional control in a stimulus‐driven way by the interaction between the ventral and dorsal attention networks. [ABSTRACT FROM AUTHOR]

In this paper, we propose several methods to improve the performance of multiple object tracking (MOT), especially for humans, in dynamic environments such as robots and autonomous vehicles. The first method is to restore and re-detect unreliable results to improve the detection. The second is to restore noisy regions in the image before the tracking association to improve the identification. To implement the image restoration function used in these two methods, an image inference model based on SRGAN (super-resolution generative adversarial networks) is used. Finally, the third method includes an association method using face features to reduce failures in the tracking association. Three distance measurements are designed so that this method can be applied to various environments. In order to validate the effectiveness of our proposed methods, we select two baseline trackers for comparative experiments and construct a robotic environment that interacts with real people and provides services. Experimental results demonstrate that the proposed methods efficiently overcome dynamic situations and show favorable performance in general situations. [ABSTRACT FROM AUTHOR]

Multiple Object Tracking (MOT) plays an important role in solving many fundamental problems in video analysis and computer vision. Most MOT methods employ two steps: Object Detection and Data Association. The first step detects objects of interest in every frame of a video, and the second establishes correspondence between the detected objects in different frames to obtain their tracks. Object detection has made tremendous progress in the last few years due to deep learning. However, data association for tracking still relies on hand crafted constraints such as appearance, motion, spatial proximity, grouping etc. to compute affinities between the objects in different frames. In this paper, we harness the power of deep learning for data association in tracking by jointly modeling object appearances and their affinities between different frames in an end-to-end fashion. The proposed Deep Affinity Network (DAN) learns compact, yet comprehensive features of pre-detected objects at several levels of abstraction, and performs exhaustive pairing permutations of those features in any two frames to infer object affinities. DAN also accounts for multiple objects appearing and disappearing between video frames. We exploit the resulting efficient affinity computations to associate objects in the current frame deep into the previous frames for reliable on-line tracking. Our technique is evaluated on popular multiple object tracking challenges MOT15, MOT17 and UA-DETRAC. Comprehensive benchmarking under twelve evaluation metrics demonstrates that our approach is among the best performing techniques on the leader board for these challenges. The open source implementation of our work is available at https://github.com/shijieS/SST.git. [ABSTRACT FROM AUTHOR]

Individual differences in attentional abilities provide an interesting approach in studying visual attention as well as the relation of attention to other psychometric measures. However, recent research has demonstrated that many tasks from experimental research are not suitable for individual differences research, as they fail to capture these differences reliably. Here, we provide a test for individual differences in visual attention which relies on the multiple object tracking task (MOT). This test captures individual differences reliably in 6 to 15 min. Within the task, the participants have to maintain a set of targets (among identical distractors) across an interval of object motion. It captures the efficiency of attentional deployment. Importantly, this test was explicitly designed and tested for reliability under conditions that match those of most laboratory research (restricted sample of students, approximately n = 50). The test is free to use and runs fully under open-source software. In order to facilitate the application of the test, we have translated it into 16 common languages (Chinese, Danish, Dutch, English, Finnish, French, German, Italian, Japanese, Norwegian, Polish, Portuguese, Russian, Spanish, Swedish, and Turkish). The test can be downloaded at https://osf.io/qy6nb/. We hope that this MOT test supports researchers whose field of study requires capturing individual differences in visual attention reliably. [ABSTRACT FROM AUTHOR]

To build a smart basketball court, one basic task is to track players with the aid of the basketball court monitoring. This task can be regarded as a special case of the multiple object tracking (MOT) problem. But different from it, the task puts request to the existing MOT methods with both good accuracy and operational efficiency (toward real time) in the new basketball scenario. To deal with this task, we make the following attempts: (1) Considering the differences between pedestrians and basketball players and the lack of corresponding dataset for basketball players tracking, we construct a new MOT dataset under the basketball court monitoring scene, to better evaluate MOT methods in our task and to help promote future research in the related task, (2) evaluating the performance of the several candidate MOT methods on the new dataset, and (3) proposing the issues to be further addressed in this specific scenario. [ABSTRACT FROM AUTHOR]

The purpose of this study is to explore the dynamic visual attention distribution characteristics of basketball players and its relationship with sports performance by using the dual task paradigm of multi-target tracking task and point detection task. Methods: 48 college students (24 basketball players and 24 general college students) were selected to analyze the performance of dynamic visual information processing of sample basketball players by multi-objective tracking task. The processing characteristics of dynamic visual information of sample basketball players were discussed by increasing the tracking target load, and the dynamic information cognitive processing process of sample basketball players was analyzed by point detection task This paper discusses the distribution and processing characteristics of mental resources, and discusses the correlation between the dynamic visual attention characteristics and sports performance of sample basketball players. Results: 1) there was no significant interaction between group and target number dimension, no significant interaction between group and position dimension of point detection, significant interaction effect between group and detection stimulus appearance dimension, and the correlation degree between point detection rate and motor performance (technical statistical index) was different in different positions. 2) With the increase of the number of targets, the task difficulty increases, and the tracking accuracy of sample basketball players shows a downward trend. In the mot task, there is no significant difference between the sample basketball players and the general college students. In the double task experiment, the sample basketball players pay more attention to the empty position, which means that more attention resources are allocated to the empty position. This visual attention feature makes the sample basketball players may have more assists and fewer fouls in the game, which can promote the good sports performance. [ABSTRACT FROM AUTHOR]

Thermal sensors bring a lot of benefits for object tracking due to the robustness to lighting changes over time, which still give a great difficulty to traditional tracking methods. Despite this powerful advantage, the ambiguity due to background clutters and occlusions makes the problem of thermal sensor-based tracking intractable. In this paper, we propose a novel framework for multiple object tracking in livestock breeding spaces using a single thermal sensor. The key idea of the proposed method is to define a new rotatable elliptical motion model, which is suitable for describing complicated motions of animals. Moreover, the proposed local constraint guides our tracker to robustly chase targets even with severe occlusions. Experimental results on various thermal video sequences show the significant improvement for tracking animals compared to other approaches proposed in the literature. [ABSTRACT FROM AUTHOR]

In the tracking‐by‐detection approach of online multiple object tracking (MOT), a major challenge is how to associate object detections on the new video frame with previously tracked objects. Two important aspects that directly influence the performance of MOT are quality of detection and accuracy in data association. The authors propose an efficient and unified MOT framework for improved object detection, followed by enhanced object tracking. The object detection and tracking are considered as two independent functions in the tracking‐by‐detection paradigm. In this study, object detection accuracy has been increased by employing a faster region‐based convolutional neural network (Faster R‐CNN) modified with the feedback region proposals from the tracker. Target association is performed by the correlation filter‐based Siamese CNN model, which finds the similarity score between the input image patches. The Siamese CNN is trained using a supervised hard sample mining strategy. An optical flow‐based motion model is employed to predict the next probable location of the targets from the tracker and these region proposals are fed back to the classifier module of Faster R‐CNN. The authors' extensive analysis of publicly available MOT benchmark datasets and comparison with the state‐of‐the‐art tracking methods demonstrate competitive tracking performance of the proposed MOT framework. [ABSTRACT FROM AUTHOR]

In the real world, complex dynamic scenes often arise from the composition of simpler parts. The visual system exploits this structure by hierarchically decomposing dynamic scenes: When we see a person walking on a train or an animal running in a herd, we recognize the individual’s movement as nested within a reference frame that is, itself, moving. Despite its ubiquity, surprisingly little is understood about the computations underlying hierarchical motion perception. To address this gap, we developed a class of stimuli that grant tight control over statistical relations among object velocities in dynamic scenes. We first demonstrate that structured motion stimuli benefit human multiple object tracking performance. Computational analysis revealed that the performance gain is best explained by human participants making use of motion relations during tracking. A second experiment, using a motion prediction task, reinforced this conclusion and provided fine-grained information about how the visual system flexibly exploits motion structure. [ABSTRACT FROM AUTHOR]

Multifocal attention is the ability to simultaneously attend to multiple objects, and is critical for typical functioning. Although adults are able to use multifocal attention, little is known about the development of this ability. In two experiments, we investigated multifocal attention in 6–8-year-old children and adults using a child-friendly, computerized multiple object tracking task designed to encourage the use of multifocal attention. We also investigated whether multifocal attention in children is deployed independently across left and right hemifields of vision, as in adults. Our results suggest that children's capacity for multifocal attention increases significantly across middle childhood. We also found evidence that at least one signature of hemifield-independent multifocal attention, the bilateral field advantage, can be observed in children. [ABSTRACT FROM AUTHOR]

In daily life, humans frequently perform visuospatial tasks together (e.g., visual search) and distribute the labor in such tasks. Previous research has shown that humans prefer a left and right labor division in a joint multiple object tracking (MOT) task. Yet, findings from studies investigating individuals' tracking ability suggest attentional capacities may be more maximally used with a top and bottom labor division. We investigated whether co-actors' labor division preference is influenced by how they are seated (neighboring vs. opposite of each other) or how the MOT task is displayed (portrait vs. landscape). We find that pairs attain a higher performance using a top and bottom labor division and preferred this labor division compared to a left and right division. This preference was unaffected by the seating arrangement. For the landscape display, however, we find that participants no longer attain a higher performance for the top and bottom labor division and accordingly participants' preference for this labor division was greatly reduced as well. Overall, we propose that co-actors are sensitive to changes within their environment, which allows them to choose a labor division that maximizes use of their individual attentional capacities. [ABSTRACT FROM AUTHOR]

The goal of multi object tracking is to find location of the target objects in number of consecutive frames of a video. Tracking of multiple human objects in a scene is one of the challenging problems in computer vision applications due to illumination variation, object occlusion, abrupt motion etc. This paper introduces a new method for multiple human object tracking by exploiting the properties of Daubechies complex wavelet transform and Zernike moment. The proposed method uses combination of Daubechies complex wavelet transform and Zernike moment as a feature of objects. The motivation behind using combination of these two as a feature of object, because shift invariance and better edge representation properties make Daubechies complex wavelet transform suitable for locating object in consecutive frames whereas translation invariant property of Zernike moment is also helpful for correct object identification in consecutive frames. The proposed method is capable to handle full occlusion, partial occlusion, split and object re-enter problems. The experimental results validate the effectiveness and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

Object detection and tracking are critical and challenging problems in vehicle environment perception systems, and have received broad attention in recent years. A novel detection and tracking algorithm taking both accuracy and real-time performance into account is proposed in this paper. First, we employ a fusion algorithm based on stereo vision and deep learning in object detection, which achieves high accuracy using two complementary algorithms. Then, a prediction-association algorithm which uses a Kalman filter and Hungarian assignment for multiple object tracking is employed for object tracking. In addition, a detection and tracking framework based on stereo vision improves the robustness of environmental perception system. Experimental results demonstrate that the proposed algorithm has high accuracy and can meet the real-time performance requirement. [ABSTRACT FROM AUTHOR]

Object detection and tracking are critical and challenging problems in vehicle environment perception systems, and have received broad attention in recent years. A novel detection and tracking algorithm taking both accuracy and real-time performance into account is proposed in this paper. First, we employ a fusion algorithm based on stereo vision and deep learning in object detection, which achieves high accuracy using two complementary algorithms. Then, a prediction-association algorithm which uses a Kalman filter and Hungarian assignment for multiple object tracking is employed for object tracking. In addition, a detection and tracking framework based on stereo vision improves the robustness of environmental perception system. Experimental results demonstrate that the proposed algorithm has high accuracy and can meet the real-time performance requirement. [ABSTRACT FROM AUTHOR]

Multiple object tracking (MOT) at road intersections is a major research area in the field of intelligent transportation systems (ITS). In dynamic scenes like urban traffic, there is a variety of moving objects ranging from simple pedestrians to large trucks. Occlusion due to the interaction between these moving objects is one of the major problems in traffic tracking and monitoring at intersections. This paper presents an occlusion handling approach using an efficient feature-based multiple object tracking technique. A considerable amount of research work has been done in the area of occlusion handling but most of the proposed techniques can either handle occlusion of only two objects or fail to segment the individual vehicles throughout occlusion. The proposed approach uses radius nearest neighbour (RNN) classification to group the unmatched feature points along with matched feature points to update the object model. In order to validate the efficiency of this method, it is tested on four different urban traffic sequences from the Urban Tracker dataset and four traffic sequences from the Ko-PER Intersection Dataset. Experimental results depict the tracking approach of RNN based scheme compared to the other schemes for better occlusion handling and individual object segmentation. [ABSTRACT FROM AUTHOR]

Face tracking is one of key steps in real surveillance system. In this paper, a real-time face tracking system using multiple objects tracking algorithm is proposed. Multiple objects tracking algorithms typically consist of two parts: object detection and data association. In our system, we particularly use Multi-task convolutional neural network (MTCNN) to detect faces. Simultaneously, aiming at dealing with the tracking failure caused by object occlusion or rapid object movement, we use multiple features including appearance feature, motion feature, and shape feature for tracking. Furthermore, a judgement method is applied to measure whether tracking is successful or not. After that, depending on the tracking state, we adjust the weights of different features for feature fusion. From the experimental results, it can be concluded that, compared with the traditional tracking algorithms the proposed multi-object tracking algorithm has a better tracking effect in the real scenes. [ABSTRACT FROM AUTHOR]

The way in which vision represents objects as being the same despite movement and qualitative changes has been extensively investigated in contemporary psychology. However, the formal properties of the visual sameness relation are still unclear, for example, whether it is an identity-like, equivalence relation. The paper concerns one aspect of this problem: the transitivity of visual sameness. Results obtained by using different experimental paradigms are analysed, in particular studies using streaming/bouncing stimuli, multiple object tracking experiments and investigations concerning object-specific preview benefit, and it is argued that the transitive interpretation of visual sameness is the most plausible given the current stage of knowledge. What is more, it is claimed that the way in which visual sameness is represented suggests that in some cases it should be characterized as a "primitive sameness", similarly as in philosophical theories postulating "thisness". [ABSTRACT FROM AUTHOR]

One of the most important tasks for the visual system is to construct an internal representation of the spatial properties of objects, including their size. Size perception includes a combination of bottom-up (retinal inputs) and top-down (e.g., expectations) information, which makes the estimates of object size malleable and susceptible to numerous contextual cues. For example, it has been shown that size perception is prone to adaptation: brief previous presentations of larger or smaller adapting stimuli at the same region of space changes the perceived size of a subsequent test stimulus. Large adapting stimuli cause the test to appear smaller than its veridical size and vice versa. Here, we investigated whether size adaptation is susceptible to attentional modulation. First, we measured the magnitude of adaptation aftereffects for a size discrimination task. Then, we compared these aftereffects (on average 15–20%) with those measured while participants were engaged, during the adaptation phase, in one of the two highly demanding central visual tasks: Multiple Object Tracking (MOT) or Rapid Serial Visual Presentation (RSVP). Our results indicate that deploying visual attention away from the adapters did not significantly affect the distortions of perceived size induced by adaptation, with accuracy and precision in the discrimination task being almost identical in all experimental conditions. Taken together, these results suggest that visual attention does not play a key role in size adaptation, in line with the idea that this phenomenon can be accounted for by local gain control mechanisms within area V1. [ABSTRACT FROM AUTHOR]