Search

Our ability to track the paths of multiple visual objects moving between the hemifields requires effective integration of information between the two cerebral hemispheres. Coherent neural oscillations in the gamma band (35-70 Hz) are hypothesized to drive this information transfer. Here we manipulated the need for interhemispheric integration using a novel multiple object tracking (MOT) task in which stimuli either moved between the two visual hemifields, requiring interhemispheric integration, or moved within separate visual hemifields. We used electroencephalography (EEG) to measure interhemispheric coherence during the task. Human observers (21 women; 20 men) were poorer at tracking objects between versus within hemifields, reflecting a cost of interhemispheric integration. Critically, gamma coherence was greater in trials requiring interhemispheric integration, particularly between sensors over parieto-occipital areas. In approximately half of the participants, the observed cost of integration was associated with a failure of the cerebral hemispheres to become coherent in the gamma band. Moreover, individual differences in this integration cost correlated with endogenous gamma coherence at these same sensors, although with generally opposing relationships for the real and imaginary part of coherence. The real part (capturing synchronization with a near-zero phase lag) benefited between-hemifield tracking; imaginary coherence was detrimental. Finally, instantaneous phase coherence over the tracking period uniquely predicted between-hemifield tracking performance, suggesting that effective integration benefits from sustained interhemispheric synchronization. Our results show that gamma coherence mediates interhemispheric integration during MOT and add to a growing body of work demonstrating that coherence drives communication across cortically distributed neural networks. [ABSTRACT FROM AUTHOR]

With the increasing popularity of Unmanned Aerial Vehicles (UAVs) in computer vision-related applications, intelligent UAV video analysis has recently attracted the attention of an increasing number of researchers. To facilitate research in the UAV field, this paper presents a UAV dataset with 100 videos featuring approximately 2700 vehicles recorded under unconstrained conditions and 840k manually annotated bounding boxes. These UAV videos were recorded in complex real-world scenarios and pose significant new challenges, such as complex scenes, high density, small objects, and large camera motion, to the existing object detection and tracking methods. These challenges have encouraged us to define a benchmark for three fundamental computer vision tasks, namely, object detection, single object tracking (SOT) and multiple object tracking (MOT), on our UAV dataset. Specifically, our UAV benchmark facilitates evaluation and detailed analysis of state-of-the-art detection and tracking methods on the proposed UAV dataset. Furthermore, we propose a novel approach based on the so-called Context-aware Multi-task Siamese Network (CMSN) model that explores new cues in UAV videos by judging the consistency degree between objects and contexts and that can be used for SOT and MOT. The experimental results demonstrate that our model could make tracking results more robust in both SOT and MOT, showing that the current tracking and detection methods have limitations in dealing with the proposed UAV benchmark and that further research is indeed needed. [ABSTRACT FROM AUTHOR]

The extent to which similar capacity limits in visual attention and visual working memory indicate a common shared underlying mechanism is currently still debated. In the spatial domain, the multiple object tracking (MOT) task has been used to assess the relationship between spatial attention and spatial working memory though existing results have been inconclusive. In three dual task experiments, we examined the extent of interference between attention to spatial positions and memory for spatial positions. When the position monitoring task required keeping track of target identities through colour-location binding, we found a moderate detrimental effect of position monitoring on spatial working memory and an ambiguous interaction effect. However, when this task requirement was removed, load increases in neither task were detrimental to the other. The only very moderate interference effect that remained resided in an interaction between load types but was not consistent with shared capacity between tasks—rather it was consistent with content-related crosstalk between spatial representations. Contrary to propositions that spatial attention and spatial working memory may draw on a common shared set of core processes, these findings indicate that for a purely spatial task, perceptual attention and working memory appear to recruit separate core capacity-limited processes. [ABSTRACT FROM AUTHOR]

Electrical cables consist of numerous wires, the three-dimensional (3D) shape of which significantly impacts the cables' overall properties, such as bending stiffness. Although X-ray computed tomography (CT) provides a non-destructive method to assess these properties, accurately determining the 3D shape of individual wires from CT images is challenging due to the large number of wires, low image resolution, and indistinguishable appearance of the wires. Previous research lacked quantitative evaluation for wire tracking, and its overall accuracy heavily relied on the accuracy of wire detection. In this study, we present a long short-term memory-based approach for wire tracking that improves robustness against detection errors. The proposed method predicts wire positions in subsequent frames based on previous frames. We evaluate the performance of the proposed method using both actual annotated cables and artificially noised annotations. Our method exhibits greater tracking accuracy and robustness to detection errors compared with the previous method. [ABSTRACT FROM AUTHOR]

Multiple Object Tracking is an important vitally important fundamental task in computer vision. Visual tracking becomes challenging when objects move in groups and are obscured from each other. There are two mainstream solution strategies for these group models. One is to transform the data association problem into a graph matching problem for solving, while the other is to apply the social power model as an advanced constraint for group tracking. In the former case, the solving difficulty geometric growth as the number of tracked objects increases, and the computing efficiency for real-time tracking demand cannot be met. The latter strategy tends to set up fixed-size groups or offline training rules, resulting in a lack of flexibility that limits their scenario generalization. According to the shortcomings of existing methods, this paper proposes a novel multiple object tracking method with spatio-temporal correlation and graph neural networks. Firstly, the relational features of the historical trajectories are extracted through the spatio-temporal relationship learning module, which models the spatio-temporal correlations of the objects and dynamically constructs the group structure online. Then, the graph neural network is combined with appearance and motion information, and the similarity between each detection and tracklet is used as a weight in node feature aggregation to make powerful distinctions between node features. Meanwhile, the spatio-temporal correlation method is also used to solve target loss issues caused by occlusion. Even collocated with linearly assigned data association method, good tracking results are still achieved, with a low computational complexity. Experiments on three challenging public datasets, namely MOT16, MOT17, and MOT20, validated the accuracy and efficiency of the proposed tracking method. • Spatio-temporal correlation is modeled based on historical trajectories. • Undetected objects can be tracked with spatio-temporal correlation constraints. • Appearance and motion information are used to select graph nodes for aggregation. • Secondary matching enhances target association between detections and tracklets. [ABSTRACT FROM AUTHOR]

Data association is a crucial component for any multiple object tracking (MOT) method that follows the tracking-by-detection paradigm. To generate complete trajectories such methods employ a data association process to establish assignments between detections and existing targets during each timestep. Recent data association approaches try to solve either a multi-dimensional linear assignment task or a network flow minimization problem or tackle it via multiple hypotheses tracking. However, during inference an optimization step that computes optimal assignments is required for every sequence frame inducing additional complexity to any given solution. To this end, in the context of this work we introduce Transformer-based Assignment Decision Network (TADN) that tackles data association without the need of any explicit optimization during inference. In particular, TADN can directly infer assignment pairs between detections and active targets in a single forward pass of the network. We have integrated TADN in a rather simple MOT framework, designed a novel training strategy for efficient end-to-end training and demonstrated the high potential of our approach for online visual tracking-by-detection MOT on several popular benchmarks, i.e. MOT17, MOT20 and UA-DETRAC. Our proposed approach demonstrates strong performance in most evaluation metrics despite its simple nature as a tracker lacking significant auxiliary components such as occlusion handling or re-identification. The implementation of our method is publicly available at https://github.com/psaltaath/tadn-mot. • A novel data association method, TADN, for online MOT without LDAP during inference. • Online tracking-by-detection integrating TADN with motion and appearance submodules. • Validation in MOT17, MOT20 and UA-DETRAC demonstrate the superior performance of TADN. • Ablation study experiments regarding TADN's architecture and configuration options. [ABSTRACT FROM AUTHOR]

Accurately detecting and counting fruits is essential for orchard yield estimation and smart management. However, two drawbacks are inherent in the current citrus counting algorithms: First, the robustness needs to be improved, particularly illumination changes and dense occlusion. Secondly, the actual operating efficiency of the system is low. To tackle the above issues, this paper proposed a robust and efficient fruit-counting pipeline based on Unmanned Aerial Vehicle (UAV). First, to obtain UAV video streaming data online, a live broadcast platform and a flight control Application called FlyCounter were developed. Secondly, the Illumination-Adaptive-Transformer network is used to enhance the low-illumination citrus image in real-time. Then, for the specific challenging scenario, a novel model named Fruit-YOLO is designed to accurately detect citrus in data streams. Finally, the DeepSORT is adopted to track and count fruits in video sequences. The results indicate that for detection performance, the P, R and mAP of the Fruit-YOLO of input enhanced image are 0.898, 0.854 and 0.929 respectively, which are 1.7%, 4.6% and 3.5% higher than the original image respectively. Regarding counting performance, for the daytime and evening scenes, the ID switch, MOTA and Error Mean of the pipeline are 63.7, 0.86 and 9.81% respectively. The MAPE of the offline and online operations of the pipeline are 4.36% and 12.66% respectively. The time and resources consumed by the system under parallel operations with different numbers of UAVs were analyzed. In this study, an online counting pipeline based on UAVs that can work in low-light scenarios is implemented for the first time. The system has good performance in both daytime and nighttime scenarios, enabling efficient counting in orchards and extending operating time. OBJECTIVE: To implement a robust and efficient fruit counting system for large-scale unstructured orchards. METHODS: First, a new UAV platform and a flight control application are proposed to transmit UAV data streams; then, a detection network named Fruit-YOLO is designed, and combined with the IAT network and DeepSORT algorithm to estimate the overall number of fruits in the orchard. RESULTS: The system achieved a mAP of 0.929 and a MAPE of 12.66%. CONCLUSION: The system has good performance in both daytime and nighttime scenarios, enabling efficient counting in orchards and extending working time. [Display omitted] • An End-to-end and efficient counting system for orchards using UAVs. • A new UAV platform and flight application. • A novel detection model is proposed for UAV scenarios. • Introducing IAT network to improve the system's performance under low illumination. • Improve work efficiency by using multiple UAVs to operate simultaneously. [ABSTRACT FROM AUTHOR]

Multiple object tracking in drone videos is a vital vision task with broad application prospects, but most trackers use spatial or appearance clues alone to correlate detections. Our proposed Multi-Tracker uses a novel similarity measure that combines position and appearance information. We designed the GM-YOLO network to provide high-quality detections as input to Multi-Tracker. Add a Coordinate Attention mechanism and a weighted Bidirectional Feature Pyramid Network structure to the Backbone, each feature point's effective receptive field is modeled as a Gaussian distribution. To accurately obtain the motion and appearance features of the object, the adaptive noise covariance Kalman filter is used to get the position information, MB-OSNet network is designed to use global features to learn contour information to retrieve images from a wider field of view while incorporating Part-Level elements that contain more fine-grained data. Finally, the motion and appearance features are jointly compared to realize multi object tracking. The performance of the GM-YOLO object detector and the Multi-Tracker was verified on the VisDrone MOT and UAVDT datasets. • Integrates position and appearance data for drone video multi-object tracking. • Enhances detector with Coordinate Attention and weighted Bidirectional Feature Pyramid Network. • Models feature points' receptive fields as Gaussian distributions. • Utilizes adaptive noise covariance Kalman filter for precise position data. • Feature extraction network leverages global features and Part-Level elements for fine-grained contour learning and wider field of view. [ABSTRACT FROM AUTHOR]

Multiple object tracking (MOT) is a sophisticated computer vision task that aims to detect and track the trajectories of all objects within a given scene. MOT necessitates the establishment of unique identifiers for each object in the scene and currently, the majority of MOT works adopt tracking-by-detection, using re-identification techniques to associate objects based on appearance or motion features. However, traditional MOT methods may yield suboptimal results when appearance features are unreliable or geometric features are confounded by irregular motions. Consequently, this paper proposes a cascaded matching tracker called IMMSegTrack, which replaces detection boxes with segmentation contours for IoU matching and generates convincing prediction outcomes by blending multiple adaptive Kalman filter models. With the guide of interactive multiple model and pixel-level matching, better performance could be achieved through well-designed cascaded association. Our experimental innovations mainly focus on the prediction and matching aspects within the MOT framework, while tests on DanceTrack and SoccerNet datasets have been conducted using the original models of YOLOv8 to obtain desired results. The relevant codes for the experiments are available at https://github.com/xwh129/IMMSegTrack.git. In addition, it is considerable to retrain model weights and expand the experiments to more datasets for better performance. • Employment of IMM to perform better in irregular motion estimations. • Three-stages cascaded matching to achieve reasonable associations. • Utilizing contours to achieve pixel-level comparisons in IOU matching. • Removals of trajectories on the brim to cut down redundant cached frames. [ABSTRACT FROM AUTHOR]

The paper considers the problem of tracking an unknown and time-varying number of unlabeled moving objects using multiple unordered measurements with unknown association to the objects. The proposed tracking approach integrates Bayesian nonparametric modeling with Markov chain Monte Carlo methods to estimate the parameters of each object when present in the tracking scene. In particular, we adopt the dependent Dirichlet process (DDP) to learn the multiple object state prior by exploiting inherent dynamic dependencies in the state transition using the dynamic clustering property of the DDP. Using the DDP to draw the mixing measures, Dirichlet process mixtures are used to learn and assign each measurement to its associated object identity. The Bayesian posterior to estimate the target trajectories is efficiently implemented using a Gibbs sampler inference scheme. A second tracking approach is proposed that replaces the DDP with the dependent Pitman–Yor process in order to allow for a higher flexibility in clustering. The improved tracking performance of the new approaches is demonstrated by comparison to the generalized labeled multi-Bernoulli filter.

Multiple Object Tracking (MOT) focuses on tracking all the objects in a video. Most MOT solutions follow a tracking-by-detection or a joint detection tracking paradigm to generate the object trajectories by exploiting the correlations between the detected objects in consecutive frames. However, according to our observations, considering only the correlations between the objects in the current frame and the objects in the previous frame will lead to an exponential information decay over time, thus resulting in a misidentification of the object, especially in scenes with dense crowds and occlusions. To address this problem, we propose an effectively finite-tailed updating (FTU) strategy to generate the appearance template of the object in the current frame by exploiting its local temporal context in videos. To be specific, we model the appearance template for the object in the current frame on the appearance templates of the objects in multiple earlier frames and dynamically combine them to obtain a more effective representation. Extensive experiments have been conducted, and the experimental results show that our tracker outperforms the state-of-the-art methods on MOT Challenge Benchmark. We have achieved 73.7% and 73.0% IDF1, and 46.1% and 45.0% MT on the MOT16 and MOT17 datasets, which are 0.9% and 0.7% IDFI higher, and 1.4% and 1.8% MT higher than FairMOT repsectively.

In this work we present a novel end-to-end solution for tracking objects (i.e., vessels), using video streams from aerial drones, in dynamic maritime environments. Our method relies on deep features, which are learned using realistic simulation data, for robust object detection, segmentation and tracking. Furthermore, we propose the use of rotated bounding-box representations, which are computed by taking advantage of pixel-level object segmentation, for improved tracking accuracy, by reducing erroneous data associations during tracking, when combined with the appearance-based features. A thorough set of experiments and results obtained in a realistic shipyard simulation environment, demonstrate that our method can accurately, and fast detect and track dynamic objects seen from a top-view.

Multiple-object tracking (MOT) has received an increasing attention due to the rapid development of autonomous driving. However, the MOT problem is still challenging mainly due to the occlusion and scale variation. Motivated by the fact that the discriminative correlation filters-based (DCFB) tracking algorithms can tackle these problems and significantly improve the accuracy of single object tracking, how to exploit the DCFB tracking algorithms for MOT is worthy studying. Moreover, the corrupted training samples due to the occlusion make DCFB tracking methods to update the appearance model of target uncorrected and result in tracking drift. In this paper, we exploit Markov decision process to integrate the DCFB tracking method into our MOT framework and address the update problem of the appearance model in DCFB tracking method. Moreover, in order to overcome the challenges of occlusion and scale variation, to prevent target drift during tracking, we use two DCFB trackers with different update frequencies and a novel update strategy to predict the location of targets. The part-based method is used to extract robust features to tackling the challenges of occlusion and scale change. In order to verify the efficiency of our algorithm, experiments are performed based on KITTI tracking benchmark. The results demonstrate that our method achieves state-of-the-art performance and outperforms the state-of-the-art algorithms in road scenarios.

The human brain can effortlessly perform vision processes using the visual system, which helps solve multi-object tracking (MOT) problems. However, few algorithms simulate human strategies for solving MOT. Therefore, devising a method that simulates human activity in vision has become a good choice for improving MOT results, especially occlusion. Eight brain strategies have been studied from a cognitive perspective and imitated to build a novel algorithm. Two of these strategies gave our algorithm novel and outstanding results, rescuing saccades and stimulus attributes. First, rescue saccades were imitated by detecting the occlusion state in each frame, representing the critical situation that the human brain saccades toward. Then, stimulus attributes were mimicked by using semantic attributes to reidentify the person in these occlusion states. Our algorithm favourably performs on the MOT17 dataset compared to state-of-the-art trackers. In addition, we created a new dataset of 40,000 images, 190,000 annotations and 4 classes to train the detection model to detect occlusion and semantic attributes. The experimental results demonstrate that our new dataset achieves an outstanding performance on the scaled YOLOv4 detection model by achieving a 0.89 mAP 0.5.

The idea of SLAM (Simultaneous Localization and Mapping) being a solved problem revolves around the static world assumption, even though autonomous systems are gaining environmental perception capabilities by exploiting the advances in computer vision and data-driven approaches. The computational demands and time complexities remain the main impediment in the effective fusion of the paradigms. In this paper, a framework to solve the dynamic SLAM problem is proposed. The dynamic regions of the scene are handled by making use of Visual-LiDAR based MODT (Multiple Object Detection and Tracking). Furthermore, minimal computational demands and real-time performance are ensured. The framework is tested on the KITTI Datasets and evaluated against the publicly available evaluation tools for a fair comparison with state-of-the-art SLAM algorithms. The results suggest that the proposed dynamic SLAM framework can perform in real-time with budgeted computational resources. In addition, the fused MODT provides rich semantic information that can be readily integrated into SLAM.

In sports, peripheral vision is expected to play an important role in tasks that demand distributed attention and motion-change detection. By using the Multiple-Object-Tracking (MOT) task, these demands were simulated in a well-controlled laboratory environment. Participants tracked four target out of ten moving objects (6 distractors) and pressed a button when one of the ten objects stopped. Detection rates for tracked targets were compared to detection rates of non-tracked distractors at eccentricities between 5° and 25°. The study's aim was to test how the location of attention affects peripheral motion detection. The results show a large attention effect because target stops were detected in 89 % and distractor stops only in 55 % of the trials. Distractor stops were more likely detected when they occurred closer to the fovea while target stops were detected at all eccentricities. That means, orienting attention at target objects facilitates the peripheral detection of their motion changes in monitoring tasks. Having distractors closer to the fovea increases the chance to also detect motion changes of unattended objects. On a theoretical level, results support a tracking mechanism with object-based attention, serial covert attention shifts and flexible but limited attentional resources. On a practical level, sports' experts should use their extensive knowledge to locate attention on most-relevant objects and reduce the eccentricity to other objects to detect motion changes of attended and unattended objects.

Previous research has revealed the uniqueness-facilitation effect in the multiple object tracking (MOT) task: simple distinct identities and surface features of moving targets could facilitate attentional tracking. By adapting compound stimuli, the present study investigated whether the global or local properties played the main role in the uniqueness-facilitation effect in the MOT task. The uniqueness of local properties, of global properties or of both local and global properties were considered. Observers’ tracking performance in alternative conditions were compared with that in the homogeneous condition wherein all stimuli have identical local and global properties. Results from two experiments suggest that the global properties played the key role in facilitating tracking. The distinctiveness of local properties can also facilitate tracking with global properties being homogeneous. However, when the stimuli’s global properties are distinct from each other—whether the local properties being unique or not—observers’ tracking performance can achieve the same level as that in the unitary-uniqueness condition wherein the moving objects were distinct unitary letters. These results revealed a global superiority effect in the MOT task. Finally, the facilitation effects of the global and local properties might depend on the stimulus sparsity. When the compound stimuli had fewer local elements, the uniqueness facilitation effect on tracking decreased.

Multiple object tracking (MOT) involves simultaneous tracking of a certain number of target objects amongst a larger set of objects as they all move unpredictably over time. The prevalent explanation for successful target tracking by humans in MOT involving visually identical objects is based on the Visual Indexing Theory. This assumes that each target is indexed by a pointer using a non-conceptual mechanism to maintain an object's identity even as its properties change over time. Thus, successful tracking requires successful indexing and the absence of identification errors. Identity maintenance and successful tracking are measured in terms of identification (ID) and tracking accuracy respectively, with higher accuracy indicating better identity maintenance or better tracking. Existing evidence suggests that humans have high tracking accuracy despite poor identification accuracy, suggesting that it might be possible to perform MOT without indexing. Our work adds to existing evidence for this position through two experiments, and presents a computational model of multiple object tracking that does not require indexes. Our empirical results show that identification accuracy is aligned with tracking accuracy in humans for tracking up to three, but is lower when tracking more objects. Our computational model of MOT without indexing accounts for several empirical tracking accuracy patterns shown in earlier studies, reproduces the dissociation between tracking and identification accuracy produced earlier in the literature as well as in our experiments, and makes several novel predictions., Competing Interests: Competing Interests: The authors declare no conflict of interests., (© 2024 Massachusetts Institute of Technology.)

Thematic roles characterise the functions of participants in events, but there is no agreement on how these roles are identified in the real world. In three experiments, we examined how role identification in push events is supported by the visual object-tracking system. Participants saw one to three push events in visual scenes with nine identical randomly moving circles. After a period of random movement, two circles from one of the push events and a foil object were given different colours and the participants had to identify their roles in the push with an active sentence, such as red pushed blue. It was found that the participants could track the agent and patient targets and generate descriptions that identified their roles at above chance levels, even under difficult conditions, such as when tracking multiple push events (Experiments 1–3), fixating their gaze (Experiment 1), performing a concurrent speeded-response task (Experiment 2), and when tracking objects that were temporarily invisible (Experiment 3). The results were consistent with previous findings of an average tracking capacity limit of four objects, individual differences in this capacity, and the use of attentional strategies. The studies demonstrated that thematic role information can be maintained when tracking the identity of visually identical objects, then used to map role fillers (e.g., the agent of a push event) into their appropriate sentence positions. This suggests that thematic role features are stored temporarily in the visual object-tracking system. [ABSTRACT FROM AUTHOR]

Tracking objects over time, i.e., identity (ID) consistency, is important when dealing with multiple object tracking (MOT). Especially in complex scenes with occlusion and interaction of objects this is challenging. Significant improvements in single object tracking (SOT) methods have inspired the introduction of SOT to MOT to improve the robustness, that is, maintaining object identities as long as possible, as well as helping alleviate the limitations from imperfect detections. SOT methods are constantly generalized to capture appearance changes of the object, and designed to efficiently distinguish the object from the background. Hence, simply extending SOT to a MOT scenario, which consists of a complex scene with spatially mixed, occluded, and similar objects, will encounter problems in computational efficiency and drifted results. To address this issue, we propose a binary-channel verification model that deeply excavates the potential of SOT in refining the representation while maintaining the identities of the object. In particular, we construct an integrated model that jointly processes the previous information of existing objects and new incoming detections, by using a unified correlation filter through the whole process to maintain consistency. A delay processing strategy consisting of the three parts—attaching, re-initialization, and re-claiming—is proposed to tackle drifted results caused by occlusion. Avoiding the fuzzy appearance features of complex scenes in MOT, this strategy can improve the ability to distinguish specific objects from each other without contaminating the fragile training space of a single object tracker, which is the main cause of the drift results. We demonstrate the effectiveness of our proposed approach on the MOT17 challenge benchmarks. Our approach shows better overall ID consistency performance in comparison with previous works. [ABSTRACT FROM AUTHOR]

Tracking-by-detection is one of the most popular approaches to tracking multiple objects in which the detector plays an important role. Sometimes, detector failures caused by occlusions or various poses are unavoidable and lead to tracking failure. To cope with this problem, we construct a heterogeneous association graph that fuses high-level detections and low-level image evidence for target association. Compared with other methods using low-level information, our proposed heterogeneous association fusion (HAF) tracker is less sensitive to particular parameters and is easier to extend and implement. We use the fused association graph to build track trees for HAF and solve them by the multiple hypotheses tracking framework, which has been proven to be competitive by introducing efficient pruning strategies. In addition, the novel idea of adaptive weights is proposed to analyze the contribution between motion and appearance. We also evaluated our results on the MOT challenge benchmarks and achieved state-of-the-art results on the MOT Challenge 2017. [ABSTRACT FROM AUTHOR]

Tracking multiple people in crowds is a fundamental and essential task in the multimedia field. It is often hindered by difficulties, such as dynamic occlusion between objects, cluttered background, and abrupt illumination changes. To respond to this need, in this paper, we combine deep and depth to build a stereo tracking system for crowds. The core of the system is the fusion of the advantages of deep learning and depth information, which is exploited to achieve object segmentation and improve the multiobject tracking performance in severe occlusion. More specifically, first, to obtain more accurate detection observations in the tracking system, we present a novel object-level segmentation method. This method combines the effective detection results of deep learning with depth information to obtain precise object segmentation results. Then, we integrate the segmentation results and three-dimensional (3-D) information to extract 2-D and 3-D characteristics to represent the target, and design three similarity models to realize a stereo tracking method through data association in crowds. Finally, we build a diverse stereo dataset including various challenging indoor and outdoor scenes. The comprehensive experiments verify the effective and robust tracking performance of our system in various scenarios, and the system has rich output results including segmentation results, target distance, and tracking results. Moreover, the qualitative and quantitative comparison results show that the proposed algorithm not only has good object segmentation performance but also improves the tracking performance of completely and partially occluded objects, which is superior to the tested state-of-the-art tracking approaches. [ABSTRACT FROM AUTHOR]

Aligning workers and groups with workspaces in advance to enable peak performance and ensure safety is the essence of workspace planning. There are plans, while there have been few methods for examining these plans. We argue that capturing and visualizing actual dynamic workspaces is fundamental to plan examination. This paper describes an initial effort on integrating the latest computer vision methods to implement automatic detection and visualization of dynamic workspaces of workers on foot. To this end, object detection, multiple object tracking, and action recognition are adopted to collect two types of action data: action classes and action locations. A density-based spatial clustering algorithm is used to reason dynamic workspaces based on the action data. We evaluated each method of the integrated system and found that they have achieved the comparable performance in our envisaged settings with the original methods in their general settings. We also presented two demonstrations of the system. The research results represent an initial step towards developing a new management capability by capturing dynamic workspaces. • This paper introduces a vision-based system for detecting and visualizing dynamic workspaces. • The system represents an initial step towards developing a new management capability. • The system integrates the latest vision technologies to detect and track workers and recognize their actions, concurrently. • The system uses a density-based clustering method to reason dynamic workspaces. • Dynamic workspaces could be used to improve labor productivity and site safety. [ABSTRACT FROM AUTHOR]

Surface features can be used during multiple object tracking (MOT). Previous studies suggested that surface features might be stored in visual working memory to assist object tracking, and attentive tracking and visual working memory share common attentional resources. However, it is still unknown whether features of both the target and distractor sets will be stored, or features of the target and distractor sets are processed differently. Moreover, how feature distinctiveness and similarity between the target and distractor sets affect tracking and allocation of attentional resources are still not clear. First, we manipulated the colour complexity of the target set (CT) and the colour complexity of the distractor set (CD), respectively, in two experiments, where colours of the target and distractor sets were always distinct, to test their effects on tracking performance. If features of the target and distractor sets are stored, manipulating feature complexity of the target and distractor sets would significantly affect tracking performance. Second, this study tested whether tracking performance was affected by different levels of distinctiveness between the target and distractor sets (DTD) and explored how distinctiveness affected tracking and allocation of attentional resources. Results showed that DTD and CT significantly affect tracking performance and allocation of attentional resources, but not CD. These results indicated that when targets and distractors have distinct features, only the surface features of the targets are maintained in visual working memory. And when targets have the same colour with the distractors, they are more difficult and consume more attentional resources to track. [ABSTRACT FROM AUTHOR]

In standard multiple object tracking (MOT) tasks the relative importance of the targets being tracked is equal. This is atypical of everyday situations in which an individual may need to prioritize one target relative to another and so allocate attention unequally. We report three experiments that examined whether participants could unequally split attention using a modified MOT task in which target priority was manipulated. Specifically, we examined the effect of priority on participants' magnitude of error and used a distribution mixture analysis to investigate how priority affected both participants' probability of losing an item and tracking precision. Experiment 1 (trajectory tracking) revealed a higher magnitude of error and higher proportion of guessing for low- compared with high-priority targets. Experiments 2 (trajectory tracking) and 3 (position tracking) examined how fine-grained this ability is by manipulating target priority at finer increments. In line with Experiment 1, results from both these experiments indicated that participants could split attention unequally. There was some evidence that participants could allocate attention unequally at fine increments, but this was less conclusive. Taken together, these experiments demonstrate participants' ability to distribute attention unequally across multiple moving objects but suggest some limitation with the flexibility of attention allocation. [ABSTRACT FROM AUTHOR]

In this paper, a novel intuitionistic fuzzy clustering algorithm based on feature selection (IFC-FS) for multiple object tracking is proposed. In the proposed algorithm, the neighborhood rough set is used to achieve the adaptive selection of the multiple object features of visual objects, which are applied to calculate the distance similarity measure between the objects and the observations. At the same time, in order to incorporate the local information of objects into the intuitionistic fuzzy clustering, the local information distances between objects and observations are estimated by using the optimal subpattern assignment metric based on the reference topology set, and a new intuitionistic fuzzy clustering based on maximum entropy principle is proposed by using the new similarity distance measure. Finally, the association probabilities among the objects and the observations are reconstructed by utilizing the intuitionistic fuzzy membership degrees. The experimental results show that the proposed algorithm can effectively improve the estimated accuracy and robustness of the association probabilities between the objects and the observations, and have the ability to track accurately multiple objects in the complex background and long-time occlusion environment. [ABSTRACT FROM AUTHOR]

We describe an oscillatory neural network designed as a system of generalized phase oscillators with a central element. It is shown that a winner-take-all principle can be realized in this system in terms of the competition of peripheral oscillators for the synchronization with a central oscillator. Several examples illustrate how this network can be used for the simulation of various cognitive functions: consecutive selection of objects in the image, visual search, and multiple object tracking. [ABSTRACT FROM AUTHOR]

In recent years, there has been an increasing growth of using vision-based systems for tracking the players in team sports to evaluate and enhance their performance. Player tracking using vision systems is a very challenging task due to the nature of sports games, which includes severe and frequent interactions (e.g. occlusions) between the players. Additionally, these vision systems have high computational demands since they require processing of a huge amount of video data based on the utilization of multiple cameras with high resolution and high frame rate. As a result, most of the existing systems based on general-purpose computers are not able to perform online real-time player tracking, but track the players offline using pre-recorded video files, limiting, e.g., direct feedback on the player performance during the game. In this paper, we present a reconfigurable system to track the players in indoor sports automatically and without user interaction. The proposed system performs real-time processing of the incoming video streams from the cameras, achieving online player tracking. The teams are identified, and the players' positions are detected based on the colors of their jerseys. FPGA technology is used to handle the compute-intensive vision processing tasks by implementing the video acquisition, video preprocessing, player segmentation, and team identification & player detection modules in hardware, realizing an online real-time system. While the pixel processing is performed in the FPGA, the less compute-intensive player tracking is performed on a general purpose computer. The maximum achieved frame rate for the FPGA implementation is 96.7 fps using a mature Xilinx Virtex-4 FPGA, and can be increased to 136.4 fps using a Xilinx Virtex-7 device. The Player tracking requires an average time of 2.5 ms per frame in the host-PC. As a result, the proposed reconfigurable system supports a maximum frame rate of 78.9 fps using two cameras with a resolution of 1392 × 1040 pixels each. Our results show that the achieved average precision and recall for player detection are up to 84.02% and 96.6%, respectively. Including player tracking, the achieved average precision and recall are up to 94.85% and 94.72%, respectively. Using the proposed FPGA implementation, a speedup by a factor of 15.2 is achieved compared to an OpenCV-based software implementation on a PC equipped with a 2.93 GHz Intel i7-870 CPU. [ABSTRACT FROM AUTHOR]

The probability hypothesis density (PHD) filter is a method for tracking multiple target objects based on unlabeled detections. However, as the PHD filter employs a first-order approximation of random finite sets, it does not provide track labels, i.e., targets of consecutive time steps are not associated with each other. In this paper, an intuitive and efficient labeling strategy on top of the extended target PHD filter is proposed. The approach is based on solving a network flow problem and makes use of the Wasserstein metric to account for the spatial extent of the objects. The resulting tracker is evaluated with laser scanner data from two traffic scenarios. [ABSTRACT FROM AUTHOR]

The ability to track multiple objects is important for daily life activities such as driving, but it is subject to some restrictions. One limitation concerns the hemifields in which objects move. A previous study showed that when subjects were restricted to the use of one hemifield, both the maximum number of tracked objects and the tracking accuracy were lower than when they were permitted to use both hemifields. However, daily life involves many tracked objects moving between hemifields. In this study, we investigated the effects of such hemifield crossings on behavioral performance (Behavioral experiment) and on the amplitudes and phase synchronization of steady-state visual evoked potentials (SSVEPs) (SSVEP experiment) by comparing the Within condition, in which tracked objects moved within their respective hemifields, and the Crossover condition, in which tracked objects moved between hemifields. In the Behavioral experiment, tracking performance was worse under the Crossover condition than under the Within condition. In the SSVEP experiment, SSVEP amplitudes for target and distractor frequencies differed under the Within condition but did not differ under the Crossover condition. However, phase synchronization between the left and right hemifields exhibited the opposite trend. This study provides evidence that attention to objects moving between hemifields is suppressed relative to attention to objects moving within hemifields and that Crossover tracking diminishes attentional modulation at an early sensory processing level while modulating interhemispheric functional connectivity. • We investigated the effects of hemifield crossings while tracking multiple objects. • We analyzed behavioral performance and SSVEPs. • Attention to objects moving between hemifields is suppressed. • Crossover tracking modulated interhemispheric functional connectivity. [ABSTRACT FROM AUTHOR]