Search

Objects moving at high speed along complex trajectories often appear in videos, especially videos of sports. Such objects travel a considerable distance during exposure time of a single frame, and therefore, their position in the frame is not well defined. They appear as semi-transparent streaks due to the motion blur and cannot be reliably tracked by general trackers. We propose a novel approach called Tracking by Deblatting based on the observation that motion blur is directly related to the intra-frame trajectory of an object. Blur is estimated by solving two intertwined inverse problems, blind deblurring and image matting, which we call deblatting. By postprocessing, non-causal Tracking by Deblatting estimates continuous, complete, and accurate object trajectories for the whole sequence. Tracked objects are precisely localized with higher temporal resolution than by conventional trackers. Energy minimization by dynamic programming is used to detect abrupt changes of motion, called bounces. High-order polynomials are then fitted to smooth trajectory segments between bounces. The output is a continuous trajectory function that assigns location for every real-valued time stamp from zero to the number of frames. The proposed algorithm was evaluated on a newly created dataset of videos from a high-speed camera using a novel Trajectory-IoU metric that generalizes the traditional Intersection over Union and measures the accuracy of the intra-frame trajectory. The proposed method outperforms the baselines both in recall and trajectory accuracy. Additionally, we show that from the trajectory function precise physical calculations are possible, such as radius, gravity, and sub-frame object velocity. Velocity estimation is compared to the high-speed camera measurements and radars. Results show high performance of the proposed method in terms of Trajectory-IoU, recall, and velocity estimation.

This paper investigates the use of supervised Deep Learning (DL) networks to process sonar images for underwater navigation. State-of-the-art DL techniques for micro-navigation using sequences of optical images have been adapted to work with sonar images. Specifically, the DL networks estimate the Forward-Looking Sonar (FLS) motion in three degrees of freedom corresponding to $x$ - and $y$ -translation and rotation around $z$ -axis. The state-of-the-art DL architectures and a proposed new architecture are investigated for motion estimation. They are trained using images generated by a FLS simulator. The data sets are made using pairs of consecutive images associated with labels that represent the motion of the sonar platform between images. The results show the effectiveness of using the DL architectures, which can provide millimeter accuracy for translation motion and below 0.1° for rotation motion between two consecutive sonar images. Examples of trajectory estimation and mosaic building using simulated and real sonar images are also presented.

Simultaneous localization and mapping (SLAM) is an important tool that enables autonomous navigation of mobile robots through unknown environments. As the name SLAM suggests, it is important to obtain a correct representation of the environment and estimate a correct trajectory of the robot poses in the map. Dominant state-of-the-art approaches solve the pose estimation problem using graph optimization techniques based on the least squares minimization method. Among the most popular approaches are libraries such as g$^2$o, Ceres, GTSAM and SE-Sync. The aim of this paper is to describe these approaches in a unified manner and to evaluate them on an array of publicly available synthetic and real-world pose graph datasets. In the evaluation experiments, the computation time and the value of the objective function of the four optimization libraries are analyzed.

To achieve human-level object manipulation capability, a robot must be able to handle objects not only with prehensile manipulation, such as pick-and-place, but also with nonprehensile manipulation. To study nonprehensile manipulation, we studied robotic batting, a primitive form of nonprehensile manipulation. Batting is a challenging research area because it requires sophisticated and fast manipulation of moving objects and requires considerable improvement. In this paper, we designed a batting system for dynamic manipulation of a moving ball and proposed several algorithms to improve the task performance of batting. To improve the recognition accuracy of the ball, we proposed a circle-fitting method that complements color segmentation. This method enabled robust ball recognition against illumination. To accurately estimate the trajectory of the recognized ball, weighted least-squares regression considering the accuracy according to the distance of a stereo vision sensor was used for trajectory estimation, which enabled more accurate and faster trajectory estimation of the ball. Further, we analyzed the factors influencing the success rate of ball direction control and applied a constant posture control method to improve the success rate. Through the proposed methods, the ball direction control performance is improved.

The rapid developments in the field of Artificial Intelligence are bringing enhancements in the area of intelligent transport systems by overcoming the challenges of safety concerns. Traffic surveillance systems based on CCTV cameras can help us to achieve safe and sustainable transport systems. Trajectory estimation of vehicles is an important part of traffic surveillance systems and self-driving cars. The task is challenging due to the variations in illumination intensities, object sizes and real-time detection. We propose tracking by detection based trajectory estimation pipeline which consists of two stages: The first stage is the detection and localization of vehicles and the second stage is building associations in bounding boxes and track the associated bounding boxes. We analyze the performance of the Mask RCNN benchmark and YOLOv3 on the UA-DETRAC dataset and evaluate certain metrics like Intersection over Union, Precision-Recall curve, and Mean Average Precision. Experiments show that Mask RCNN Benchmark outperforms YOLOv3 in terms of accuracy. SORT tracker is applied on detected bounding boxes to estimate trajectories. The tracker is evaluated using mean absolute error. We demonstrate that the developed technique works successfully in crowded and crossroad scenarios.

International audience; In Virtual Reality, visuo-haptic illusions such as hand redirection introduce a discrepancy between the user's hand and its virtual avatar. This visual shift can be used, for instance, to provide multiple virtual haptic objects through a single physical proxy object. This lowcost approach improves the sense of presence, however, it is unclear how these illusions impact the hand trajectory and if there is a relationship between trajectory and the detection of illusion. In this paper, we present an empirical model predicting the hand trajectory as a function of the redirection. It relies on a cubic Bézier curve with 4 control points. We conduct a two alternative forced choice (2AFC) experiment to calibrate and validate our model. Results show that (1) our model predicts well the hand trajectory of each individual using a single parameter; (2) the hand trajectory better explains the detection of the illusion than the amplitude of the redirection alone; (3) a user specific calibration allows to predict per-user redirected trajectories and detection probabilities. Our findings provide a better understanding of visuo-haptic illusions and how they impact the user's movements. As such they may provide foundations to design novel interaction techniques, e.g. interacting in a scene with multiple physical obstacles.

Simultaneous localization and mapping (SLAM) of a monocular projective camera installed on an unmanned aerial vehicle (UAV) is a challenging task in photogrammetry, computer vision, and robotics. This paper presents a novel real-time monocular SLAM solution for UAV applications. It is based on two steps: consecutive construction of the UAV path, and adjacent strip connection. Consecutive construction rapidly estimates the UAV path by sequentially connecting incoming images to a network of connected images. A multilevel pyramid matching is proposed for this step that contains a sub-window matching using high-resolution images. The sub-window matching increases the frequency of tie points by propagating locations of matched sub-windows that leads to a list of high-frequency tie points while keeping the execution time relatively low. A sparse bundle block adjustment (BBA) is employed to optimize the initial path by considering nuisance parameters. System calibration parameters with respect to global navigation satellite system (GNSS) and inertial navigation system (INS) are optionally considered in the BBA model for direct georeferencing. Ground control points and checkpoints are optionally included in the model for georeferencing and quality control. Adjacent strip connection is enabled by an overlap analysis to further improve connectivity of local networks. A novel angular parametrization based on spherical rotation coordinate system is presented to address the gimbal lock singularity of BBA. Our results suggest that the proposed scheme is a precise real-time monocular SLAM solution for a UAV.

Driver situation awareness is critical for safety. In this paper, we propose a fast, accurate method for obtaining real-time situation awareness using a single type of sensor: monocular cameras. The system tracks the host vehicle’s trajectory using sparse optical flow and tracks vehicles in the surrounding environment using convolutional neural networks. Optical flow is used to measure the linear and angular velocity of the host vehicle. The convolutional neural networks are used to measure target vehicles’ positions relative to the host vehicle using image-based detections. Finally, the system fuses host and target vehicle trajectories in the world coordinate system using the velocity of the host vehicle and the target vehicles’ relative positions with the aid of an Extended Kalman Filter (EKF). We implement and test our model quantitatively in simulation and qualitatively on real-world test video. The results show that the algorithm is superior to state-of-the-art sequential state estimation methods such as visual SLAM in performing accurate global localization and trajectory estimation for host and target vehicles.

Estimating the number of people in rooms and zones within commercial buildings are gaining enormous attention for facilitating various domain applications. However, the deployment of state-of-Art counting sensors such as camera technologies can be economically in-viable for individual rooms or zones in large commercial and public buildings. Such sensors are also known to be highly intrusive within building deployments. In this paper, we propose a multi-modal fusion method that leverages the accuracy of camera technologies for estimating building-level counts and the non-intrusive and scalability of wireless fidelity (WiFi) trajectory data to estimate room-level counts. This multi-modal fusion method disaggregates the obtained building-level counts by applying a series of data cleaning methods and a two-step probabilistic method. We evaluate the disaggregation method with datasets from a large teaching building, and we benchmark its performance with a state-of-Art estimation algorithm and count estimates from raw WiFi trajectories. The obtained evaluation results highlight that the disaggregation algorithm outperforms other estimation methods by a minimum ratio of 35% for all room cases using the Normalized Root Mean Squared Error (NRMSE) metric.

Robotic catching of thrown objects is one of the common robotic tasks, which is explored in several works. This task includes subtask of tracking and forecasting the trajectory of the thrown object. Here we propose an algorithm for estimating future trajectory based on video signal from two cameras. Most of existing implementations use deterministic trajectory prediction and several are based on machine learning. We propose a combined forecasting algorithm where the deterministic motion model for each trajectory is generated via the genetic programming algorithm. Genetic programming is implemented on C++ with use of CUDA library and executed in parallel way on the graphical processing unit. Parallel execution allow genetic programming in real time. Numerical experiments with real trajectories of the thrown tennis ball show that the algorithm can forecast the trajectory accurately. © 2016 Research work is supported by Russian Fund for Basic Research, grant #16 -07-00243 .

We consider the problem of estimating human pose and trajectory by an aerial robot with a monocular camera in near real time. We present a preliminary solution whose distinguishing feature is a dynamic classifier selection architecture. In our solution, each video frame is corrected for perspective using projective transformation. Then, two alternative feature sets are used: (i) Histogram of Oriented Gradients (HOG) of the silhouette, (ii) Convolutional Neural Network (CNN) features of the RGB image. The features (HOG or CNN) are classified using a dynamic classifier. A class is defined as a pose-viewpoint pair, and a total of 64 classes are defined to represent a forward walking and turning gait sequence. Our solution provides three main advantages: (i) Classification is efficient due to dynamic selection (4-class vs. 64-class classification). (ii) Classification errors are confined to neighbors of the true view-points. (iii) The robust temporal relationship between poses is used to resolve the left-right ambiguities of human silhouettes. Experiments conducted on both fronto-parallel videos and aerial videos confirm our solution can achieve accurate pose and trajectory estimation for both scenarios. We found using HOG features provides higher accuracy than using CNN features. For example, applying the HOG-based variant of our scheme to the 'walking on a figure 8-shaped path' dataset (1652 frames) achieved estimation accuracies of 99.6% for viewpoints and 96.2% for number of poses., Comment: For associated dataset, see https://asankagp.github.io/aerialgaitdataset/

This paper proposes a composite Bayesian filtering approach for unmanned aerial vehicle trajectory estimation in cluttered environments. More specifically, a complete model for the measurement likelihood function of all measurements, including target-generated observation and false alarms, is derived based on the random finite set theory. To accommodate several different manoeuvre modes and system state constraints, a recursive multiple model Bayesian filtering algorithm and its corresponding Sequential Monte Carlo implementation are established. Compared with classical approaches, the proposed method addresses the problem of measurement uncertainty without any data associations. Numerical simulations for estimating an unmanned aerial vehicle trajectory generated by generalised proportional navigation guidance law clearly demonstrate the effectiveness of the proposed formulation.

This paper addresses the issue of estimating underwater vehicle trajectories using gyro-Doppler (body-fixed velocities) and acoustic positioning signals (earth-fixed positions). The approach consists of diffusion-based observers processing a whole trajectory segment at a time, allowing the consideration of important practical problems such as different information update rates, outages, and outliers in a very simple framework. Results of contraction theory are used to prove that the observers are convergent, i.e., stable in the incremental sense. Simulation and experimental results are presented to illustrate the potential of application of the method.

In this paper, we describe a collection of algorithms that are used to provide motion trajectory estimates from atherosclerotic plaque ultrasound videos. Our approach is based on the use of four different optical flow methods to estimate motion vectors (Horn and Schunk, Lucas, Nagel and Uras). To estimate the optimal motion estimation parameters, we perform hundreds of experiments on a Linux cluster, and further validate the results using synthetic simulations. Following motion estimation, we compute pixel motion trajectories over the plaque regions and vessel walls. Pixel trajectories are then used to assess plaque deformation. 836 840 Sponsors: IEEE Signal Processing Society Conference code: 72472 Cited By :4