Search

To improve the recognition model accuracy of crop disease leaves and locating diseased leaves, this paper proposes an improved Faster RCNN to detect healthy tomato leaves and four diseases: powdery mildew, blight, leaf mold fungus and ToMV. First, we use a depth residual network to replace VGG16 for image feature extraction so we can obtain deeper disease features. Second, the k-means clustering algorithm is used to cluster the bounding boxes. We improve the anchoring according to the clustering results. The improved anchor frame tends toward the real bounding box of the dataset. Finally, we carry out a k-means experiment with three kinds of different feature extraction networks. The experimental results show that the improved method for crop leaf disease detection had 2.71% higher recognition accuracy and a faster detection speed than the original Faster RCNN.

Due to the deficiency of prior information for online updating process, the tracking accuracy of fully-convolutional Siamese network (SiamFC) in complex scenes such as similar object interference, fast moving, and appearance change is not good. To solve the problem, a new object tracker based on a dynamic template updating strategy and the re-location mechanism based on the adaptive Kalman is proposed. To suppress the object interference and overcome the instability of fast-moving object tracking, an adaptive Kalman filter method is designed to change the selection method of search region and select the bounding box of the object closest to the predicted position. For the adaptation of appearance change, the high-confidence tracking results are fused with the initial template to dynamic update the template. Compared with traditional Kalman filter, the expectation of residual error for the adaptive Kalman filter method can be controlled in a low range by the adjustment of the gain online. The introduction of the adaptive Kalman based re-location mechanism improves the discriminative ability of SiamFC in interference scene. With the dynamic template updating strategy, the tracker obtains strong generalization capability to adapt to the appearance change of the tracking target. It is demonstrated that the proposed method performs real-time object tracking at the speed of 43fps and achieves competitive performance on OTB, VOT and TC128 datasets compared with other state-of-the-art trackers.

In logistics warehouse sorting, rubbish classification, and household services, scenarios exist in which rigid and soft objects are randomly piled together. In such situations, two major challenges arise in robotic picking tasks: the first is to distinguish rigid objects from soft objects, and the second is to grasp one object of each type at a time. In this study, we propose a novel robotic picking methodology for the grasping of objects mixed with towels. The proposed approach is based on a novel object detection method that can identify a rigid object placed in different directions using a rotational bounding box. Rigid objects can be separated from the mixed scene without object segmentation. Moreover, the grasping pose of a rigid object can be generated directly along its principal axis, without using a CAD model or specific pose detection method. The gripper opening width is determined according to the object size. Therefore, our method can detect whether other objects, particularly soft ones, exist around a rigid object. If no suitable grasping pose is available for the rigid objects, the grasping pose on a wrinkle of the towel is selected. The experiments demonstrate that our method can accomplish the picking task in scenes with mixed rigid and soft objects, thereby indicating its significance in robotic object detection and sorting.

Zero-shot learning aims to perform visual reasoning about unseen objects. In-line with the success of deep learning on object recognition problems, several end-to-end deep models for zero-shot recognition have been proposed in the literature. These models are successful in predicting a single unseen label given an input image but do not scale to cases where multiple unseen objects are present. Here, we focus on the challenging problem of zero-shot image tagging, where multiple labels are assigned to an image, that may relate to objects, attributes, actions, events, and scene type. Discovery of these scene concepts requires the ability to process multi-scale information. To encompass global as well as local image details, we propose an automatic approach to locate relevant image patches and model image tagging within the Multiple Instance Learning (MIL) framework. To the best of our knowledge, we propose the first end-to-end trainable deep MIL framework for the multi-label zero-shot tagging problem. We explore several alternatives for instance-level evidence aggregation and perform an extensive ablation study to identify the optimal pooling strategy. Due to its novel design, the proposed framework has several interesting features: 1) unlike previous deep MIL models, it does not use any off-line procedure (e.g., Selective Search or EdgeBoxes) for bag generation. 2) During test time, it can process any number of unseen labels given their semantic embedding vectors. 3) Using only image-level seen labels as weak annotation, it can produce a localized bounding box for each predicted label. We experiment with the large-scale NUS-WIDE and MS-COCO datasets and achieve superior performance across conventional, zero-shot, and generalized zero-shot tagging tasks.

Balancing the trade-off between real-time performance and accuracy in object tracking is a major challenge. In this paper, a novel dynamic policy gradient Agent-Environment architecture with Siamese network (DP-Siam) is proposed to train the tracker to increase the accuracy and the expected average overlap while performing in real-time. DP-Siam is trained offline with reinforcement learning to produce a continuous action that predicts the optimal object location. DP-Siam has a novel architecture that consists of three networks: an Agent network to predict the optimal state (bounding box) of the object being tracked, an Environment network to get the Q-value during the offline training phase to minimize the error of the loss function, and a Siamese network to produce a heat-map. During online tracking, the Environment network acts as a verifier to the Agent network action. Extensive experiments are performed on six widely used benchmarks: OTB2013, OTB50, OTB100, VOT2015, VOT2016 and VOT2018. The results show that DP-Siam significantly outperforms the current state-of-the-art trackers.

The more sensory channels are equipped in a virtual assembly system, the more real users feel in the whole system, however, most of the existing virtual assembly systems are based on the natural interaction method of one or two sensory channels. Thus, this paper proposes a novel virtual assembly system integrating multi-sensory channels, including gesture interaction, Chinese speech interaction, tactile interaction, 3D display and real-time display of real environment pictures in the virtual environment. To improve the operability of the virtual environment, we analyze the parallel virtual assembly sequence on the basis of two-hand interaction, and the assembly priority is prompted based on UI interface. For ease of operation, we present a method of viewpoint control based on gesture interaction and the coordinate threshold of spatial position. A hierarchical bounding box collision detection algorithm based on volume difference is proposed to improve the efficiency of collision feedback and collision avoidance. In addition, the power equipment models are exhibited in the virtual scene, as the exhibits of the virtual roaming process. Finally, to evaluate the training effect of this system, a comparative experiment is designed to compare the participants’ experience and the effect of assembly training. The experimental results show that the virtual assembly system of natural interaction with multi-sensory channels is flexible and immersive.

A reactive collision avoidance algorithm is proposed to enable safe autonomous flight in the presence of multiple dynamic obstacles. A position-controlled hexacopter equipped with a visual sensor obtaining obstacle information is considered as an Unmanned Aerial Vehicle (UAV) platform. The proposed method centers on the concept of bounding tube which intrinsically extends the static bounding box to incorporate forthcoming movement of the obstacles into the collision avoidance framework. The processing pipeline consists of separate components for each of the sequential tasks in obstacle sensing and tracking. Computation of a spherical bounding box for each obstacle is followed by discrete-time Kalman filtering for prediction of obstacle trajectory to detect potential collision. If the current course of UAV turns out highly likely to end in collision with any of the obstacles, the vehicle steers to an aiming point chosen from among the bundle of candidates produced by constructing a bounding tube that takes account of predicted obstacle motion. The bounding-tube-based aiming point generation extends seamlessly to the case with multiple moving obstacles through running in series with multi-obstacle track management that combines hierarchical clustering of sensory data points for obstacle identification and a simple geometric method for data association. Numerical simulations are conducted to verify the performance of the proposed collision avoidance algorithm.

Sequential visual task usually requires to pay attention to its current interested object conditional on its previous observations. Different from popular soft attention mechanism, we propose a new attention framework by introducing a novel conditional global feature which represents the weak feature descriptor of the current focused object. Specifically, for a standard CNN (Convolutional Neural Network) pipeline, the convolutional layers with different receptive fields are used to produce the attention maps by measuring how the convolutional features align to the conditional global feature . The conditional global feature can be generated by different recurrent structure according to different visual tasks, such as a simple recurrent neural network for multiple objects recognition, or a moderate complex language model for image captioning and weakly supervised multiple objects segmentation. Experiments show that our proposed conditional attention model achieves the best performance on the SVHN (Street View House Numbers) dataset with / without extra bounding box; for image caption, our attention model generates better scores than the popular soft attention model; and for weakly supervised multiple objects segmentation, by simply describing a sentence for each image, our attention model can segment the salient regions corresponding to the meaningful noun words.

Instance segmentation is typically based on an object detection framework. Semantic segmentation is conducted on the bounding boxes that are returned by detectors. NMS (non-maximum suppression) is a common post-processing operation in instance segmentation and object detection tasks. It is typically used after bounding box regression to eliminate redundant bounding boxes. The evaluation criteria for object detection require that the bounding box be as close as possible to the ground truth, but they do not emphasize the integrity of the included object. However, sometimes the bounding boxes cannot contain the complete objects, and the parts beyond the bounding boxes cannot be correctly predicted in the subsequent semantic segmentation. To solve this problem, we propose the Syncretic-NMS algorithm. The algorithm takes traditional NMS as the first step and processes the bounding boxes obtained by traditional NMS, judges the neighboring bounding boxes of each bounding box, and combines the neighboring boxes that are strongly correlated with the corresponding bounding boxes. The coordinates of the merged box are the four coordinate extremes of the bounding box and the highly relevant neighboring box. The neighboring box with strong correlation is merged with the corresponding bounding box. Based on an analysis of the influences of corresponding factors, the criteria for correlation judgment are specified. Experimental results on the MS COCO dataset demonstrate that Syncretic-NMS can steadily increase the accuracy of instance segmentation, while experimental results on the Cityscapes dataset prove that the algorithm can adapt to application scenario changes. The computational complexity of Syncretic-NMS is the same as that of traditional NMS. Syncretic-NMS is easy to implement, requires no additional training, and can be easily integrated into the available instance segmentation framework.

Static environment is a prerequisite for most existing vision-based SLAM (simultaneous localization and mapping) systems to work properly, which greatly limits the use of SLAM in real-world environments. The quality of the global point cloud map constructed by the SLAM system in a dynamic environment is related to the camera pose estimation and the removal of noise blocks in the local point cloud maps. Most dynamic SLAM systems mainly improve the accuracy of camera localization, but rarely study on noise blocks removal. In this paper, we proposed a novel semantic SLAM system with a more accurate point cloud map in dynamic environments. We obtained the masks and bounding boxes of the dynamic objects in the images by BlitzNet. The mask of a dynamic object was extended by analyzing the depth statistical information of the mask in the bounding box. The islands generated by the residual information of dynamic objects were removed by a morphological operation after geometric segmentation. With the bounding boxes, the images can be quickly divided into environment regions and dynamic regions, so the depth-stable matching points in the environment regions are used to construct epipolar constraints to locate the static matching points in the dynamic regions. In order to verify the preference of our proposed SLAM system, we conduct the experiments on the TUM RGB-D datasets. Compared with the state-of-the-art dynamic SLAM systems, the global point cloud map constructed by our system is the best.

Small-scale pedestrian detection and occluded pedestrian detection are two challenging tasks. However, most state-of-the-art methods merely handle one single task each time, thus giving rise to relatively poor performance when the two tasks, in practice, are required simultaneously. In this paper, it is found that small-scale pedestrian detection and occluded pedestrian detection actually have a common problem, i.e., an inaccurate location problem. Therefore, solving this problem enables to improve the performance of both tasks. To this end, we pay more attention to the predicted bounding box with worse location precision and extract more contextual information around objects, where two modules ( i.e., location bootstrap and semantic transition) are proposed. The location bootstrap is used to reweight regression loss, where the loss of the predicted bounding box far from the corresponding ground-truth is upweighted and the loss of the predicted bounding box near the corresponding ground-truth is downweighted. Additionally, the semantic transition adds more contextual information and relieves semantic inconsistency of the skip-layer fusion. Since the location bootstrap is not used at the test stage and the semantic transition is lightweight, the proposed method does not add many extra computational costs during inference. Experiments on the challenging CityPersons and Caltech datasets show that the proposed method outperforms the state-of-the-art methods on the small-scale pedestrians and occluded pedestrians ( e.g., 5.20% and 4.73% improvements on the Caltech).

Trackers based on Siamese networks show great potential in tracking accuracy and speed. However, it is still challenging to adapt offline training model to online tracking. In this paper, a Siamese based tracker (SCRPN-CISA) is proposed, which integrates three attention mechanisms and a novel Cascaded Region Proposal Networks (RPN) architecture, for improving the feature extraction ability, adaptability and discrimination ability in complex scenes. Firstly, the deep network VGG-Net-D is adopted as the backbone network in the Siamese framework to increase the feature extraction capability. Then, a Channel-Interconnection-Spatial Attention module is constructed to enhance the adaptive and discriminative capability of the model. Next, a Deconvolution Adjust Block is built to fusion cross-layer features. Finally, a Three-Layer Cascaded RPN is conceived to acquire the foreground-background classification and bounding box regression by correlation calculation, and moreover, a proposal region screening strategy is presented to obtain more accurate tracking results. Experiments on OTB-2015, UAV123, VOT2016, and VOT2019 benchmarks demonstrate that, the proposed tracker (SCRPN-CISA) achieves competitive performance compared with the state-of-the-art trackers.

Real-time 6DOF (6 Degree of Freedom) pose estimation of an uncooperative spacecraft is an important part of proximity operations, e.g., space debris removal, spacecraft rendezvous and docking, on-orbit servicing, etc. In this article, a novel efficient deep learning based approach is proposed to estimate the 6DOF pose of uncooperative spacecraft using monocular-vision measurement. Firstly, we introduce a new lightweight YOLO-liked CNN to detect spacecraft and predict 2D locations of the projected keypoints of a prior reconstructed 3D model in real-time. Then, we design two novel models for predicting the bounding box (bbox) reliability scores and the probability of keypoints existence. The two models not only significantly reduce the false positive, but also speed up convergence. Finally, the 6DOF pose is estimated and refined using Perspective-n-Point and geometric optimizer. Results demonstrate that the proposed approach achieves 73.2% average precision and 77.6% average recall for spacecraft detection on the SPEED dataset after only 200 training epochs. For the pose estimation task, the mean rotational error is 0.6812°, and the mean translation error is 0.0320m. The proposed approach achieves competitive pose estimation performance and extreme lightweight ( $\sim ~0.89$ million learnable weights in total) on the SPEED dataset while being efficient for real-time applications.

Popular virtual reality systems today allow us to experience highly immersive applications in which virtual objects are realistically perceived via stereoscopic displays and can be directly manipulated based on hand-eye coordination in a very similar way as in the real world. However, the insufficiency of sensory feedback as well as the limited degrees-of-freedom of input motion still hinders precise and elaborate manipulation in virtual reality. Aiming at more precise 3D manipulation, we present a new method of extending the user’s spatial perception ability with the ‘virtual mirrors’ , which expose the hidden spatial information of given virtual scenes to the user. The movement of a virtual mirror is automatically controlled by solving an optimization problem iteratively, in which the objective function prefers the placement of the mirror that can highlight the spatial relationship between the manipulated object and the object nearest to it. The optimization process is handled efficiently for each time step based on our method for finding the closest gap between any two objects based on the OBB (oriented bounding box) trees and our sampling-based approximate approach to the optimization problem. The usefulness of our method is demonstrated by several pilot applications under various usage scenarios, such as assembling construction toys and solving 3D dissection puzzles. The quantitative results of our user study show that the virtual mirror is very helpful in increasing the precision in 3D manipulation tasks in virtual reality.

Notice of Violation of IEEE Publication Principles "Deep Learning Assisted Image Interactive Framework for Brain Image Segmentation," by Y. Han and Z. Zhang, in IEEE Access, vol. 8, June 2020, pp. 117028-117035 After careful and considered review of the content and authorship of this paper by a duly constituted expert committee, this paper has been found to be in violation of IEEE’s Publication Principles. This paper contains significant portions of text from the papers cited below that were paraphrased without attribution. “DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation," by Guotai Wang, Maria A. Zuluaga, Wenqi Li, Rosalind Pratt, Premal A. Patel, Michael Aertsen, Tom Doel, Anna L. David, Jan Deprest, Sebastien Ourselin, and Tom Vercauteren in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 7, July 2019, pp. 1559-1572 "Interactive Medical Image Segmentation Using Deep Learning With Image-Specific Fine Tuning," by Guotai Wang, Wenqi Li, Maria A. Zuluaga, Rosalind Pratt, Premal A. Patel, Michael Aertsen, Tom Doel, Anna L. David, Jan Deprest, Sebastien Ourselin, and Tom Vercauteren in IEEE Transactions on Medical Imaging, vol. 37, no. 7, July 2018, pp. 1562-1573, Exacting medical imaging, surgical planning, and many others are very important to handle brain image segmentation. The Convolutional Neural Networks (CNN) has been developed by the efficient auto segmentation technology. In fact, the clinical outcomes are not appropriately specific and detailed. Nevertheless, the lack of sensitivity to images and lack of generality is reduced in traditionally invisible object classes. In this paper, Deep Learning Assisted Image Interactive Medical Image Segmentation (DL-IIMIS) is proposed to tackle these difficulties by including CNNs in the bounding box and scribble-based pipeline. To adapt a CNN model to one test frame, it is proposed that image fine tuning and geodesic transformations can be either unsupervised or supervised. In this frame, two applications are involved: 2-D multi-organ magnetic resonance (MR) segmentation, with only two types of training and 3-D segmentation within brain tumor center and in entire brain tumors with different MR sequences where only one MR sequence is reported. Compared with other algorithms, the proposed framework can output a better performance in brain image segmentation.

We revisit the binary splitting functionality used in decision trees to handle numerical attributes. Even if the true relationship between the class label and a few numerical attributes can be expressed directly (using a Boolean expression), resulting decision trees may appear quite large and complicated. In cases where interpretability is important, an increased computational effort on the splitting criteria that offers more compact trees might be worthwhile. We propose and empirically evaluate multidimensional splits, where a tree node may test for inclusion in a low-dimensional bounding box.

Flies carry pathogens that endanger the health of humans and animals. The color and shape of the fly species are very similar, which is difficult to recognize. This paper proposes a fly species recognition method based on improved RetinaNet and convolutional block attention module (CBAM). Firstly, the proposed method used ResNeXt101 as a feature extraction network, and the improved CBAM called Stochastic-CBAM was added. Then, we built a multi-scale feature pyramid through an improved feature pyramid network (FPN) and integrated multi-level feature information. Finally, the small full convolutional network (FCN) was used as the classification subnet and the bounding box regression subnet. The Kullback-Leibler (KL) loss replaced smooth L1 loss as a bounding box regression loss function for learning bounding box regression and positioning uncertainty at the same time. We experimentally compared the proposed method with other the state-of-the-art methods on the established dataset. Experimental results showed that the mean Average Precision (mAP) of this method reached 90.38%, which was better than the state-of-the-art methods. The average time to recognize a single image was 0.131s. This method has a good detection effect on the fly species recognition.

Object detectors typically use bounding box regressors to improve the accuracy of object localization. Currently, the two types of bounding box regression loss are $\ell _{n}$ -norm-based and intersection over union ( $IoU$ )-based. However, we found that these two types of losses have their drawbacks. First, for $\ell _{n}$ -norm-based loss, large-scale objects are more likely to obtain a larger penalty than the smaller ones when calculating localization errors, which will cause regression loss imbalance. Second, $\ell _{n}$ -norm-based loss has symmetry so that when the predicted bounding boxes are in some unique symmetrical relationships (i.e., Symmetric Trap), the regression loss remains unchanged. Third, for $IoU$ -based loss, the overlap area and the union area do not change as the shape or relative position of two bounding boxes changes in some cases(i.e., Area Maze). To address these problems, we propose the scale balanced loss( $\mathcal {L}_{SB}$ ), which is asymmetric, position-sensitive, and scale-invariant. First, in order to obtain the property of scale invariance, it is designed as a fraction to eliminate the scale information contained in the numerator and denominator. Second, by incorporating the Euclidean distance between different corner points instead of the area, $\mathcal {L}_{SB}$ is sensitive to the changes of coordinates of any corner point, so as to solve the area maze problem. Finally, by incorporating the diagonals of the overlap and the smallest enclosing rectangle, this fraction is no longer symmetric, thus solving the symmetry trap problem. To validate the proposed algorithm, we have replaced the $\ell _{n}$ -norm-based loss of YOLOv3 and SSD with $\mathcal {L}_{GIoU}$ and $\mathcal {L}_{SB}$ and evaluate their performance on Pascal Visual Object Classes and Microsoft Common Objects in Context benchmarks. The final results show that $\mathcal {L}_{SB}$ has improved their average precisions at different $IoU$ thresholds and scales. We envision that this regression loss can also improve the performance of other visual tasks.

Recent state-of-the-art methods on focus region detection (FRD) rely on deep convolutional networks trained with costly pixel-level annotations. In this study, we propose a FRD method that achieves competitive accuracies but only uses easily obtained bounding box annotations. Box-level tags provide important cues of focus regions but lose the boundary delineation of the transition area. A recurrent constraint network (RCN) is introduced for this challenge. In our static training, RCN is jointly trained with a fully convolutional network (FCN) through box-level supervision. The RCN can generate a detailed focus map to locate the boundary of the transition area effectively. In our dynamic training, we iterate between fine-tuning FCN and RCN with the generated pixel-level tags and generate finer new pixel-level tags. To boost the performance further, a guided conditional random field is developed to improve the quality of the generated pixel-level tags. To promote further study of the weakly supervised FRD methods, we construct a new dataset called FocusBox, which consists of 5000 challenging images with bounding box-level labels. Experimental results on existing datasets demonstrate that our method not only yields comparable results than fully supervised counterparts but also achieves a faster speed.

This paper introduces a novel method for real-time 2D hand pose estimation from monocular color images, which is named as SRHandNet. Existing methods can not time efficiently obtain appropriate results for small hand. Our key idea is to simultaneously regress the hand region of interests (RoIs) and hand keypoints for a given color image, and iteratively take the hand RoIs as feedback information for boosting the performance of hand keypoints estimation with a single encoder-decoder network architecture. Different from previous region proposal network (RPN), a new lightweight bounding box representation, which is called region map, is proposed. The proposed bounding box representation map together with hand keypoints heatmaps are combined into the unified multi-channel feature maps, which can be easily acquired with only one forward network inference and thus improve the runtime efficiency of the network. Our proposed SRHandNet can run at 40fps for hand bounding box detection and up to 30fps accurate hand keypoints estimation under the desktop environment without implementation optimization. Experiments demonstrate the effectiveness of the proposed method. State-of-the-art results are also achieved out competing all recent methods.

The state-of-the-art trackers using deep learning technology have little special strategy to gain the bounding box well when the target suffers drastic geometric deformation. In this paper, we take full use of the convolutional neural network (CNN) features of the deepest layer to represent the semantic feature model, and affine transformation to be as the space information model. A tracking method based on geometrical transformation region CNN is proposed. Firstly, affine transformation is applied to predict possible locations of a target, and the candidate bounding boxes obtained by affine transformation sampling can locate the possible geometric regions of the target more effectively before extracting features from CNN. Furthermore, RoI pooling with different sizes and shapes are designed to describe the geometric deformation region of the target. Then, multi-tasks loss function including the affine transformation regression is designed to refine the affine bounding box. Finally, the affine transformation NMS (Non-maximum suppression) is used to ensure the tracking bounding box having the largest IoU value. Extensive experimental results show that the proposed algorithm performs favorably against the compared methods in the public benchmarks.

We propose Mask SSD, an efficient and effective approach to address the challenging instance segmentation task. Based on a single-shot detector, Mask SSD detects all instances in an image and marks the pixels that belong to each instance. It consists of a detection subnetwork that predicts object categories and bounding box locations, and an instance-level segmentation subnetwork that generates the foreground mask for each instance. In the detection subnetwork, multi-scale and feedback features from different layers are used to better represent objects of various sizes and provide high-level semantic information. Then, we adopt an assistant classification network to guide per-class score prediction, which consists of objectness prior and category likelihood. The instance-level segmentation subnetwork outputs pixel-wise segmentation for each detection while providing the multi-scale and feedback features from different layers as input. These two subnetworks are jointly optimized by a multi-task loss function, which renders Mask SSD direct prediction on detection and segmentation results. We conduct extensive experiments on PASCAL VOC, SBD, and MS COCO datasets to evaluate the performance of Mask SSD. Experimental results verify that as compared with state-of-the-art approaches, our proposed method has a comparable precision with less speed overhead.

In recent years, the deep learning is applied to the field of traffic sign detection methods which achieves excellent performance. However, there are two main challenges in traffic sign detection to be solve urgently. For one thing, some traffic signs of small size are more difficult to detect than those of large size so that the small traffic signs are undetected. For another, some false signs are always detected because of interferences caused by the illumination variation, bad weather and some signs similar to the true traffic signs. Therefore, to solve the undetection and false detection, we first propose a cascaded R-CNN to obtain the multiscale features in pyramids. Each layer of the cascaded network except the first layer fuses the output bounding box of the previous one layer for joint training. This method contributes to the traffic sign detection. Then, we propose a multiscale attention method to obtain the weighted multiscale features by dot-product and softmax, which is summed to fine the features to highlight the traffic sign features and improve the accuracy of the traffic sign detection. Finally, we increase the number of difficult negative samples for dataset balance and data augmentation in the training to relieve the interference by complex environment and similar false traffic signs. The data augment method expands the German traffic sign training dataset by simulation of complex environment changes. We conduct numerous experiments to verify the effectiveness of our proposed algorithm. The accuracy and recall rate of our method are 98.7% and 90.5% in GTSDB, 99.7% and 83.62% in CCTSDB and 98.9% and 85.6% in Lisa dataset respectively.

Object tracking is one of the most fundamental and important fields in computer vision with a wide range of applications. Although great progress has been made in object tracking combined with detection, there is still enormous challenges in real-time applications and for the computer cannot effectively capture the temporal correlations of targets and background clutter. In order to improve the performance of tracking algorithms under complex unconstrained conditions, we propose a novel tracking framework based on adaptive detection, called adaptive detection tracking (ADT). First, we exploit the temporal correlation of the recurrent neural network to predict the target's motion direction and efficiently update the region of interest (RoI) in the narrow range of the next frame. Then, the algorithm utilizes the correlation filter to initialize the defined region of interest based on the threshold. If the Interaction of Union (IoU) of the predicted bounding box and the groundtruth bounding box is greater than the set threshold, the predicted bounding box will be directly output as the tracking results, whereas the detection is adaptively carried out in the determined RoI. Finally, the predicted bounding box refines the direction model as the input of the next frame to complete the whole tracking flow. Our proposed adaptive detection tracking mechanism can efficiently realize non-frame-by-frame adaptive detection with excellent tracking accuracy and is more robust in the unconstrained scenes, especially for occlusion. Comprehensive experiments demonstrate that our approach consistently achieves state-of-the-art results and runs in real-time on six large tracking benchmarks, including OTB100, VOT2016, VOT2017, TC128, UAV123 and LaSOT datasets.

A fusion image combining both anatomical and functional information obtained by registering medical images of two different modalities, Positron Emission Tomography (PET) and Computed Tomography (CT), is of great significance for medical image analysis and diagnosis. Medical image registration relies on similarity measure which is low between PET/CT image voxels and therefore PET/CT registration is a challenging task. To address this issue, this paper presents an unsupervised end-to-end method, DenseRegNet, for deformable 3D PET/CT image registration. The method consists of two stages: (1) predicting 3D displacement vector field (DVF); and (2) registering 3D image. In the 3D DVF prediction stage, a two-level similarity measure together with a deformation regularization is proposed as loss function to optimize network training.In the image registration stage, a resampler and a spatial transformer are utilized to obtain the registration results. In this paper, 663 pairs of Uptake Value (SUV) and Hounsfield Unit (Hu) patches of 106 patients, 227 pairs of SUV and Hu patches of 35 patients and 259 pairs of SUV and Hu patches of 35 patients are randomly selected as training, validation and test set, respectively. Normalized cross correlation (NCC), intersection over union (IoU) of liver bounding box and euclidean distance (ED) on landmark points are used to evaluate the registration results. Experiment results show that the proposed method, DenseRegNet, achieves the best results in terms of liver bounding box IoU and ED, and the second highest value of NCC. For a trained model, given a new pair of PET/CT images, the registration result can be obtained with only one forward calculation within 10 seconds. Through qualitative and quantitative analyses, we demonstrate that, compared with other deep learning registration models, the proposed DenseRegNet achieves improved results in the challenging deformable PET/CT registration task.

We investigate the localization of subtle yet discriminative parts for fine-grained image recognition. Based on the observation that such parts typically exist within a hierarchical structure (e.g., from a coarse-scale “head” to a fine-scale “eye” when recognizing bird species), we propose a novel progressive-attention convolutional neural network (PA-CNN) to progressively localize parts at multiple scales. The PA-CNN localizes parts in two steps, where a part proposal network (PPN) generates multiple local attention maps, and a part rectification network (PRN) learns part-specific features from each proposal and provides the PPN with refined part locations. This coupling of the PPN and PRN allows them to be optimized in a mutually reinforcing manner, leading to improved pinpointing of fine-grained parts. Moreover, the convolutional parameters for a PPN at a finer scale can be inherited from the PRN at a coarser scale, enabling a rich part hierarchy (e.g., eye and beak in a bird’s head) to be learned in a stacked fashion. Case studies show that PA-CNN can precisely identify parts without using bounding box/part annotations. In addition, quantitative evaluations demonstrate that PA-CNN yields state-of-the-art performance in three challenging fine-grained recognition tasks. i.e., CUB-2000–2011, FGVC-Aircraft, and Stanford Cars.

Great progress has been made in the field of pedestrian detection, but the following two problems have not yet been well addressed. One problem is the missed detection of small scale pedestrain as false negative failure, and the other one is confusion with anthropomorphic negative samples like vertical structures as false positive failure. In this paper, to tackle the above two problems, we use the light-field camera to capture pedestrian images for the following reasons: (i) the light-field camera can obtain multi-depth refocused images in a single exposure using one sensor, (ii) compared with 2D images, these refocused images can provide different key representations for different parts of the image. We further establish a light-field pedestrian dataset with 1766 images for pedestrian detection. A multi-focus detection network proposed in this work consists of multiple-branch detection models and takes multiple refocused images as inputs. In order to select the appropriate candidate proposal bounding box as final detection results, we design a cumulative probability selection (CPS) layer to combine each refocused image branch and accumulate the probability of each candidate neighboring proposal. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods on our light-field pedestrian dataset.

Due to the deteriorated quality of feature in the propagation process of the neural network, it may be hard for traditional detector to identify a small object by just utilizing information within one region proposal. To overcome the limitation of the traditional object detector, we proposed a graph based relation-aware network, to capture the relation information from labels, and images. The semantic relation network is proposed to mine the global semantic relation in labels, and the spatial relation network is proposed to capture the local spatial relation in images. The feature representation is further improved by aggregating the outputs of the two networks. Instead of directly disseminating visual features in the network, the relation-aware network explores more advanced feature information. Experiments on the PASCAL VOC, and MS COCO datasets demonstrate that key relation information significantly improve the performance of object detection with better ability to detect small objects, and reasonable bounding box. The results on COCO dataset demonstrate our method can detect objects robustly, increasing the detection performance of small objects from average precision, and average recall by 31.8%, and 32.3% respectively in performance relative to Faster R-CNN.

Direct regression and anchor are the two mainly effective and prevailing mechanisms in the paradigm of scene text detection. However, the use of direct regression-based methods may be challenging during optimization without the help of anchors as references. Unfortunately, the anchor-based methods always suffer from the careful design of the anchors, degrading the robustness to complex scenes. To address the above-mentioned problems, we propose a novel hidden anchor mechanism (HAM) especially for scene text detection. The predictions of anchors are innovatively regarded as hidden layers, and the weighted sum of the predictions is integrated into a direct regression-based network. Hence, the architecture of our HAM still has the characteristic of simplicity as with direct regression-based methods. Moreover, it is easier to optimize anchors as references with this type of method than with direct regression-based methods. In this way, our network can take advantage of both direct regression and anchor mechanisms. In addition, we decouple three kinds of one-dimensional anchors from three-dimensional anchors, greatly reducing the number of anchors in text bounding box matching without performance degradation. We also propose a post-processing technique for long text detection, named iterative regression box (IRB), which takes a few additional computational costs and can be easily generalized to other methods. Experiments on several public datasets demonstrate that the proposed method achieves state-of-the-art performance. Code is available at https://github.com/hjbplayer/HAM .

Overhead contact systems (OCSs) are the power supply facility of high-speed trains and plays a vital role in the operation of high-speed trains. The dropper is an important guarantee for the suspension system of the OCS. Faults of the dropper, such as slack and breakage, can cause a certain threat to the power supply system. How to use artificial intelligence technologies to detect faults is an urgent technical problem to be solved. Because droppers are very small in whole images, a feasible solution to the problem is to identify and locate the droppers first, then segment them, and then identify the fault type of the segmented droppers. This paper proposes an improved Faster R-CNN algorithm that can accurately identify and locate droppers. The innovations of the method consist of two parts. First, a balanced attention feature pyramid network (BA-FPN) is used to predict the detection anchor. Based on the attention mechanism, BA-FPN performs feature fusion on feature maps of different levels of the feature pyramid network to balance the original features of each layer. After that, a center-point rectangle loss (CR Loss) is designed as the bounding box regression loss function of Faster R-CNN. Through a center-point rectangle penalty term, the anchor box quickly moves closer to the ground-truth box during the training process. We validate the improved Faster R-CNN through extensive experiments on the VOC 2012 and MSCOCO 2014 datasets. Experimental results prove the effectiveness of the proposed network combined with attention feature fusion and center-point rectangle loss. On the OCS dataset, the accuracy using the combination of the improved Faster R-CNN and ResNet-101 reached 86.8% mAP@0.5 and 83.9% mAP@0.7, which was the best performance among all results.

Mixed dish, which mixes different types of dishes in one plate, is a popular kind of food in East and Southeast Asia. Identifying the dish type in the mixed dish is essential for dietary tracking, which gains increasing research attention recently. Nevertheless, mixed dish detection is a challenging task because of large visual variances among dishes in different canteens, which is known as the domain shifting problem. Since collecting and annotating sufficient training samples in each canteen for model training is difficult, a more practical way is developing detection models that can adapt quickly to cross-canteen mixed-dish detection with less supervision information. To this end, we propose a novel framework called Weakly-supervised Mean Teacher Network (WMT-Net) that addresses this specific detection task in a weakly supervised manner, where bounding box annotations are not required in the target domain. The proposed WMT-Net constructs Mean Teacher learning by maintaining the image-level consistency between teacher and student modules. Specifically, WMT-Net firstly learns instance-level information from the source dataset in a fully supervised fashion for the student model. Then the whole architecture is optimized with weakly supervised learning: 1) weakly supervised training in student model to reduce the domain gap in global semantics between source data and target data, 2) image-level consistency to align the image-level predictions between teacher model and student model. Experimental results on mixed-dish dataset show that even the proposed WMT-Net is trained in a weakly supervised fashion on the target domain, the performances attained by WMT-Net are very close to the model trained in a fully supervised fashion, which verify the effectiveness of WMT-Net. In addition, the proposed WMT-Net also achieves 44.6% mAP on Pascal VOC to Clipart cross-domain detection, which improves 7.2% mAP compared with the state-of-the-arts method and further demonstrates its generalization capabilities.

Robot-assisted surgery (RAS), a type of minimally invasive surgery, is used in a variety of clinical surgeries because it has a faster recovery rate and causes less pain. Automatic video analysis of RAS is an active research area, where precise surgical tool detection in real time is an important step. However, most deep learning methods currently employed for surgical tool detection are based on anchor boxes, which results in low detection speeds. In this paper, we propose an anchor-free convolutional neural network (CNN) architecture, a novel frame-by-frame method using a compact stacked hourglass network, which models the surgical tool as a single point: the center point of its bounding box. Our detector eliminates the need to design a set of anchor boxes, and is end-to-end differentiable, simpler, more accurate, and more efficient than anchor-box-based detectors. We believe our method is the first to incorporate the anchor-free idea for surgical tool detection in RAS videos. Experimental results show that our method achieves 98.5% mAP and 100% mAP at 37.0 fps on the ATLAS Dione and Endovis Challenge datasets, respectively, and truly realizes real-time surgical tool detection in RAS videos.

In the process of 3D modeling of geological body with fault, it is the primary problem to calculate the intersecting line between fault plane and various geological models. To quickly locate geological models intersecting fault planes, this paper adopts the algorithm of intersection detection of hybrid bounding boxes. Firstly, the principal component analysis is used to generate the bounding box of the tightly bounding geological model. Then, based on the separation axis theorem, an improved algorithm is proposed to realize the intersection detection of hybrid bounding boxes, which effectively reduces the number of detection and improves the efficiency. Experimental results show that this algorithm can greatly shorten the time of intersection test, and improve the efficiency of intersection calculation. This paper provides an algorithm basis for quickly locating the geological model cut by fault plane.

The goal of fine-grained vehicle recognition is to identify the exact subtype of the vehicle from a given image. It plays an important role in the intelligent traffic surveillance system. Although fine-grained vehicle recognition has attracted more and more research interest, it remains as an open problem for vehicle images taken from arbitrary viewpoints. In this study, we present a one-stage deep multi-task learning framework for fine-grained vehicle recognition in traffic surveillance, which performs the fine-grained vehicle recognition and viewpoint estimation simultaneously. In the proposed framework, the fine-grained vehicle recognition is the main task which classifies images into different types, and the viewpoint estimation task is the auxiliary task to learn helpful viewpoint-aware features for improving the main task. We evaluate our method on two common used large-scale fine-grained vehicle recognition datasets, including BoxCars116k and CompCars. The experimental results testify that the proposed multi-task framework can improve the accuracy by incorporating the viewpoint information of the vehicles. In comparison to the state-of-the-art, our approach goes beyond, only requiring 3D bounding box for the training phase, which is important for future inferences using the trained model.

Aiming at the poor expressive ability of image statistical information during the reconstruction process of traditional 3D image reconstruction method based on virtual reality technology, resulting in low accuracy of 3D image after reconstruction, a new image detection and 3D image reconstruction based on virtual reality technology are studied method. This paper first proposed a new two-level cascade convolutional neural network structure. The first level of the network predicts target positioning based on the image-level labels of the training image, generates a bounding box of the target in the original image, and generates a cropped image. The cropped image is input to the second-level network. The cropped image may contain areas where the target is stuck in the original image. Level 2 networks only use the adhesion area as training data. Secondly, the visualization software development platform and virtual reality 3D image processing software are selected as the platform for 3D image reconstruction. After the original image is imported into the computer through data input and file analysis steps, the original image is detected. The detected image is in the virtual in the real software, the bounding box method is first used to construct the three-dimensional data field of image reconstruction, and the three-dimensional direct volume of the image is drawn according to the three-dimensional data field of image reconstruction. Preferably, the three-dimensional image reconstruction output formula is obtained through the three-dimensional image direct volume to realize the three-dimensional image reconstruction based on the virtual reality technology. The simulation results show that the method proposed in this paper can effectively detect images. The average traversal coverage of 3D image reconstruction is up to 0.979, and the reconstruction accuracy is higher than 0.97.

Non-maximum suppression (NMS) is widely used in object detection pipelines for removing duplicated bounding boxes. The inconsistency between the confidence for NMS and the real localization confidence seriously affects detection performance. Prior works propose to predict Intersection-over-Union (IoU) between bounding boxes and corresponding ground-truths to improve NMS, while accurately predicting IoU is still a challenging problem. We argue that the complex definition of IoU and feature misalignment make it difficult to predict IoU accurately. In this paper, we propose a novel Decoupled IoU Regression (DIR) model to handle these problems. The proposed DIR decouples the traditional localization confidence metric IoU into two new metrics, Purity and Integrity. Purity reflects the proportion of the object area in the detected bounding box, and Integrity refers to the completeness of the detected object area. Separately predicting Purity and Integrity can divide the complex mapping between the bounding box and its IoU into two clearer mappings and model them independently. In addition, a simple but effective feature realignment approach is also introduced to make the IoU regressor work in a hindsight manner, which can make the target mapping more stable. The proposed DIR can be conveniently integrated with existing two-stage detectors and significantly improve their performance. Through a simple implementation of DIR with HTC, we obtain 51.3% AP on MS COCO benchmark, which outperforms previous methods and achieves state-of-the-art., ACMMM 2021 Poster

International audience; This paper is devoted to tackling constrained multi-objective optimisation under uncertainty problems. A Surrogate-Assisted Bounding-Box approach (SABBa) is formulated here to deal with approximated robustness and reliability measures, which can be adaptively refined. A Bounding-Box is defined as a multi-dimensional product of intervals, centred on the estimated objectives and constraints, that contains the true underlying values. The accuracy of these estimations can be tuned throughout the optimisation so as to reach high levels only on promising designs, which allows quick convergence toward the optimal area. In SABBa, this approach is supplemented with a Surrogate-Assisting (SA) strategy, which permits to further reduce the overall computational cost. The adaptive refinement within the Bounding-Box approach is guided by the computation of the Pareto Optimal Probability (POP) of each box. We first assess the proposed method on several analytical uncertainty-based optimisation test-cases with respect to an a priori metamodel approach in terms of a probabilistic modified Hausdorff distance to the true Pareto optimal set. The method is then applied to three engineering applications: the design of two-bar truss in structural mechanics, the shape optimisation of an Organic Rankine Cycle turbine blade and the design of a thermal protection system for atmospheric reentry.

Weakly supervised object detection (WSOD), which is an effective way to train an object detection model using only image-level annotations, has attracted considerable attention from researchers. However, most of the existing methods, which are based on multiple instance learning (MIL), tend to localize instances to the discriminative parts of salient objects instead of the entire content of all objects. In this paper, we propose a WSOD framework called the Spatial Likelihood Voting with Self-knowledge Distillation Network (SLV-SD Net). In this framework, we introduce a spatial likelihood voting (SLV) module to converge region proposal localization without bounding box annotations. Specifically, in every iteration during training, all the region proposals in a given image act as voters voting for the likelihood of each category in the spatial dimensions. After dilating the alignment on the area with large likelihood values, the voting results are regularized as bounding boxes, which are then used for the final classification and localization. Based on SLV, we further propose a self-knowledge distillation (SD) module to refine the feature representations of the given image. The likelihood maps generated by the SLV module are used to supervise the feature learning of the backbone network, encouraging the network to attend to wider and more diverse areas of the image. Extensive experiments on the PASCAL VOC 2007/2012 and MS-COCO datasets demonstrate the excellent performance of SLV-SD Net. In addition, SLV-SD Net produces new state-of-the-art results on these benchmarks., Comment: arXiv admin note: text overlap with arXiv:2006.12884

Efficient and robust facial landmark localisation is crucial for the deployment of real-time face analysis systems. This paper presents a new loss function, namely Rectified Wing (RWing) loss, for regression-based facial landmark localisation with Convolutional Neural Networks (CNNs). We first systemically analyse different loss functions, including L2, L1 and smooth L1. The analysis suggests that the training of a network should pay more attention to small-medium errors. Motivated by this finding, we design a piece-wise loss that amplifies the impact of the samples with small-medium errors. Besides, we rectify the loss function for very small errors to mitigate the impact of inaccuracy of manual annotation. The use of our RWing loss boosts the performance significantly for regression-based CNNs in facial landmarking, especially for lightweight network architectures. To address the problem of under-representation of samples with large pose variations, we propose a simple but effective boosting strategy, referred to as pose-based data balancing. In particular, we deal with the data imbalance problem by duplicating the minority training samples and perturbing them by injecting random image rotation, bounding box translation and other data augmentation strategies. Last, the proposed approach is extended to create a coarse-to-fine framework for robust and efficient landmark localisation. Moreover, the proposed coarse-to-fine framework is able to deal with the small sample size problem effectively. The experimental results obtained on several well-known benchmarking datasets demonstrate the merits of our RWing loss and prove the superiority of the proposed method over the state-of-the-art approaches.

With the advent of the 5G era, wireless network data transmission and real-time information acquisition face certain technical challenges. Simultaneously, how to ensure fast and safe information transmission becomes the research topic of the information age. The improvement of this research is based on the traditional fine-grained algorithm, and this study uses spatial constraints and evaluation scores to obtain the bounding box positions of objects and objects. Simultaneously, Part-based R-CNN is used to extract convolution features for each filtered bounding box and join the local features to form a feature representation vector. In addition, the research is based on the application of high-resolution acquisition of information entropy, abnormal signal detection and transient signal measurement methods and techniques in typical time domain test instruments, that is, digital oscilloscope, and this study is experimentally verified. The research indicates that the method proposed in this paper is scientific and can provide theoretical reference for subsequent related research.

This paper proposes an algorithm to reason on constraints expressed in terms of polynomials with integer coefficients whose variables take values from finite subsets of the integers. The proposed algorithm assumes that an initial approximation of the domains of variables is available in terms of a bounding box, and it recursively subdivides the box into disjoint boxes until a termination condition is met. The algorithm includes three termination conditions that allow using it for three related reasoning tasks: constraint satisfaction, enumeration of solutions, and hyper-arc consistency enforcement. Considered termination conditions are based on suitable lower and upper bounds for polynomial functions over boxes that are determined using new results proved in the paper. The algorithm is particularly appropriate to reason on high-degree polynomial constraints because the proposed method to determine lower and upper bounds can outperform alternative methods when high-degree polynomials in a moderate number of variables are considered.

For alleviating the human labor associated with annotating the training data for learning object detectors, recent research has focused on semi-supervised object detection (SSOD) and weakly supervised object detection (WSOD) approaches. In SSOD, instead of annotating all the instances in the whole training set, people only need to annotate the part of the training instances using bounding boxes. In WSOD, people need to annotate the image-level tags on all training images to indicate the object categories contained by the corresponding images since more detailed bounding box annotations are no longer needed. Along this line of research, this paper makes a further step to alleviate the human labor in annotating training data, leading to the problem of object detection with semi-annotated weak labels (ODSAWLs). Instead of labeling image-level tags on all training images, ODSAWL only needs the image-level tags for a small portion of the training images, and then, the object detectors can be learned from a small portion of the weakly-labeled training images and from the remaining unlabeled training images. To address such a challenging problem, this paper proposes a cross model co-training framework that collaborates an object localizer and a tag generator in an alternative optimization procedure. Specifically, during the learning procedure, these two (deep) models can transfer the needed knowledge (including labels and visual patterns) between each other. The whole learning procedure is accomplished in a few stages under the guidance of a progressive learning curriculum. To demonstrate the effectiveness of the proposed approach, we implement the comprehensive experiments on three benchmark datasets, where the obtained experimental results are quite encouraging. Notably, by using only about 15% weakly labeled training images, the proposed approach can effectively approach, or even outperform, the state-of-the-art WSOD methods.

A higher-order accurate meshing algorithm for nonsmooth surfaces defined via Boolean set operations from smooth surfaces is presented. Input data are a set of level-set functions and a bounding box containing the domain of interest. This geometry definition allows the treatment of edges as intersection curves. Initially, the given bounding box is partitioned with an octree that is used to locate corners and points on the intersection curves. Once a point on an intersection curve is found, the edge is traced. Smooth surfaces are discretized using marching cubes and then merged together with the advancing-front method. The piecewise linear geometry is lifted by projecting the inner nodes of the Lagrangian elements onto the surface or intersection curve. To maintain an accurate mesh, special attention is paid to the accurate meshing of tangential intersection curves. Optimal convergence properties for approximation problems are confirmed in numerical studies.

This paper tackles the task of person search, which is a new challenging computer version task in real-world scenarios. The challenge of this task is mainly from: i) Due to viewpoints, occlusions, etc., the visual appearance of a particular person varies greatly, making re-identification difficult; ii) the bounding box where pedestrians are not available, the model needs to search for the person in the entire gallery image. The task of person search is consisted of pedestrian detection and person re-identification (re-id). Instead of completing the two tasks separately, we put the two pieces together to address these issues. We design a more suitable end-to-end framework for person search, which both improve re-id and detection at the same time. Through the change of the anchor with structural prior, the pedestrian detection can be refined faster and better. By introducing self-attention, the framework enhances the fusion of global information. An Online Instance Aggregation Matching (OIAM) loss function is proposed to train the network, and it effectively solves the problem of many categories but few samples of the same category. Extensive experiments are conducted on the PRW and CUHK-SYSU datasets, and our proposed method can outperform other person search methods in both mAP and top-1 evaluation protocols.

In this paper, a novel multi-task framework for object detection is proposed. The framework uses multi-label classification as an auxiliary task to improve object detection, and can be trained and tested end-to-end. The object detection branch adopts R-FCN methods to solve the object detection task. The multi-label branch uses attention mechanism to solve the multi-label classification task. The features, which are generated by the attention mechanism in the multi-label branch, contain rough localization information of the objects. Thus, the features can be useful for the object detection. Both the box-level features and the image-level features of multi-label are fused to improve the accuracy of the object detection. The proposed framework does not require any extra annotation, since the ground truth of the multi-label classification can be directly obtained from the bounding box annotations. This is different from other multi-task frameworks such as StuffNet and Mask R-CNN which need extra semantic segmentation and instance segmentation annotations. Extensive experiments on PASCAL VOC 2007, PASCAL VOC 2012 and MS COCO demonstrate the effectiveness of the proposed approach. Code has been made publicly available at: https://github.com/GT9505/MONet .

In this paper, we address the challenging problem of detecting pedestrians, which are heavily occluded and/or far from cameras. Unlike most existing pedestrian detection methods which only use coarse-resolution feature maps with fixed receptive fields, our approach exploits multi-grained deep features to make the detector robust to visible parts of occluded pedestrians and small-size targets. Specifically, we jointly train a multi-scale network and a human parsing network in a weakly supervised manner with only bounding box annotations. We carefully design the multi-scale network to predict pedestrians of particular scales with the most appropriate feature maps, by matching their receptive fields with the target sizes. The human parsing network generates a fine-grained attention map, which helps guide the detector to focus on the visible parts of occluded pedestrians and small-size instances. Both networks are computed in parallel and form a unified single stage pedestrian detector, which assures a suitable tradeoff between accuracy and speed. Moreover, we introduce an adversarial hiding network to make our detector more robust to occlusion situations, which generates occlusions on pedestrians with the goal to fool the detector that in turn adapts itself to learn to localize these adversarial instances. Experiments on three challenging pedestrian detection benchmarks show that our proposed method achieves a state-of-the-art performance and executes $2\times $ faster than the competitive methods.

Deep visual feature-based method has demonstrated impressive performance in visual tracking attributing to its powerful capability of visual feature representation. However, in some complex environments such as dramatic change of appearance, illumination variation and rotation, the extracted deep visual feature is insufficient for accurately characterizing the target. To solve this problem, we present an integrated tracking framework which combines a Long Short-Term Memory (LSTM) network and a Convolutional Neural Network (CNN). Firstly, the LSTM extracted dynamics feature of target on time sequence, resulting the state of target at present time step. With that state, the accurate preprocessed bounding box was obtained. Then, deep convolutional feature of the target was extracted using a CNN, based on the processed bounding box. Finally, the position of the target was determined based on the score of the feature. During tracking stage, in order to improve the adaptation of the network, the parameters of the network were updated using samples of the target captured while successful tracking. The experiment shows that the proposed method achieves outstanding tracking performance and robustness in cases of partial occlusion, out-of-view, motion blur and fast motion.

3D object recognition and pose estimation are challenging tasks in industrial scenarios. In this paper, we propose an accurate and robust algorithm for object recognition and 6DOF pose estimation for bin-picking applications. We first split the whole point cloud of the object into four parts according to the bounding box of the point cloud. The surface shape characteristics are extracted from these four parts using an extended fast point feature histogram and an extended viewpoint direction component. These surface shape characteristics of four parts are further concatenated to generate a Partition Viewpoint Feature Histogram (PVFH) descriptor. Comparison with the state-of-the-art global descriptors have demonstrated the effectiveness of PVFH. Experimental results on industrial parts demonstrate that the PVFH feature descriptor ensures more accurate pose estimation, and higher computational efficiency.

In this paper, we propose a robust detection and tracking strategy for autonomous aerial refueling of unmanned aerial vehicles. The proposed framework includes two modules: a faster deep-learning-based detector (DLD) and a more accurate reinforcement-learning-based tracker (RLT). In the detection stage, the DLD achieves faster speed by combining the efficient MoblieNet with the you only look once framework. In the tracking stage, RLT is proposed to obtain target’s position accurately and fastly by performing hierarchically positioning and adjusting target bounding box according to the reinforcement learning. The precision of drogue object tracking is 98.7%, which is obviously higher than the other comparison methods. The speed of our network can achieve 15 frames/s on GPU Titan X. The experimental results validate the effectiveness and robustness of the proposed framework.