Search

In recent years, researchers have explored reinforcement learning based object detection methods. However, existing methods always suffer from barely satisfactory performance. The main reasons are that current reinforcement learning based methods generate a sequence of inaccurate regions without a reasonable reward function, and regard the non-optimal one at the final step as the detection result for lack of an effective region selection and refinement strategy. To tackle the above problems, we propose a novel reinforcement learning based object detection framework, namely ReinforceNet, possessing the capability of the region selection and refinement by integrating reinforcement learning agents’ action space with Convolutional Neural Network based feature space. In ReinforceNet, we redevelop a reward function that enables the agent to be trained effectively and provide more accurate region proposals. In order to further optimize them, we design Convolutional Neural Network based region selection network (RS-net) and bounding box refinement network (BBR-net). Particularly, the former consists of two sub-networks: Intersection-of-Union network (IoU-net) and Completeness network (CPL-net) which are employed jointly for selecting the optimal region proposal. The latter aims to refine the selected one as the final result. Extensive experimental results on two standard datasets PASCAL VOC 2007 and VOC 2012 demonstrate that ReinforceNet is capable of improving the region selection and learning better agent action representations for reinforcement learning, resulting in the state-of-the-art performance.

Accurate and automatic localization of the common carotid artery (CCA) is extremely important because the narrowing of the CCA is a silent disease. CCA disease doesn't cause any symptoms in its early stages, and people don't realize that they usually have a problem until they have a stroke. A stroke occurs when the brain doesn't receive enough blood for a long time. Brain damage from a stroke can lead to loss of speech or vision, and major strokes can cause death. In this paper, we proposed a technique to localize the CCA in transverse section ultrasound (US) images using deep learning. First, we applied preprocessing to the images in the dataset before detecting the bounding box containing the CCA. We used a faster regional proposal convolutional neural network (Faster R-CNN) to detect the rectangular region(bounding box) around the CCA. Then we applied a circle localization technique to contour and localize the CCA in the US images. The proposed method has been performed on ultrasonic transverse images of the signal processing (SP) Lab. We compared our results with the clinicians' contours obtaining a great match between them. The accuracy of the bounding box detection was 97.5 and a Jaccard similarity of 90.86% between our proposed system and the clinicians' manual contours. Our proposed system has shown results that outperform other systems in Literature.

A tracking controller for unmanned aerial vehicles (UAVs) is developed to track moving targets in the presence of occlusion. The controller can track moving targets based on a bounding box of the target detected by a deep neural network using the you-only-look-once (YOLO) method. The features generated from the YOLO approach relaxes the assumption of continuous availability of the feature points for applications, which facilitates estimation using an unscented Kalman filter (UKF) and the design of image-based tracking controller in this work. The challenge is that when occlusion is present, the bounding box of the moving target becomes unobtainable and makes the estimation diverge. To solve this, a motion model derived by quadratic programming is employed as a process model in the UKF, wherein the estimated velocity is implemented as a feedforward term in the developed tracking controller in order to enhance the tracking performance. Since no motion constraint is assumed for the target, the developed controller can be applied to track various moving targets. Simulations are used to demonstrate the performance of the developed estimator and controller in the presence of occlusion. Experiments are also conducted to verify the efficacy of the developed estimator and controller.

This project deals with the tracking and following of a single object in a sequence of frames and the velocity of the object is determined. Algorithms are developed for improving the image quality, segmentation, feature extraction and for deterring the velocity. Segmentation is performed to detect the object after reducing the noise from that scene. The object is tracked by plotting a rectangular bounding box around it in each frame. The velocity of the object is determined by calculating the distance that the object moved in a sequence of frames with respect to the frame rate that the video is recorded. The algorithms developed can also be used for other applications (real-time, object classification, etc.). In this project, we use an inbuilt AVI file and then we show it after decompress it with the help of VIRTUALDUB1.9.10 software. After that, we apply all the above algorithms for tracking the object.

The grading size, which can be defined as the side length of the smallest square hole through which an aggregate with irregular shape can directly pass, is an important morphology parameter and can be used to calculate the gradation of mixture material. The grading bounding box, which can be defined as the circumscribed cuboid with central axis length being the grading size, is an important visual tool for observing the size and direction of the aggregate. Virtual test to calculate the grading size of a virtual aggregate is environmentally friendly and efficient, but the result provided by current research is imprecise and the grading bounding box is also rarely mentioned. In this paper, the multilevel complete projection algorithm is proposed to precisely calculate the grading size of a virtual aggregate. The whole process of the algorithm can be expressed by formula after the operation of sphere discretization by converting the virtual aggregate shell into the discrete aggregate. Then, the discrete aggregate is projected onto a complete series of the plane to form several 2D figures, and then, each 2D figure is projected onto a complete series of the orthogonal axis to form orthogonal segments. The grading size can finally be obtained by comparing the length of the above orthogonal segments based on the key central axis length principle. The influencing factors of computational accuracy and efficiency are considered in the algorithm. Finally, the grading bounding box can be built by using the Rodrigues transformation according to the information obtained from the above algorithm.

In recent years, with the rapid development of artificial intelligence, information technology, intelligent digital video surveillance systems, real-time sports competition playback, and other technologies have emerged one after another, making the advantages of deep learning-based football posture detection tasks become more obvious. Related models and methods have been applied to the research field of sports posture estimation and have achieved great improvement, surpassing the traditional football posture estimation method based on manual design features in one fell swoop. In addition, the application of video foreground detection has developed rapidly and has great application value in sports analysis. Therefore, this paper proposes a novel football motion detection approach combining foreground detection and deep learning for real-time detection of football player posture. The main task of foreground target detection is to extract the interesting foreground target in the real monitoring scene and use it as the target of interest for subsequent analysis. Then, we propose a triple DetectNet detection framework based on deep learning technology, which can quickly and robustly realize the three-dimensional pose estimation of multiperson motion. For input, the triple DetectNet framework uses three neural networks and is executed in three stages; the first stage is to use the DetectNet (DN) network to detect the bounding box of each person separately, the second stage uses the 2DPoseNet (2DPN) network to estimate each of the corresponding two-dimensional poses of the individual, and the third stage uses the 3DPoseNet (3DPN) network to obtain the 3D pose of the person. This paper also conducted experiments on four datasets, and the results proved the superiority and success of this algorithm.

Object detection is a classic problem in computer vision. The main bottleneck of object detection lies in the fusion of multi-scale features. In this paper, we systematically study the design choices of neural network architecture for real-time object detection, and propose an Align-Yolact to improve the instance segmentation accuracy. Firstly, we propose a weighted bounding box, which improves the accurate positioning of the bounding box. Secondly, we add a bi-directional feature pyramid network to the feature fusion, which improves the mask quality and small target accuracy. Owing to these optimizations and better backbones, we achieve the SOTA results including both detection efficiency and accuracy.

Scene text detection has become an important research topic. It can be broadly applied to much industrial equipment, such as smart phones, intelligent scanners, and IoT devices. Many existing scene text detection methods have achieved advanced performance. However, text in scene images is presented with differing orientations and varying shapes, rendering scene text detection a challenging task. This paper proposes a method for detecting texts in scene images. First, four stages of low-level features is extracted using DenseNet121. Low-level features are then merged by transposed convolution and skip connection. Second, the merged feature map is used to generate a score map, box map, and angle map. Finally, the Locality-Aware Non-Maximum Suppression (LANMS) is applied as post-processing to generate the final bounding box. The proposed method achieves an F-measure of 0.826 on ICDAR 2015 and 0.761 on MSRA-TD500, respectively.

Detection accuracy and speed are crucial in object detection in computer vision. This work proposes a novel technique called On-Category Anchors (OC-Anchors) to improve the accuracy of real-time single-stage object detectors. The key concept of the OC-Anchors technique is to create anchors based on the categories of foreground objects. The OC-Anchors are set to reflect the bounding box features of the foreground object category. This approach improves the accuracy of predicting the bounding boxes of objects. The performance of the proposed OC-Anchors technique is examined in detail in the YOLOv2 framework with the COCO dataset. The results show that the OC-Anchors technique significantly improves the detection accuracy in tests on COCO test-dev, without substantially affecting the prediction speed. The improvement in average precision ranges from 21.6 to 27.1%.

Vehicle bounding box tracking (VBBT) is a new problem that is becoming increasingly important in autonomous driving. It is defined as a problem in which not only the position but also the size of a target vehicle is estimated using a sensor. In this paper, novel VBBT using a low-end three-dimensional (3D) laser scanner is proposed. Compared to previous methods, the proposed VBBT has three distinctions: (1) the center of a rectangular vehicle is defined as its position, and the motion model that uses the center of the vehicle as the state is developed; (2) a new measurement model is proposed that models the measured size of the target vehicle as a sample from a uniform distribution; and (3) a Bayesian filter for the proposed motion and measurement model is developed and it is named as the Pareto Kalman filter (PKF). Finally, the proposed method is applied to six scenarios, and its validity is demonstrated through experimentation.

The purpose of this work was to assess the resolution to which micro-CT and intra-operative CT systems can quantify distances between radiopaque fiducial markers. Twenty-two markers were cast in a silicone phantom, then imaged at ten random rotations and translations within the field of view of a micro-CT and an intraoperative CT. A bounding box method and a mask-based weighted binary method were used to calculate the location of all markers in an image, then the Euclidian distance between neighbouring marker coordinates was calculated. The standard deviation in the inter-marker distance measurements from each of the marker position methods across the ten repeated trials was calculated for each marker identification method to provide a measure of the precision of the strain measurement with each scanner. The imaging systems measured 3D distances between markers to within 0.007 mm and 0.028 mm in the micro-CT and intra-operative CT, respectively, using the bounding box method, and to within 0.011 mm and 0.040 mm in the micro-CT and intra-operative systems, respectively, using the weighted-mask method. The bounding box method was found to be the most precise and is highly promising for applications in high resolution regional soft-tissue strain measurements.

3D object detection is an essential component of scene perception and motion prediction in autonomous driving. Previous methods represent objects as the truncated signed distance fields (3D bounding box), which can only provide the geometric constraints of point-to-line. In this work, we define the object as a more compact representation, quadric (ellipsoid) in a 3D scene and a conic (ellipse) in an image, which can provide stronger geometric constraints of surface-to-curve. Specifically, we estimate a ellipsoid from a conic fitted by a 2D bounding box to obtain 3D object localization and occupancy. We further to formulate this constraint relation as a nonlinear optimization problem in dual space, which enables us to easy recover stable and accurate 3D object parameters by adding only three additional direction-aware branches to the existing 2D detection networks. In addition, we decouple the dimensions of object and update the length and orientation of objects in our iterative algorithm when the estimations from the 2D detection networks have different deviations. The final detection results can be obtained after passing through our geometry-related refinement network. We evaluate our method on the KITTI object detection benchmark and achieve the best performance among published monocular competitors.

In this paper, we propose an end-to-end detection-segmentation system to implement detailed face labeling. Fully convolutional networks (FCN) has become the mainstream algorithm in the field of semantic segmentation due to the state-of-the-art performance. However, a general FCN usually produces smooth and homogeneous results. Moreover, when semantic category is extremely unbalanced in samples such as face labeling problem, features for some categories cannot be well explored by FCN. To alleviate these problems, a face image is firstly encoded to multi-level feature maps by a pyramid FCN, then features of different facial components are extracted separately according to the bounding box provided by a one-stage detection head. Three class-specific sub-networks are employed to process the extracted features to obtain the respective segmentation results. The skin-hair region can be decoded directly from the back end of the pyramid FCN. Finally, the overall segmentation result is obtained by combining different branches. Moreover, the proposed method trained on a single-face labeled dataset, can be directly used to implement detailed multi-face labeling tasks without any network modification and additional module or data. The overall structure can be trained in an end-to-end manner while maintaining a small network size (12 MB). Experiments show that the proposed method can generate more accurate (single or multi) face labeling results comparing with previous works and gets the state-of-the-art results in HELEN face dataset.

Kidney volume is an essential biomarker for a number of kidney disease diagnoses, for example, chronic kidney disease. Existing total kidney volume estimation methods often rely on an intermediate kidney segmentation step. On the other hand, automatic kidney localization in volumetric medical images is a critical step that often precedes subsequent data processing and analysis. Most current approaches perform kidney localization via an intermediate classification or regression step. This paper proposes an integrated deep learning approach for (i) kidney localization in computed tomography scans and (ii) segmentation-free renal volume estimation. Our localization method uses a selection -convolutional neural network that approximates the kidney inferior-superior span along the axial direction. Cross-sectional (2D) slices from the estimated span are subsequently used in a combined sagittal-axial Mask-RCNN that detects the organ bounding boxes on the axial and sagittal slices, the combination of which produces a final 3D organ bounding box. Furthermore, we use a fully convolutional network to estimate the kidney volume that skips the segmentation procedure. We also present a mathematical expression to approximate the ‘volume error’ metric from the ‘Sorensen–Dice coefficient.’ We accessed 100 patients’ CT scans from the Vancouver General Hospital records and obtained 210 patients’ CT scans from the 2019 Kidney Tumor Segmentation Challenge database to validate our method. Our method produces a kidney boundary wall localization error of ~2.4mm and a mean volume estimation error of ~5%.

Multipedestrian Online Tracking and positioning are the research hotspots and difficulties in the field of computer vision. In this article, we propose a multipedestrian online tracking method based on social force-predicted deformable key-points mapping and compressive sensing. First, part affinity fields are utilized to detect body key-points of pedestrians, and then rectangle boundaries of moving pedestrians can be constrained based on these key-points, the outer rectangle of the key-points replacing the conventional bounding box. Since pedestrian tracking is vulnerable to occlusion and dynamic deformation, a deformable convolution network is applied to extract delicate features. There is one more point, based on the theory of compressive sensing, a very sparse measurement matrix is designed to efficiently compress the features for the pedestrians, which maps features from high-dimensional space to low-dimensional space and preserves the structure of the image feature space of pedestrians. Compressed features are used to measure the similarity between pedestrians in adjacent frames. Furthermore, a pedestrian social force model is adopted, which can predict the trajectory of pedestrians to deal with the problems of partial occlusion and similar appearance, meanwhile making the estimated location more accurately. Finally, extensive experiments on the MOT16 benchmark demonstrate the superior performance of the proposed tracking method. The main tracking performance parameters are significantly improved, with multiobject tracking accuracy and multiobject tracking precision reaching 62.8% and 81.2%, respectively.

Estimating the heights of objects in the field of view has applications in many tasks such as robotics, autonomous platforms and video surveillance. Object height is a concrete and indispensable characteristic people or machine could learn and capture. Many actions such as vehicle avoiding obstacles will be taken based on it. Traditionally, object height can be estimated using laser ranging, radar or stereo camera. Depending on the application, cost of these techniques may inhibit their use, especially in autonomous platforms. Use of available sensors with lower cost would make the adoption of such techniques at higher rates. Our approach to height estimation requires only a single 2D image. To solve this problem we introduce the Monocular Object Height Estimation Network (MOHE-Net) that includes a cascade of two networks. The first network performs the object detection task. This network detects the bounding box of objects of interest. This information is then input to a second network to estimate the object height and is a linear Multi-layer Perceptron (MLP). The linear MLP model models the camera-scene geometry and does not require training or contain activation function as normal MLP did. The developed approach works for static camera set up as well as moving platform. The proposed approach performs state-of-the-art and can be deployed for obstacle avoidance on autonomous platforms. Our code is available at https://github.com/OSUPCVLab/Ford2019/tree/master/Moving%20Object%20Height% 20Estimation%20Network

SiamPRN algorithm performs well in visual tracking, but it is easy to drift under occlusion and fast motion scenes because it uses $$\ell _1$$ ℓ 1 -smooth loss function to measure the regression location of bounding box. In this paper, we propose a multivariate intersection over union (MIOU) loss in SiamRPN tracking framework. Firstly, MIOU loss includes three geometric factors in regression: the overlap area ratio, the center distance ratio, and the aspect ratio, which can better reflect the coincidence degree of target box and prediction box. Secondly, we improve the definition of aspect ratio loss to avoid gradient explosion, improve the optimization performance of prediction box. Finally, based on SiamPRN tracker, we compared the tracking performance of $$\ell _1$$ ℓ 1 -smooth loss, IOU loss, GIOU loss, DIOU loss, and MIOU loss. Experimental results show that the MIOU loss has better target location regression than other loss functions on the OTB2015 and VOT2016 benchmark, especially for the challenges of occlusion, illumination change and fast motion.

The purpose of weakly supervised object localization (WSOL) is to localize an object requiring only classification labels. However, most WSOL methods tend to find a specific part of an object. Further, they introduce more complex optimization problems than the classification problem to compensate for the lack of resources such as bounding box annotation. To be more efficient WSOL, we propose a new architecture that utilizes feature pyramid network (FPN) and multi-scale information to deal with simplified optimization and to improve the localization. In our proposed model, FPN produces multi-scale and high-quality feature maps, and then these feature maps are gathered to conduct classification. Therefore, we can use high-quality and abundant information for localization, which induces several advantages. First, our proposed model improves localization. Second, we don’t have to require solving complex optimization problem. In particular, the second advantage alleviates a significant burden such as hyperparameter tuning. Also, we confirmed through experiments that our proposed method outperforms state-of-the-art methods on the CUB-200-2011 and ILSVRC datasets.

As a new method of Earth observation, video satellite can provide high-temporal resolution remote sensing images for object tracking. Object tracking in satellite videos is promising yet challenging in computer vision. Although many algorithms for satellite video object tracking have been proposed, none of them solve the problem of tracking rotating object. Due to the nadir view, the rotation of an object is very common in the satellite videos. This problem urgently needs to be addressed. In this paper, a rotation-adaptive correlation filter (RACF) tracking algorithm is proposed to address the problem caused by the rotation of object. The proposed algorithm provides the following improvements: (a) A method of estimating the object rotation angle to keep the feature map stable during the object rotation is proposed. This method can overcome the drawback of histogram of oriented gradient (HOG) based trackers, which cannot deal with the rotation of objects in satellite videos; and (b) making the algorithm capable of estimating the change in the bounding box size caused by object’s rotation. The experimental results demonstrate that our algorithm can track object with a 99.84% precision score and 92.96% success score in six videos from the Jilin-1 satellite constellation.

Facial landmark detection is one crucial step for face-based image/video analysis. Despite the fact that recently many facial landmark detection models have achieved remarkable performance, most state-of-the-art heatmap regression-based methods heavily rely on initialization of the face detector. However, there inevitably exists semantic gaps among different annotators or face detectors. An improper facial bounding box will tremendously drop off the performance of the facial landmark detection model. Facial landmark detection would be more practical if robust to face images cropped by diverse manners (see Figure 1 , the col.1 shows face images cropped by a proper bounding box, col.2 and col.3 show face images cropped by an oversize and a small bounding boxes respectively). To this end, we present a “Unconstrained Facial Landmark Detection(UFLD)” mechanism, that aims at enhancing the robustness of facial landmark detection, to deal with the inconsistent cropping manner issue. UFLD consists of two aspects: a Transformation-Invariant Landmark Detector(TILD) and an Availability-Guided Solver(AGS). TILD gives the ability to detect consistent landmarks for face images cropped by diverse manners. And AGS can alleviate the by-effect of “landmarks outside the image” caused by improper cropping results or TILD, and further promote the performance. The proposed mechanism achieved above 6.5% improvement in standard normalized landmarks mean error reduction on face images cropped by diverse manners compared to baselines.

Internet of Things is the interconnectivity between things, individuals and cloud administrations by means of web, which empowers new plans of action. Because of these exchanges, immense volumes of information are smartly created and is shipped off cloud-based server through web; the information is being handled and broken down, bringing about significant and convenient activities for observing the car parking. The serious issue that is arising currently at a worldwide scale and developing dramatically is the gridlock issue brought about by vehicles. A worldwide scale and developing dramatically is the gridlock issue brought about by vehicles. Among that, finding a better parking sparking space in urban areas has become a major problem with an increase of the numbers of vehicles on a daily bases. Therefore making it difficult in having a better and safe parking spot. The system proposes an intelligent smart parking system using computer vision and internet of things. The proposed system starts by acquiring a dataset. The dataset is made up images of various vehicles, which was collected from the faculty of science car park at the Rivers State University, Port Harcourt, Rivers State Nigeria. We proposed two methods for vehicle/parking slot detection. The first method is the use of convolution neural network algorithm which is used with a haar cascade classifier in detection of multiple vehicles in a single picture and video, and put rectangular boxes on identified vehicles. This first method obtained an accuracy of 99.80%. In the second method, we made use of a Mask R-CNN, here we download a pre-trained model weights which was trained on a coco dataset to identify various objects in videos and pictures. The Mask R-CNN model was used to identify various vehicles by putting a bounding box on each of the vehicle detected, but one of the problem of the Mask R-CNN is that it quite slow in training, and it could not really detect all vehicles tested on a high quality high definition video. In summary our, trained model was able to detect vehicles and parking slot on high quality video and it consumes lesser graphic card.

Object detection is most required task in computer vision to meet the requirement of autonomous processes. The key challenge in object detection is to separate the foreground known as dominant information of the image from background. In this paper, a new model Squeeze and Excitation Region-Convolutional Neural Network (SER-CNN) is proposed which combines a Squeeze and Excitation (SE) embedded CNN with a Region proposal CNN (R-CNN) to detect salient objects. In SER-CNN, SE blocks are embedded in CNN and this extracts features made up of grand pixels which are accompanied with channel information. R-CNN generates salient map by fusing grand pixels with bounding box techniques to detect objects which is similar to the ground truth. DUT-OMRON, Pascal VOC and SOD dataset applied to find the performance of the SER-CNN. MAE and maxF performance evaluation measures are used for comparing the proposed work with the existing ones.

Multiple Object Tracking (MOT) in the wild has a wide range of applications in surveillance retrieval and autonomous driving. Tracking-by-Detection has become a mainstream solution in MOT, which is composed of feature extraction and data association. Most of the existing methods focus on extracting targets’ individual features and optimizing the association by hand-crafted algorithms. In this paper, we specially consider the interrelation cue between targets and we propose Human-Interaction Model (HIM) to extract interaction features between the tracked target and its surrounding. The interaction model has more discriminative features to distinguish objects, especially in crowded (dense) scene. Meanwhile we propose an efficient end-to-end model, Deep Association Network (DAN), to optimize the association with graph-based learning mechanism. Both HIM and DAN are constructed by three kinds of deep networks, which include Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) and Graph Neural Network (GNN). The CNNs extract appearance features from bounding box images, the RNNs encoder motion features from historical positions of trajectory. And then the GNNs aim to extract interaction features and optimize graph structure to associate the objects in different frames. In addition, we present a novel end-to-end training strategy for Deep Association Network and Human-Interaction Model. Our experimental results demonstrate performance of our method reaches the state-of-the-art on MOT15, MOT16 and DukeMTMCT datasets.

Manual harvesting of loose fruits in the oil palm plantation is both time consuming and physically laborious. Automatic harvesting system is an alternative solution for precision agriculture which requires accurate visual information of the targets. Current state-of-the-art one-stage object detection method provides excellent detection accuracy; however, it is computationally intensive and impractical for embedded system. This paper proposed an improved YOLO model to detect oil palm loose fruits from unmanned aerial vehicle images. In order to improve the robustness of the detection system, the images are augmented by brightness, rotation, and blurring to simulate the actual natural environment. The proposed improved YOLO model adopted several improvements; densely connected neural network for better feature reuse, swish activation function, multi-layer detection to enhance detection on small targets and prior box optimization to obtain accurate bounding box information. The experimental results show that the proposed model achieves outstanding average precision of 99.76% with detection time of 34.06 ms. In addition, the proposed model is also light in weight size and requires less training time which is significant in reducing the hardware costs. The results exhibit the superiority of the proposed improved YOLO model over several existing state-of-the-art detection models.

Vision-based traffic surveillance has been one of the most promising fields for improvement and research. Still, many challenging problems remain unsolved, such as addressing vehicle occlusions and reducing false detection. In this work, a method for vehicle detection and tracking is proposed. The proposed model considers background subtraction concept for moving vehicle detection but unlike conventional approaches, here numerous algorithmic optimization approaches have been applied such as multi-directional filtering and fusion based background subtraction, thresholding, directional filtering and morphological operations for moving vehicle detection. In addition, blob analysis and adaptive bounding box is used for Detection and Tracking. The Performance of Proposed work is measured on Standard Dataset and results are encouraging.

Object detection is widely used in object tracking; anchor-free object tracking provides an end-to-end single-object-tracking approach. In this study, we propose a new anchor-free network, the Siamese center-prediction network (SiamCPN). Given the presence of referenced object features in the initial frame, we directly predict the center point and size of the object in subsequent frames in a Siamese-structure network without the need for perframe post-processing operations. Unlike other anchor-free tracking approaches that are based on semantic segmentation and achieve anchor-free tracking by pixel-level prediction, SiamCPN directly obtains all information required for tracking, greatly simplifying the model. A center-prediction sub-network is applied to multiple stages of the backbone to adaptively learn from the experience of different branches of the Siamese net. The model can accurately predict object location, implement appropriate corrections, and regress the size of the target bounding box. Compared to other leading Siamese networks, SiamCPN is simpler, faster, and more efficient as it uses fewer hyperparameters. Experiments demonstrate that our method outperforms other leading Siamese networks on GOT-10K and UAV123 benchmarks, and is comparable to other excellent trackers on LaSOT, VOT2016, and OTB-100 while improving inference speed 1.5 to 2 times.

Visual perception of the objects in a 3D environment is a key to successful performance in autonomous driving and simultaneous localization and mapping (SLAM). In this paper, we present a real time approach for estimating the distances to multiple objects in a scene using only a single-shot image. Given a 2D Bounding Box (BBox) and object parameters, a 3D distance to the object can be calculated directly using 3D reprojection; however, such methods are prone to significant errors because an error from the 2D detection can be amplified in 3D. In addition, it is also challenging to apply such methods to a real-time system due to the computational burden. In the case of the traditional multi-object detection methods, %they mostly pay attention to existing works have been developed for specific tasks such as object segmentation or 2D BBox regression. These methods introduce the concept of anchor BBox for elaborate 2D BBox estimation, and predictors are specialized and trained for specific 2D BBoxes. In order to estimate the distances to the 3D objects from a single 2D image, we introduce the notion of \textit{anchor distance} based on an object's location and propose a method that applies the anchor distance to the multi-object detector structure. We let the predictors catch the distance prior using anchor distance and train the network based on the distance. The predictors can be characterized to the objects located in a specific distance range. By propagating the distance prior using a distance anchor to the predictors, it is feasible to perform the precise distance estimation and real-time execution simultaneously. The proposed method achieves about 30 FPS speed, and shows the lowest RMSE compared to the existing methods., Comment: submitted to RA-letter with ICRA2021 option

Machine learning (ML)-enabled approaches are considered a substantial support technique of detection and classification of obstacles of traffic participants in self-driving vehicles. Major breakthroughs have been demonstrated the past few years, even covering complete end-to-end data processing chain from sensory inputs through perception and planning to vehicle control of acceleration, breaking and steering. YOLO (you-only-look-once) is a state-of-the-art perception neural network (NN) architecture providing object detection and classification through bounding box estimations on camera images. As the NN is trained on well annotated images, in this paper we study the variations of confidence levels from the NN when tested on hand-crafted occlusion added to a test set. We compare regular pedestrian detection to upper and lower body detection. Our findings show that the two NN using only partial information perform similarly well like the NN for the full body when the full body NN's performance is 0.75 or better. Furthermore and as expected, the network, which is only trained on the lower half body is least prone to disturbances from occlusions of the upper half and vice versa., Preprint SEAA 2021

In order to conduct optical neurophysiology experiments on a freely swimming zebrafish, it is essential to quantify the zebrafish head to determine exact lighting positions. To efficiently quantify a zebrafish head's behaviors with limited resources, we propose a real-time multi-stage architecture based on convolutional neural networks for pose estimation of the zebrafish head on CPUs. Each stage is implemented with a small neural network. Specifically, a light-weight object detector named Micro-YOLO is used to detect a coarse region of the zebrafish head in the first stage. In the second stage, a tiny bounding box refinement network is devised to produce a high-quality bounding box around the zebrafish head. Finally, a small pose estimation network named tiny-hourglass is designed to detect keypoints in the zebrafish head. The experimental results show that using Micro-YOLO combined with RegressNet to predict the zebrafish head region is not only more accurate but also much faster than Faster R-CNN which is the representative of two-stage detectors. Compared with DeepLabCut, a state-of-the-art method to estimate poses for user-defined body parts, our multi-stage architecture can achieve a higher accuracy, and runs 19x faster than it on CPUs.

The lethal infection, World Health Organization (WHO) reported coronavirus (COVID-19) as a pandemic Lack of proper vaccine, low levels of immunity against COVID-19 has led to vulnerability of the human beings Due to lack of efficient vaccine treatment, the only options left to fight against this pandemic are lockdown and social distance This work offers an autonomous monitoring system on social distancing using deep learning techniques The proposed architecture tracks the humans on roads and calculates their distance between each other This surveillance detects the furore violation of social distance utilizing CCTV cameras The proposed framework uses YOLO v3 object-detection model built on COCO dataset and used to classify human class among 79 classes The bounding box's dimensions and centroid coordinates are computed in the two-dimensional feature space from the pairwise vectorized L2 norm and a threshold is fixed for computing the distance maintained between each other We illustrate the superior performance of our framework checked against other state of the art methods regarding inference speed, mean average precision and loss defined from the localization [ABSTRACT FROM AUTHOR] Copyright of Turkish Journal of Electrical Engineering & Computer Sciences is the property of Scientific and Technical Research Council of Turkey and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission However, users may print, download, or email articles for individual use This abstract may be abridged No warranty is given about the accuracy of the copy Users should refer to the original published version of the material for the full abstract (Copyright applies to all Abstracts )

Sparse registration can effectively solve the interference by outliers and noise in the registration procedure and thus has become a useful method. However, experimental research on noncharacteristic point cloud registration with sparse ICP (point to point) and sparse TDM (point to plane) reveals that sparse ICP easily converges to wrong local minima, and sparse TDM easily slides in the tangent plane. To overcome these shortcomings, this study proposes a sparse mixed ICP algorithm that combines the solving procedure of sparse ICP and sparse TDM by using the sigmoid weight function. Sparse mixed ICP can avoid convergence to the wrong local minima; moreover, it can restrain the sliding in the tangent plane and retain sparse registration capabilities to handle outliers and noise. Meanwhile, the success rate of the registration is increased by subdividing the bounding box to decrease the outliers effectively. Experiment results show that the proposed algorithm can handle the registration of noncharacteristic point clouds effectively and efficiently and has strong robustness of point cloud registration with different levels of outliers and noise.

In this era of artificial intelligence, a wide variety of techniques are available in healthcare industry especially to study about various changes happening in the human body. Intelligent assistance using brain-like framework helps to understand and analyze various types of complex data by utilizing most recent innovations such as deep learning and computer vision. Activities are complex practices, including continuous actions as well as interleaved actions that could be processed with fully interconnected neuron-like processing machine in a way the human brain works. Human postures have the ability to express different body movements in different environments. An optimal method is required to identify and analyze different kinds of postures so that the recognition rate has to be increased. The system should handle ambiguous circumstances that include diverse body movements, multiple views and changes in the environments. The objective of this research is to apply real-time pose estimation models for object detection and abnormal activity recognition with vision-based complex key point analysis. Object detection based on bounding box with a mask is successfully implemented with detectron2 deep learning model. Using PoseNet model, normal and abnormal activities are successfully distinguished, and the performance is evaluated. The proposed system implemented a state of the art computing model for the development of public healthcare industry. The experimental results show that the models have high levels of accuracy for detecting sudden changes in movements under varying environments.

This work presentsObject Landmarks, a new type of visual feature designed for visual localization over major changes in distance and scale. AnObject Landmarkconsists of a bounding box$${\mathbf {b}}$$bdefining an object, a descriptor$${\mathbf {q}}$$qof that object produced by a Convolutional Neural Network, and a set of classical point features within$${\mathbf {b}}$$b. We evaluateObject Landmarkson visual odometry and place-recognition tasks, and compare them against several modern approaches. We find thatObject Landmarksenable superior localization over major scale changes, reducing error by as much as 18% and increasing robustness to failure by as much as 80% versus the state-of-the-art. They allow localization under scale change factors up to 6, where state-of-the-art approaches break down at factors of 3 or more.

Fruit yield estimation is one of the challenging tasks using on-plant images to support smart farming and provide information about the product so that storage and export facility can be arranged. The detection and counting of on-plant fruits is a challenging task in complex vision. In this paper, we modify the intersection of union (IoU) in original Faster R-CNN (FR-CNN) for on-plant fruit detection. The modified IoU (MIoU) introduces good distance metric with the minimum area containing ground truth and predicted bounding box. Again, the MIoU pays extra attention to overlapping areas, which overcome the inefficiency of original R-CNN and enhance the detection accuracy. The proposed FR-CNN with MIoU achieved the correlation coefficient (R2) of mango, pomegranate, tomato, apple & orange are 0.98,0.92, 0.96, 0.98 & 0.95 respectively with the variation of imaging condition. In the same image sample, FR-CNN achieved the correlation coefficient (R2) of mango, pomegranate, tomato, apple & orange are 0.81,0.91,0.89, 0.90 & 0.92 respectively. So, the FR-CNN with MIoU enhances the detection accuracy compared to original FR-CNN. Again, the F1 score for apple, orange, tomato, pomegranate and mango is 0.9534, 0.9794, 0.9424, 0.9534 and mango 0.9383 respectively. In addition, the proposed method is efficient enough with less complex compared to state-of-art models for fruit detection. Again, the proposed methodology is evaluated using other state-of-art fruits datasets, namely ACFR dataset and KFuji RGB-DS dataset. The proposed methodology achieved F1 score of 0.9523 & 0.9432 for yield estimation of apple & mango of ACFR dataset and 0.8912 for KFuji RGB-DS dataset.

Ship detection in remote sensing plays an important role in civil and military fields. Owing to the complex background and uncertain direction, ship detection is full of challenge by using the commonly used object-detection methods. In this article, a new framework for detecting the arbitrary direction ships is proposed based on the improvement in the Faster region-based convolutional network (R-CNN), in which the shape of the bounding box is described by three sides, namely, vertical side, horizontal side, and short side, respectively. The inclination of the ship is obtained by calculating the arc-tangent value of the vertical side to the horizontal side. First, the better performing ResNet-101 is adopted to extract features over an entire image, which are shared by the region proposal network (RPN) and the head network. Then, the multidirection proposal regions that may contain ships are generated by the RPN. Next, the global and local features of the proposal regions are combined as the whole features of the regions by a multiregion feature-fusion (MFF) module, which can provide more detailed information of the regions. Finally, the head network uses the whole features of the proposal regions for bounding-box recognition through multitask learning, including classification, regression, and incline direction prediction (left or right). The proposed method is tested and compared with other state-of-the-art ship-detection methods on two open remote-sensing data sets and some large-scale and real images. The experimental results validate that the proposed approach has achieved better performance.

Collision detection is currently a hot issue in virtual reality and other fields. The efficiency and accuracy of collision detection directly affect the real-time update effect of the virtual reality environment, and it is also an important indicator that affects the user's interactive experience. In a complex virtual reality scene, if the traditional collision detection algorithm (Sphere-OBB) is adopted and the tree structure traversal is used to realize the bounding box traversal detection, the accuracy remains unchanged, but the detection complexity is reduced. If the RAPID collision detection algorithm is used, the separated axis test method and the two-layer hybrid hierarchical surrounding tree structure are used, although the amount of calculation is large, the detection efficiency is improved. Using the separation axis (SAT) algorithm and using the separation axis theorem to determine the vector axis can save a lot of calculation time. The purpose of this research is to propose an improved hybrid-level bounding box collision detection optimization algorithm (ASO) based on the traditional hybrid-level bounding box collision detection algorithm. Firstly, based on the spatio-temporal correlation theory, the hybrid hierarchical bounding box hierarchical tree structure is improved to AABB and OBB from top to bottom. The synchronous descent rule is used to realize the traversal of nodes, and then the triangle area weighting method is used to improve the calculation method of the bottom OBB bounding box node center, solve the bounding box vertex covariance matrix, and improve the efficiency and accuracy of collision detection. The experimental results show that the algorithm proposed in this paper is 35.6% faster than the RAPID detection speed and 29.9% faster than the separation axis (SAT) detection speed under the same accuracy. In the multi-object collision detection, compared with the latest research, the algorithm in this paper shortens the intersection detection time, improves the collision detection efficiency, and meets the real-time update requirements of complex virtual reality scenes.

Intrusion detection is a crucial task in power grid surveillance system by providing early warning for power grid security. Construction machinery and engineering vehicles, as the most common intrusion objects, have become the major concern for preventing external damages in power grid maintenance. In this paper, by considering the diversity of scales of intrusion objects and complexity of application scenarios under power grid surveillance, we compiled a dataset which contains 8177 images captured by 653 different power grid surveillance cameras. Based on this dataset, we proposed an improved context-aware mask region-based convolutional neural network (Mask R-CNN) model, namely ID-Net, for intrusion object detection. A modulated deformable convolutional operation is integrated into the backbone network for learning robust feature representations from geometric variations in engineering vehicles. By considering the correlation between objects and their context, a self-attention-based module is leveraged for long-range context relation modeling. For small objects detection, a feature integration module is applied for multi-scale feature fusion under a pyramid hierarchy. Then, a cascaded coarse-to-fine region proposal network is incorporated for progressively refining the bounding box location regression. Experimental results have demonstrated that our model can achieve competitive performance in comparison with state-of-the-art object detection methods.

Fine-grained recognition is a very challenging issue, since it is difficulty to mine discriminative and subtle feature for objects with similar visual appearance. Because massive manual annotations (e.g., bounding box for discriminative regions) are time-consuming and labor-consuming, existing methods designed single form of attention model outputted discriminative regions in a weakly supervised way. In this paper, we proposed a novel method named a Spatial-Channel Aware Attention Filters (SCAF) to address the weakly supervised fine-grained recognition problem. Compared with the previous attention models, SCAF can obtain attentions-aware features from two dimensions, i.e., the spatial location of image and the channel of feature maps. With the proposed SCAF, the model can enhance the discriminative regions on both spatial and channel dimensions simultaneously. In addition, the multi-channel network multi-level structure are designed to extract richer regional features. Moreover, focal loss is introduced to balance the samples’ distribution of fine-grained image dataset. Comprehensive and comparable experiments are conducted in publicly available datasets, and the experimental results show that our method can achieve the state-of-the-art performance on fine-grained recognition tasks. For instance, we achieve 99.370%, 80.749% accuracy on two underwater datasets respectively, i.e., Fish4Knowlege and Wild Fish.

In recent years, the performance of the convolutional neural network-based pedestrian detection method has improved significantly. However, an imbalance remains between detection accuracy and speed. In this paper, we employ a one-stage object detection framework and propose a pedestrian detection method based on the multi-scale attention mechanism of a convolutional neural network to improve the imbalance between accuracy and speed. First, a multi-scale convolution module is designed to extract corresponding features at different scales. Second, using the attention module, association information between features is mined from space and channel perspectives to strengthen the original features. Then, the enhanced features are passed through a classification and regression module to perform object positioning and bounding box regression. Finally, to learn more pedestrian location information, we improve the loss function to realise better network training. The proposed method achieved considerable results on the challenging CityPersons and Caltech pedestrian detection datasets.

In the in situ visualization of large-scale simulation, if using the traditional sort-last parallel rendering method, the performance cannot be fully improved due to the heavy communication cost in the final stage: compositing of images. In order to resolve this problem, in this paper, we proposed a composition-free parallel volume rendering method. This is achieved by our proposed hybrid parallelization based on the image space partition and octree-based data space partition. Firstly, in the data distribution stage, an octree structure decoupled from volume representation was proposed, which only stored the indicts of sub-blocks, the adjacent sub-blocks, and the space coordinate information of its bounding box. Before rendering, to avoid the compositing stage, a ray-data intersection list was created for each pixel, which sorted by visibility order and identified NON-EMPTY sub-blocks along the ray. Then in the rendering stage, to improve the efficiency of rendering, a ray-data intersection lists-based rendering method was proposed. Finally, we only need to align the partial images to form the final output image. No compositing process is necessary in our parallel volume rendering method. In order to demonstrate the efficiency of our approach, two existing volume rendering methods have been carried out to compare with our composition-free method.

The intelligent processing and utilization of visual perception information is the key technology of Internet of Things (IoT), and object detection based on deep learning is of great significance for improving the intelligence and security of IoT. Due to the influence of factors such as changes in the state of object in actual scene, occlusion and background changes, object detection method of deep learning still has following problems: features extracted by backbone network are noisy and not representative, the positive and negative samples are not balanced, and labeled object is inaccurate due to occlusion. Therefore, this paper proposes an object detection method based on global feature augmentation and adaptive regression. HRFPN extracts more representative high-resolution features and performs global augmentation, which can effectively distinguish feature differences between object and background. In training phase, uniform sampling is introduced to mine hard samples, and the positive and negative samples in RPN phase are balanced to improve detection performance, and adaptive bounding box regression loss is proposed to reduce the influence of object occlusion and boundary blur. Experimental results on PASCAL VOC2007 and MS COCO2017 datasets show that the proposed detection method is superior to the latest methods such as Cascade RCNN, CornerNet and Mask RCNN, which has better robustness and accuracy.

We propose a high-speed convolutional neural network approach for weakly supervised localization (WSL) and weakly supervised object detection (WSOD). The proposed method, called center-of-features localization (COFL), performs localization of objects in a visual scene by combining both multi-label classification and regression for the number of instances of each class object. A modified Xception network architecture is used as the main feature extractor, and a classification-plus-regression loss function is used to perform the detection task. The method does not require bounding box annotations but only image labels and counts of the objects of each class in the image. This combination can produce a clear localization of objects in the scene through a masking technique between class activation maps (CAMs) and regression activation maps (RAMs). The proposed method was trained and tested on the PASCAL VOC2007 and VOC2012 datasets; it attained a mean average precision (mAP) of 47.0% and a correct localization CorLoc of 64.1% on PASCAL VOC2007 and a mAP of 42.3% and a CorLoc of 65.5% on PASCAL VOC2012 while performing object detection at a speed of ~50 fps. These results demonstrate that the network can perform object detection accurately in real-time using only image labels and object counts, which are inexpensive to annotate compared with the bounding box annotations typically employed in fully supervised object detection methods. The network far outperforms other weakly supervised methods and some fully supervised methods in terms of processing time while achieving fair accuracy.

Oriented object detection has recently attracted increasing attention for its importance in aerial image processing. Popular detection methods for oriented and densely packed objects usually utilize the rotation angle to reduce the overlap of bounding boxes over the horizontal line. However, those angle-based methods remain challenging due to the angular periodicity and regression inconsistent, which are uniformly summarized as regression uncertainty. Previous methods usually heal the regression uncertainty problem by adding complex limits on the loss function. In this paper, instead of expressing accurate oriented information by the rotation angle, we propose a novel projection-based method that employs six-parameter to describe rotated bounding box: two points position on the projected line(one center point and one chosen vertex), a projection ratio and a quadrant label, named as ProjBB. ProjBB is angle-free and has three main advantages: i) More accurate angle distance without periodicity; ii) Projection ratio which is aspect ratio sensitive; iii) Theoretical uniqueness guarantee. Extensive experiments on two large-scale public aerial image datasets(i.e. DOTA and HRSC2016) and the scene text dataset ICDAR2015 show the competitive results of our proposed approach.

In this article, we propose an effective scheme to generate an artificial training database (DB) to mitigate the deficiency in the amount of training DB for ship detection using satellite synthetic aperture radar (SAR) images. In the proposed scheme, SAR signatures of ship targets are first obtained from two different types of image: 1) Korea Multipurpose Satellite-5 images with automatic identification system information and 2) simulated SAR images that are acquired using electromagnetic numerical analysis. Then, the ship signatures are combined with sea clutter models to generate realistic SAR patches. For overall ship detection tasks (i.e., training and testing), the oriented bounding box concept was adopted to appropriately represent slender ship targets in SAR images. When compared with the rash collection of a large number of real SAR images, the proposed scheme is more efficient in terms of cost and time. Experimental results demonstrated that the proposed method can considerably aid in increasing the ship detection capacity of deep learning models. Thus, it is expected that the proposed scheme can be usefully exploited to solve the lack of the amount of training DB.

Currently, reliable and accurate ship detection in optical remote sensing images is still challenging. Even the state-of-the-art convolutional neural network (CNN)-based methods cannot obtain very satisfactory results. To more accurately locate the ships in diverse orientations, some recent methods conduct the detection via the rotated bounding box. However, it further increases the difficulty of detection because an additional variable of ship orientation must be accurately predicted in the algorithm. In this article, a novel CNN-based ship-detection method is proposed by overcoming some common deficiencies of current CNN-based methods in ship detection. Specifically, to generate rotated region proposals, current methods have to predefine multioriented anchors and predict all unknown variables together in one regression process, limiting the quality of overall prediction. By contrast, we are able to predict the orientation and other variables independently, and yet more effectively, with a novel dual-branch regression network, based on the observation that the ship targets are nearly rotation-invariant in remote sensing images. Next, a shape-adaptive pooling method is proposed to overcome the limitation of a typical regular region of interest (ROI) pooling in extracting the features of the ships with various aspect ratios. Furthermore, we propose to incorporate multilevel features via the spatially variant adaptive pooling. This novel approach, called multilevel adaptive pooling, leads to a compact feature representation more qualified for the simultaneous ship classification and localization. Finally, a detailed ablation study performed on the proposed approaches is provided, along with some useful insights. Experimental results demonstrate the great superiority of the proposed method in ship detection.

Siamese tracking methods have become the focus of visual tracking in recent years. Advanced Siamese trackers perform well on certain benchmarks, but there are still some limitations. First, most Siamese trackers adopt the initial frame as a single template, which leads to underfitting and reduces the ability to predict instances. Second, mainstream trackers report a rectangular bounding box as a prediction, resulting in poor accuracy of non-rigid objects. Therefore, we propose the template enhancement and mask generation for Siamese tracking. Given that the essence of Siamese trackers is instance learning, we propose constructing an alternative template explicitly to address the underfitting of the instance space. Moreover, in order to improve the tracking accuracy, we obtain the descriptor aggregation to transform the semantic segmentation outputs for mask prediction. Finally, we propose the SiamEM through the fusion of the above approaches. Comprehensive experiments show that template enhancement and mask generation significantly improve Siamese trackers on benchmarks.

Regression loss function in object detection model plays an important factor during training procedure. The IoU based loss functions, such as CIOU loss, achieve remarkable performance, but still have some inherent shortages that may cause slow convergence speed. The paper proposes a Scale-Sensitive IOU(SIOU) loss for the object detection in multi-scale targets, especially the remote sensing images to solve the problem where the gradients of current loss functions tend to be smooth and cannot distinguish some special bounding boxes during training procedure in multi-scale object detection, which may cause unreasonable loss value calculation and impact the convergence speed. A new geometric factor affecting the loss value calculation, namely area difference, is introduced to extend the existing three factors in CIOU loss; By introducing an area regulatory factor $\gamma $ to the loss function, it could adjust the loss values of the bounding boxes and distinguish different boxes quantitatively. Furthermore, we also apply our SIOU loss to the oriented bounding box detection and get better optimization. Through extensive experiments, the detection accuracies of YOLOv4, Faster R-CNN and SSD with SIOU loss improve much more than the previous loss functions on two horizontal bounding box datasets, i.e, NWPU VHR-10 and DIOR, and on the oriented bounding box dataset, DOTA, which are all remote sensing datasets. Therefore, the proposed loss function has the state-of-the-art performance on multi-scale object detection.