Search

The autonomous exploration task we consider requires Unmanned Aerial Vehicles (UAVs) to actively navigate through unknown environments with the goal of fully perceiving and mapping the environments. Some existing exploration strategies suffer from rough cost budgets, ambiguous Information Gain (IG), and unnecessary backtracking exploration caused by Fragmented Regions (FRs). In our work, a hierarchical spatio-temporal aware exploration framework is proposed to alleviate these problems. At the local exploration level, the Asymmetrical Traveling Salesman Problem (ATSP) are solved by comprehensively considering exploration time, IG, and heading consistency to avoid blindly exploring. Specifically, the exploration time is reasonably budgeted by fast marching in an artificial potential field. Meanwhile, a transformer-based map occupancy predictor is designed to assist in IG calculation by imagining spatial clues out of the Field of View (FoV), facilitating the prescient exploration. We verify that our local exploration is effective in alleviating the unnecessary back-and-forth movements caused by FRs and the interference of potential obstacle occlusion on the IG calculation. At the global exploration level, the classical Next Best View Points (NBVP) are generalized to Next Best Sub-Regions (NBSR) to choose informative sub-regions for further forward-looking exploration based on a well-designed utility function. Safe flight paths and dynamically feasible trajectories are reasonably generated throughout the exploration process by fast marching and B-spline curve optimization. Comparative simulations and benchmark tests demonstrate that our proposed exploration strategy is quite competitive in terms of exploration path length, total exploration time, and exploration ratio.

Automated detection of cervical cancer cells/clumps has the potential to significantly reduce error rate and increase productivity in cervical cancer screening. However, most traditional methods rely on the success of accurate cell segmentation and discriminative hand-crafted features extraction. Recently there are emerging deep learning-based methods which train Convolutional Neural Networks (CNN) to classify cell patches or to detect cells from the whole image. But the former is computationally expensive, while the latter often requires a large-scale dataset with expensive annotations. In this paper we propose an efficient cervical cancer cells/clumps detection method, called Comparison detector, to deal with the limited data problem. Specifically, we utilize the state-of-the-art proposal-based object detection method, Faster R-CNN with Feature Pyramid Network (FPN) as the baseline and replace the classification of each proposal by comparing it with the prototype representation of each category. In addition, we propose to learn the prototype representation of the background category from data instead of manually choosing them by some heuristic rules. Experimental results show that the proposed Comparison detector yields significant improvement on the small dataset, achieving a mean Average Precision (mAP) of 26.3% and an Average Recall (AR) of 35.7%, both improved by about 20% comparing to the baseline. Moreover, when training on the medium-sized dataset, our Comparison detector gains a mAP of 48.8% and an AR of 64.0%, improving the AR by 5.1% and the mAP by 3.6% respectively. Our method is promising for the development of automation-assisted cervical cancer screening systems. Code and datasets are available at https://github.com/kuku-sichuan/ComparisonDetector .

Cervical abnormal cell detection is a challenging task as the morphological discrepancies between abnormal and normal cells are usually subtle. To determine whether a cervical cell is normal or abnormal, cytopathologists always take surrounding cells as references to identify its abnormality. To mimic these behaviors, we propose to explore contextual relationships to boost the performance of cervical abnormal cell detection. Specifically, both contextual relationships between cells and cell-to-global images are exploited to enhance features of each region of interest (RoI) proposals. Accordingly, two modules, dubbed as RoI-relationship attention module (RRAM) and global RoI attention module (GRAM), are developed and their combination strategies are also investigated. We establish a strong baseline by using Double-Head Faster R-CNN with feature pyramid network (FPN) and integrate our RRAM and GRAM into it to validate the effectiveness of the proposed modules. Experiments conducted on a large cervical cell detection dataset reveal that the introduction of RRAM and GRAM both achieves better average precision (AP) than the baseline methods. Moreover, when cascading RRAM and GRAM, our method outperforms the state-of-the-art (SOTA) methods. Furthermore, we also show the proposed feature enhancing scheme can facilitate both image-level and smear-level classification. The code and trained models are publicly available at https://github.com/CVIU-CSU/CR4CACD., Comment: 10 pages, 14 tables, and 3 figures

In this paper, we present a fast yet effective method for pixel-level scale-invariant image fusion in spatial domain based on the scale-space theory. Specifically, we propose a scale-invariant structure saliency selection scheme based on the difference-of-Gaussian (DoG) pyramid of images to build the weights or activity map. Due to the scale-invariant structure saliency selection, our method can keep both details of small size objects and the integrity information of large size objects in images. In addition, our method is very efficient since there are no complex operation involved and easy to be implemented and therefore can be used for fast high resolution images fusion. Experimental results demonstrate the proposed method yields competitive or even better results comparing to state-of-the-art image fusion methods both in terms of visual quality and objective evaluation metrics. Furthermore, the proposed method is very fast and can be used to fuse the high resolution images in real-time. Code is available at https://github.com/yiqingmy/Fusion.

In this paper we propose a very efficient method to fuse the unregistered multi-focus microscopical images based on the speed-up robust features (SURF). Our method follows the pipeline of first registration and then fusion. However, instead of treating the registration and fusion as two completely independent stage, we propose to reuse the determinant of the approximate Hessian generated in SURF detection stage as the corresponding salient response for the final image fusion, thus it enables nearly cost-free saliency map generation. In addition, due to the adoption of SURF scale space representation, our method can generate scale-invariant saliency map which is desired for scale-invariant image fusion. We present an extensive evaluation on the dataset consisting of several groups of unregistered multi-focus 4K ultra HD microscopic images with size of 4112 x 3008. Compared with the state-of-the-art multi-focus image fusion methods, our method is much faster and achieve better results in the visual performance. Our method provides a flexible and efficient way to integrate complementary and redundant information from multiple multi-focus ultra HD unregistered images into a fused image that contains better description than any of the individual input images. Code is available at https://github.com/yiqingmy/JointRF., 14 pages,11 figures

Color variations of histological images due to multi-factor hinder the performance of computer-aided diagnosis (CAD) systems. Previous stain normalization methods have achieved excellent results. While in practice, a multidomain stain normalization method is still be needed when more than two color variations exist in dataset. In this paper, we propose a multidomain stain normalization model inspired by MUNIT [1], with the idea of disentangling the representations of content and style. We assume that the latent space of histological images can be decomposed into domain-shared content space and domain-specific style space. The stain normalization aims to transfer the styles cross domains and maintain the contents. In addition, we propose to use the earth mover’s distance(EMD) to evaluate the effectiveness of stain normalization. We evaluate our approach against the state-of-the-art methods quantitatively and qualitatively.

Background and Objective:Computer-aided cervical cancer screening based on an automated recognition of cervical cells has the potential to significantly reduce error rate and increase productivity compared to manual screening. Traditional methods often rely on the success of accurate cell segmentation and discriminative hand-crafted features extraction. Recently, detector based on convolutional neural network is applied to reduce the dependency on hand-crafted features and eliminate the necessary segmentation. However, these methods tend to yield too much false positive predictions. Methods:This paper proposes a global context-aware framework to deal with this problem, which integrates global context information by an image-level classification branch and a weighted loss. And the prediction of this branch is merged into cell detection for filtering false positive predictions. Furthermore, a new ground truth assignment strategy in the feature pyramid called soft scale anchor matching is proposed, which matches ground truths with anchors across scales softly. This strategy searches the most appropriate representation of ground truths in each layer and add more positive samples with different scales, which facilitate the feature learning. Results:Our proposed methods finally get 5.7 % increase in mean average precision and 18.5 % increase in specificity with sacrifice of 2.6 % delay in inference time. Conclusions:Our proposed methods which totally avoid the dependence on segmentation of cervical cells, show the great potential to reduce the workload for pathologists in automation-assisted cervical cancer screening.

The microscopic urine sediment examination is a crucial part in the evaluation of renal and urinary tract diseases. Recently, there are emerging CNNs-based detectors to detect the urine sediment particles in an end-to-end manner. However, it is not very compatible to transfer CNNs-based detector directly from natural images application to microscopic images, especially in which small objects are in majority. This paper proposes a bidirectional context propagation network called BCPNet for urine sediment particle detection. In BCPNet, spatial details encoded by shallow convolutional layers are propagated upward to improve the localisation ability of deep features. On the contrary, high semantic information encoded by deep convolutional layers is propagated downward to enhance the distinctiveness of shallow features. With the refinement by convolutional block attention modules, the enriched features are more powerful to both localisation and classification. Experimental results on urine sediment particle dataset USE demonstrate effectiveness of the proposed BCPNet.

During the process of whole slide imaging, it is necessary to focus thousands of fields of view to obtain a high-quality image. To make the focusing procedure efficient and effective, we propose a novel autofocus algorithm for whole slide imaging. It is based on convolution and recurrent neural networks to predict the out-of-focus distance and subsequently update the focus location of the camera lens in an iterative manner. More specifically, we train a convolution neural network to extract focus information in the form of a focus feature vector. In order to make the prediction more accurate, we apply a recurrent neural network to combine focus information from previous search iteration and current search iteration to form a feature aggregation vector. This vector contains more focus information than the previous one and is subsequently used to predict the out-of-focus distance. Our experiments indicate that our proposed autofocus algorithm is able to rapidly determine the optimal in-focus image. The code is available at https://github.com/hezhujun/autofocus-rnn.

Nuclei segmentation is an essential step in DNA ploidy analysis by image-based cytometry (DNA-ICM) which is widely used in cytopathology and allows an objective measurement of DNA content (ploidy). The routine fully supervised learning-based method requires often tedious and expensive pixel-wise labels. In this paper, we propose a novel weakly supervised nuclei segmentation framework which exploits only sparsely annotated bounding boxes, without any segmentation labels. The key is to integrate the traditional image segmentation and self-training into fully supervised instance segmentation. We first leverage the traditional segmentation to generate coarse masks for each box-annotated nucleus to supervise the training of a teacher model, which is then responsible for both the refinement of these coarse masks and pseudo labels generation of unlabeled nuclei. These pseudo labels and refined masks along with the original manually annotated bounding boxes jointly supervise the training of student model. Both teacher and student share the same architecture and especially the student is initialized by the teacher. We have extensively evaluated our method with both our DNA-ICM dataset and public cytopathological dataset. Without bells and whistles, our method outperforms all existing weakly supervised entries on both datasets.

Identifying patterns in 2D linear projections is important in understanding multi-dimensional datasets. However, local patterns, which are composed of partial data points, are usually obscured by noises and missed in traditional quality measure approaches that measure the whole dataset. In this paper, we propose an interactive interface to explore 2D linear projections with visual patterns on subsets. First, we propose a voting-based algorithm to recommend optimal projection, in which the identified pattern looks the most salient. Specifically, we propose three kinds of point-wise quality metrics of 2D linear projections for outliers, clusterings, and trends, respectively. For each sampled projection, we measure its importance by accumulating the metrics of selected points. The projection with the highest importance is recommended. Second, we design an exploring interface with a scatterplot, a projection trail map, and a control panel. Our interface allows users to explore projections by specifying interested data subsets. At last, we employ three datasets and demonstrate the effectiveness of our approach through three case studies of exploring clusters, outliers, and trends.

While automation-assisted reading system can improve efficiency, their performance often relies on the success of accurate cell segmentation and hand-craft feature extraction. This paper presents an efficient and totally segmentation-free method for automated cervical cell screening that utilizes modern object detector to directly detect cervical cells or clumps, without the design of specific hand-crafted feature. Specifically, we use the state-of-the-art CNN-based object detection methods, YOLOv3, as our baseline model. In order to improve the classification performance of hard examples which are four highly similar categories, we cascade an additional task-specific classifier. We also investigate the presence of unreliable annotations and coped with them by smoothing the distribution of noisy labels. We comprehensively evaluate our methods on our test set which is consisted of 1014 annotated cervical cell images with size of 4000 × 3000 and complex cellular situation corresponding to 10 categories. Our model achieves 97.5% sensitivity (Sens) and 67.8% specificity (Spec) on cervical cell image-level screening. Moreover, we obtain a best mean average precision (mAP) of 63.4% on cervical cell-level diagnosis, and improve the average precision (AP) of hard examples which are the most valuable but most difficult to distinguish. Our automation-assisted cervical cell reading system not only achieves cervical cell image-level classification but also provides more detailed location and category reference information of abnormal cells. The results indicate feasible performance of our method, together with the efficiency and robustness, providing a new idea for future development of computer-assisted reading systems in clinical cervical screening.

Pedestrian detecting and tracking are critical techniques in video monitoring. However, real-time pedestrian detection is still challenging in surveillance videos with complex background. In existing frameworks, feature extractions are usually time-consuming to achieve high detection accuracy. In this paper, we propose to combine sparse static and dynamic features to improve the feature extraction speed while keeping high detection accuracy. Firstly, the static sparse feature is extracted using a fast feature pyramid in each frame. Secondly, sparse optical flow is used to extract sparse dynamic feature among successive frames. Thirdly, we combine the two types of feature in the Adaboost classification. Experiments show that the average miss rate of our approach is 17%. The detection rate is up to 22 fps in a Matlab implementation. It shows that our approach achieves optimal detection accuracy compared to the state-of-the-art real-time pedestrian detection algorithms.

Urine sediment examination (USE) is an important topic in kidney disease analysis and it is often the prerequisite for subsequent diagnostic procedures. We propose DFPN(Feature Pyramid Network with DenseNet) method to overcome the problem of class confusion in the USE images that it is hard to be solved by baseline model which is the state-of-the-art object detection model FPN with RoIAlign pooling. We explored the importance of two parts of baseline model for the USE cell detection. First, adding attention module in the network head, and the class-specific attention module has improved mAP by 0.7 points with pre-trained ImageNet model and 1.4 points with pre-trained COCO model. Next, we introduced DenseNet to the baseline model(DFPN) for cell detection in USE, so that the input of the network's head own multiple levels of semantic information, compared to the baseline model only has high-level semantic information. DFPN achieves top result with a mAP of 86.9% on USE test set after balancing between the classification loss and bounding-box regression loss, which improve 5.6 points compared to baseline model, and especially erythrocyte's AP is greatly improved from 65.4% to 93.8%, indicating class confusion has been basically resolved. And we also explore the impacts of training schedule and pre-trained model. Our method is promising for the development of automated USE.

Attributes of the retinal vessel play important role in systemic conditions and ophthalmic diagnosis. In this paper, a supervised method based on Extreme Learning Machine (ELM) is proposed to segment retinal vessel. Firstly, a set of 39-D discriminative feature vectors, consisting of local features, morphological features, phase congruency, Hessian and divergence of vector fields, is extracted for each pixel of the fundus image. Then a matrix is constructed for pixel of the training set based on the feature vector and the manual labels, and acts as the input of the ELM classifier. The output of classifier is the binary retinal vascular segmentation. Finally, an optimization processing is implemented to remove the region less than 30 pixels which is isolated from the retinal vascilar. The experimental results testing on the public Digital Retinal Images for Vessel Extraction (DRIVE) database demonstrate that the proposed method is much faster than the other methods in segmenting the retinal vessels. Meanwhile the average accuracy, sensitivity, and specificity are 0.9607, 0.7140 and 0.9868, respectively. Moreover the proposed method exhibits high speed and robustness on a new Retinal Images for Screening (RIS) database. Therefore it has potential applications for real-time computer-aided diagnosis and disease screening.

Segmentation of optic disc (OD) and optic cup (OC) is critical in automated fundus image analysis system. Existing state-of-the-arts focus on designing deep neural networks with one or multiple dense prediction branches. Such kind of designs ignore connections among prediction branches and their learning capacity is limited. To build connections among prediction branches, this paper introduces gradient boosting framework to deep classification model and proposes a gradient boosting network called BoostNet. Specifically, deformable side-output unit and aggregation unit with deep supervisions are proposed to learn base functions and expansion coefficients in gradient boosting framework. By stacking aggregation units in a deep-to-shallow manner, models’ performances are gradually boosted along deep to shallow stages. BoostNet achieves superior results to existing deep OD and OC segmentation networks on the public dataset ORIGA.

Diabetic retinopathy (DR) is one of the leading cause of blindness, but the classification of DR requires experienced ophthalmologist to distinguish the presence of various small features, which is time-consuming and difficult. Convolution neural network (CNN), which enables learning hierarchical and discriminative features without experiences of clinicians, is an alternative method to address the above issue. In this paper, we investigate four factors of employing deep CNN to DR classification problem, including network architectures, preprocessing, class imbalance and fine-tuning. The best performance we achieved is an accuracy of 79% opposed to an accuracy of 75% for Harry's scheme [14].

The appearance based facial tracking methods, such as active appearance models and candide models, are widely used in intelligent user interface and facial expression recognition. This paper proposes a novel method to detect and correct the failures in appearance based facial tracking. A sparse coding strategy is applied to learn an efficient feature representation for the difference between the warped image and the face template. The features are extracted by directly project the difference image to the space spanned by the dictionary of the parse coding. An iterative regression based method is proposed to detect and correct the failures according to the features. Experimental evaluation on an open dataset shows a global performance improvement of the tracking algorithm.

The urine sediment analysis of particles in microscopic images can assist physicians in evaluating patients with renal and urinary tract diseases. Manual urine sediment examination is labor-intensive, subjective and time-consuming, and the traditional automatic algorithms often extract the hand-crafted features for recognition. Instead of using the hand-crafted features, in this paper we propose to exploit convolutional neural network (CNN) to learn features in an end-to-end manner to recognize the urinary particle. We treat the urinary particle recognition as object detection and exploit two state-of-the-art CNN-based object detection methods, Faster R-CNN and single shot multibox detector (SSD), along with their variants for urinary particle recognition. We further investigate different factors involving these CNN-based methods to improve the performance of urinary particle recognition. We comprehensively evaluate these methods on a dataset consisting of 5,376 annotated images corresponding to 7 categories of urinary particle, i.e., erythrocyte, leukocyte, epithelial cell, crystal, cast, mycete, epithelial nuclei, and obtain a best mean average precision (mAP) of 84.1% while taking only 72 ms per image on a NVIDIA Titan X GPU.

The combination of optimal oriented flux and oriented flux antisymmetry measures is used to yield slenderized vessels.A thin-vessel centerline extraction method is proposed based on height ridge traversal and leaf node line-growing.Our method can effectively extract thin vessels.Our method can identify hepatic vein and portal vein. Background and objectiveAccurate segmentation of liver vessels from abdominal computer tomography angiography (CTA) volume is very important for liver-vessel analysis and living-related liver transplants. This paper presents a novel liver-vessel segmentation and identification method. MethodsFirstly, an anisotropic diffusion filter is used to smooth noise while preserving vessel boundaries. Then, based on the gradient symmetry and antisymmetry pattern of vessel structures, optimal oriented flux (OOF) and oriented flux antisymmetry (OFA) measures are respectively applied to detect liver vessels and their boundaries, and further to slenderize vessels. Next, according to vessel geometrical structure, a centerline extraction measure based on height ridge traversal and leaf node line-growing (LNLG) is proposed for the extraction of liver-vessel centerlines, and an intensity model based on fast marching is integrated into graph cuts (GCs) for effective segmentation of liver vessels. Finally, a distance voting mechanism is applied to separate the hepatic vein and portal vein. ResultsThe experiment results on abdominal CTA images show that the proposed method can effectively segment liver vessels, achieving an average accuracy, sensitivity, and specificity of 97.7%, 79.8%, and 98.6%, respectively, and has a good performance on thin-vessel extraction. ConclusionsThe proposed method does not require manual selection of the centerlines and vessel seeds, and can effectively segment liver vessels and identify hepatic vein and portal vein.