Search

Registration is a fundamental task in medical robotics and is often a crucial step for many downstream tasks such as motion analysis, intra-operative tracking and image segmentation. Popular registration methods such as ANTs and NiftyReg optimize objective functions for each pair of images from scratch, which are time-consuming for 3D and sequential images with complex deformations. Recently, deep learning-based registration approaches such as VoxelMorph have been emerging and achieve competitive performance. In this work, we construct a test-time training for deep deformable image registration to improve the generalization ability of conventional learning-based registration model. We design multi-scale deep networks to consecutively model the residual deformations, which is effective for high variational deformations. Extensive experiments validate the effectiveness of multi-scale deep registration with test-time training based on Dice coefficient for image segmentation and mean square error (MSE), normalized local cross-correlation (NLCC) for tissue dense tracking tasks. Two videos are in https://www.youtube.com/watch?v=NvLrCaqCiAE and https://www.youtube.com/watch?v=pEA6ZmtTNuQ, ICRA 2021; 8 pages, 4 figures, 2 big tables

Estimating 3D hand pose directly from RGB images is challenging but has gained steady progress recently by training deep models with annotated 3D poses. However annotating 3D poses is difficult and as such only a few 3D hand pose datasets are available, all with limited sample sizes. In this study, we propose a new framework of training 3D pose estimation models from RGB images without using explicit 3D annotations, i.e., trained with only 2D information. Our framework is motivated by two observations: 1) Videos provide richer information for estimating 3D poses as opposed to static images; 2) Estimated 3D poses ought to be consistent whether the videos are viewed in the forward order or reverse order. We leverage these two observations to develop a self-supervised learning model called temporal-aware self-supervised network (TASSN). By enforcing temporal consistency constraints, TASSN learns 3D hand poses and meshes from videos with only 2D keypoint position annotations. Experiments show that our model achieves surprisingly good results, with 3D estimation accuracy on par with the state-of-the-art models trained with 3D annotations, highlighting the benefit of the temporal consistency in constraining 3D prediction models.

Estimating 3D hand poses from a single RGB image is challenging because depth ambiguity leads the problem ill-posed. Training hand pose estimators with 3D hand mesh annotations and multi-view images often results in significant performance gains. However, existing multi-view datasets are relatively small with hand joints annotated by off-the-shelf trackers or automated through model predictions, both of which may be inaccurate and can introduce biases. Collecting a large-scale multi-view 3D hand pose images with accurate mesh and joint annotations is valuable but strenuous. In this paper, we design a spin match algorithm that enables a rigid mesh model matching with any target mesh ground truth. Based on the match algorithm, we propose an efficient pipeline to generate a large-scale multi-view hand mesh (MVHM) dataset with accurate 3D hand mesh and joint labels. We further present a multi-view hand pose estimation approach to verify that training a hand pose estimator with our generated dataset greatly enhances the performance. Experimental results show that our approach achieves the performance of 0.990 in $\text{AUC}_{\text{20-50}}$ on the MHP dataset compared to the previous state-of-the-art of 0.939 on this dataset. Our datasset is public available. \footnote{\url{https://github.com/Kuzphi/MVHM}} Our datasset is available at~\href{https://github.com/Kuzphi/MVHM}{\color{blue}{https://github.com/Kuzphi/MVHM}}.

The high irregularity of multiple sclerosis (MS) lesions in sizes and numbers often proves difficult for automated systems on the task of MS lesion segmentation. Current State-of-the-art MS segmentation algorithms employ either only global perspective or just patch-based local perspective segmentation approaches. Although global image segmentation can obtain good segmentation for medium to large lesions, its performance on smaller lesions lags behind. On the other hand, patch-based local segmentation disregards spatial information of the brain. In this work, we propose SynergyNet, a network segmenting MS lesions by fusing data from both global and local perspectives to improve segmentation across different lesion sizes. We achieve global segmentation by leveraging the U-Net architecture and implement the local segmentation by augmenting U-Net with the Mask R-CNN framework. The sharing of lower layers between these two branches benefits end-to-end training and proves advantages over simple ensemble of the two frameworks. We evaluated our method on two separate datasets containing 765 and 21 volumes respectively. Our proposed method can improve 2.55% and 5.0% for Dice score and lesion true positive rates respectively while reducing over 20% in false positive rates in the first dataset, and improve in average 10% and 32% for Dice score and lesion true positive rates in the second dataset. Results suggest that our framework for fusing local and global perspectives is beneficial for segmentation of lesions with heterogeneous sizes.

Hand pose estimation is more challenging than body pose estimation due to severe articulation, self-occlusion and high dexterity of the hand. Current approaches often rely on a popular body pose algorithm, such as the Convolutional Pose Machine (CPM), to learn 2D keypoint features. These algorithms cannot adequately address the unique challenges of hand pose estimation, because they are trained solely based on keypoint positions without seeking to explicitly model structural relationship between them. We propose a novel Nonparametric Structure Regularization Machine (NSRM) for 2D hand pose estimation, adopting a cascade multi-task architecture to learn hand structure and keypoint representations jointly. The structure learning is guided by synthetic hand mask representations, which are directly computed from keypoint positions, and is further strengthened by a novel probabilistic representation of hand limbs and an anatomically inspired composition strategy of mask synthesis. We conduct extensive studies on two public datasets - OneHand 10k and CMU Panoptic Hand. Experimental results demonstrate that explicitly enforcing structure learning consistently improves pose estimation accuracy of CPM baseline models, by 1.17% on the first dataset and 4.01% on the second one. The implementation and experiment code is freely available online. Our proposal of incorporating structural learning to hand pose estimation requires no additional training information, and can be a generic add-on module to other pose estimation models., The paper has be accepted and will be presented at 2020 IEEE Winter Conference on Applications of Computer Vision (WACV). The code is freely available at https://github.com/HowieMa/NSRMhand

Complex networks represented as node adjacency matrices constrains the application of machine learning and parallel algorithms. To address this limitation, network embedding (i.e., graph representation) has been intensively studied to learn a fixed-length vector for each node in an embedding space, where the node properties in the original graph are preserved. Existing methods mainly focus on learning embedding vectors to preserve nodes proximity, i.e., nodes next to each other in the graph space should also be closed in the embedding space, but do not enforce algebraic statistical properties to be shared between the embedding space and graph space. In this work, we propose a lightweight model, entitled Network2Vec, to learn network embedding on the base of semantic distance mapping between the graph space and embedding space. The model builds a bridge between the two spaces leveraging the property of group homomorphism. Experiments on different learning tasks, including node classification, link prediction, and community visualization, demonstrate the effectiveness and efficiency of the new embedding method, which improves the state-of-the-art model by 19% in node classification and 7% in link prediction tasks at most. In addition, our method is significantly faster, consuming only a fraction of the time used by some famous methods., 8 pages. 8 figures. Will appear at workshop on the conference ICDM 2020

Image-based virtual try-on systems with the goal of transferring a desired clothing item onto the corresponding region of a person have made great strides recently, but challenges remain in generating realistic looking images that preserve both body and clothing details. Here we present a new virtual try-on network, called VTNFP, to synthesize photo-realistic images given the images of a clothed person and a target clothing item. In order to better preserve clothing and body features, VTNFP follows a three-stage design strategy. First, it transforms the target clothing into a warped form compatible with the pose of the given person. Next, it predicts a body segmentation map of the person wearing the target clothing, delineating body parts as well as clothing regions. Finally, the warped clothing, body segmentation map and given person image are fused together for fine-scale image synthesis. A key innovation of VTNFP is the body segmentation map prediction module, which provides critical information to guide image synthesis in regions where body parts and clothing intersects, and is very beneficial for preventing blurry pictures and preserving clothing and body part details. Experiments on a fashion dataset demonstrate that VTNFP generates substantially better results than state-of-the-art methods.

Mass segmentation provides effective morphological features which are important for mass diagnosis. In this work, we propose a novel end-to-end network for mammographic mass segmentation which employs a fully convolutional network (FCN) to model a potential function, followed by a CRF to perform structured learning. Because the mass distribution varies greatly with pixel position, the FCN is combined with a position priori. Further, we employ adversarial training to eliminate over-fitting due to the small sizes of mammogram datasets. Multi-scale FCN is employed to improve the segmentation performance. Experimental results on two public datasets, INbreast and DDSM-BCRP, demonstrate that our end-to-end network achieves better performance than state-of-the-art approaches. \footnote{https://github.com/wentaozhu/adversarial-deep-structural-networks.git}, Comment: Accepted by ISBI2018. arXiv admin note: substantial text overlap with arXiv:1612.05970

Visual question answering (VQA) is of significant interest due to its potential to be a strong test of image understanding systems and to probe the connection between language and vision. Despite much recent progress, general VQA is far from a solved problem. In this paper, we focus on the VQA multiple-choice task, and provide some good practices for designing an effective VQA model that can capture language-vision interactions and perform joint reasoning. We explore mechanisms of incorporating part-ofspeech (POS) tag guided attention, convolutional n-grams, triplet attention interactions between the image, question and candidate answer, and structured learning for triplets based on image-question pairs 1. We evaluate our models on two popular datasets: Visual7W and VQA Real Multiple Choice. Our final model achieves the state-of-the-art performance of 68.2% on Visual7W, and a very competitive performance of 69.6% on the test-standard split of VQA Real Multiple Choice.

Due to its indispensability for device-free passive (DfP) sensing, DfP human detection has attracted numerous research efforts during the past years. Although previous works have achieved considerable detection performance, they mainly focus on moving human detection, making mobility a prerequisite for reliable detection. Besides, existing static human detection systems usually require dense deployment or controlled settings. In this paper, we propose a robust respiration-rate-estimation-based passive static human detection system, R-PSHD. Specifically, different from recent works which leverage the amplitude of channel state information (CSI) for DfP sensing, we resort to the more sensitive phase information for minute respiration detection. To deal with the randomness of raw phase, R-PSHD exploits phase difference between antennas for feature extraction. Moreover, due to varying sensitivity of different subcarriers, R-PSHD tries to identify the useful subcarriers and only uses them for accurate estimation. Experimental results with different people during a week demonstrate that R-PSHD achieves great performance with both TP and TN rate higher than 90%.

WiFi technology has fostered numerous mobile computing applications, e.g. indoor localization, gesture and activity recognition, device-free localization, etc., due to its ubiquity. The awareness of LOS and NLOS is a prerequisite for WiFi-based methods, since the WiFi signals received under NLOS conditions may contain a lot of noise and multipath effects, exerting great influences on the accuracy of location or identification. Traditional schemes based on commodity WiFi devices can achieve real-time LOS/NLOS identification. However, these methods face the challenges of limited bandwidth and coarse multipath resolution. In this work, we explore the amplitude feature of PHY layer information, and accordingly propose AmpN, a real-time LOS identification scheme based on commodity WiFi infrastructure that is applicable in both static and mobile scenarios. AmpN employs BP neural network algorithm in static scenario and K-Mean method in dynamic scenario, respectively. Experimental results demonstrate that AmpN outperforms existing approaches, achieving overall LOS and NLOS detection rates of 94.2% and 97.6% in static case, and above 97% LOS and NLOS detection rates in mobile context. In addition, the detection delay is less than 0.4s when the link state switches from LOS to NLOS.

Wi-Fi has been widely deployed with the rapid development of wireless communication technique. Wi-Fi hotspots are popular in campus, making convenient wireless network access possible. However, during class teaching, in order to avoid students browsing the web based on Wi-Fi hotspots and distracting, we hope Wi-Fi hotspots adaptively shield network access to students in the classroom, while grant access to users outside the classroom. In this work, we prototype TA3C system, a teaching-oriented adaptive Wi-Fi authorized access control scheme, using Channel State Information (CSI) to locate instead of coarse-grained and temporally unstable Received Signal Strength Indication (RSSI). CSI can distinguish multipath signals, stay stable in the same propagation environment, and show different characteristics in different propagation environments, based on which, we can distinguish outdoor and indoor environments, and identify user's location to decide whether to offer him wireless network access or not. Experiment results show that TA3C effectively achieve adaptive Wi-Fi authorized access control, which provides a guarantee for the quality of class teaching. Compared to traditional indoor localization techniques, TA3C does not need accurate location information but simply recognizes user's location indoors or outdoors, which means it does not need dedicated hardware, realizing the low-cost indoor localization technique.

A robotic grinding method based on force feedback is proposed in this paper. This method is used in polishing metal products like faucet handle which is made in zinc alloy. Force control technology is important in the grinding system. In the grinding process, grinding force is a pressure in a specified direction which is applied to the surface of the mold by polishing tools. The force should be controlled in some stable value to ensure the grinding quality. In order to maintain a constant and stable force throughout the process of grinding or polishing, force sensor is always required in order to measure the contact force between grinding tool and mold surface. At the same time, the measured data should be sent to the controller and according to it the controller can adjust the value of the predefined grinding force in different process conditions. Before robotic grinding, off-line programming method is used to simulate the grinding process. After simulation process, the grinding robot with force control method is applied in grinding faucet handle instead of human. Good grinding results are produced from the experiments with such methods.

Recent advances in DNA sequencing have enabled a flood of sequencing-based applications for studying biology and medicine. A key requirement of these applications is to rapidly and accurately map DNA subsequences to a reference genome. This DNA subsequence mapping problem shares core technical challenges with the similarity query processing problem studied in the database research literature. To solve this problem, existing techniques first extract signatures from a query, then retrieve candidate mapping positions from an index using the extracted signatures, and finally verify the candidate positions. The efficiency of these techniques depends critically on signatures selected from queries, while signature selection relies on an indexing scheme of a reference genome. The q-gram inverted indexing, one of the most widely used indexing schemes, can discover candidate positions quickly, but has the limitation that signatures of queries are restricted to fixed-length q-grams. To address the problem, we propose a flexible way to generate variable-length signatures using a fixed-length q-gram index. The proposed technique groups a few q-grams into a variable-length signature, and generates candidate positions for the variable-length signature using the inverted lists of the q-grams. We also propose a novel dynamic programming algorithm to balance between the filtering power of signatures and the overhead of generating candidate positions for the signatures. Through extensive experiments on both simulated and real genomic data, we show that our technique substantially improves the performance of read mapping in terms of both mapping speed and accuracy.

Robust Device-free passive (Dfp) detection is an essential primitive for a broad range of applications such as intrusion detection and smart space. Most recent works focus on finer-grained Channel State Information (CSI), instead of the variable Received Signal Strength (RSS). However, existing solutions have some limitations, being feasible only in the line of sight (LOS) or for more than one targeted entities. Moreover, space diversity supported by the MIMO systems hasn't been fully investigated. Motivated by this observation, we propose a novel scheme for Robust Passive Motion Detection (R-PMD). In our scheme, the variance of CSI amplitude feature is extracted as a new metric and the earth mover's distance (EMD) is utilized to determine the detection results. Besides, CSIs across multiantennas are further exploited to improve the detection precision and robustness. We prototype R-PMD on commercial WiFi devices and evaluate it in a typical indoor scenario. Experiment results show R-PMD can achieve great performance in terms of sensitivity and robustness.

The explosive growth in the amount of data produced by next-generation sequencing poses significant computational challenges on how to store, transmit and query these data, efficiently and accurately. A unique characteristic of the genomic sequence data is that many of them can be highly similar to each other, which has motivated the idea of compressing sequence data by storing only their differences to a reference sequence, thereby drastically cutting the storage cost. However, an unresolved question in this area is whether it is possible to perform search directly on the compressed data, and if so, how. Here we show that directly querying compressed genomic sequence data is possible and can be done efficiently. We describe a set of novel index structures and algorithms for this purpose, and present several optimization techniques to reduce the space requirement and query response time. We demonstrate the advantage of our method and compare it against existing ones through a thorough experimental study on real genomic data.

Considering that the lifespan of electrolytic capacitors in power supply don't match that of LED, we propose a Buck converter which uses active ripple compensation circuit to replace the traditional electrolytic capacitor filter. When the converter works on long duty ratio, the power loss in compensation circuit is small and ripple current and voltage are low. The operation principle of this new topology with peak current mode control (PCM) is discussed. The transfer function of PCM with slope compensation is established. The current loop stability with different slope compensation is discussed through frequency response analysis. Finally, simulation and experiment results of the power supply for LED verified correctness of the theory.

This paper presents a multi-axis control system of bar tacker. A bar tacker must meet the following requirements: feeding accurately, cutting automatically, needle stopping precisely and quickly, et ac. In order to achieve these functions, multi-thread synchronization control was introduced to ensure the multi-axis collaborative motion. And time-based PI method with velocity feed-forward was used to control the needle stop position. The software of this control system was programmed based on embedded Windows platform. At present this control system has been tested in the factory and the test results show that this method can accurately control the multi-axis motion and produce good sewing patterns. Besides lockstitch bar tacker, this control system can also be applied to other intelligent sewing machines by minor changes and improvement.

Relevance feedback (RF) is an effective method for content-based image retrieval (CBIR), and it is also a feasible step to shorten the semantic gap between low-level visual feature and high-level perception. In this paper, a SVM-based RF algorithm is proposed to improve performance of image retrieval. In classifier training, a sample expanding scheme is adopted to balance the proportion of positive samples and negative samples. And then, a fusion scheme for multiple classifiers based on adaptive weighting is proposed to vote the final query results. The experimental results on Corel image dataset show the effectiveness of the proposed algorithm.

Content-based copy detection (CBCD) recently has appeared a promising technique for video monitoring and copyright protection. In this paper, a novel framework for CBCD is proposed. Robust global features and local Speeded Up Robust Features (SURF) are first combined to describe video contents, and the density sampling method is proposed to improve the generation of visual codebook. Secondly, Smith-Waterman algorithm is introduced to find the similar video segments, meanwhile, a video matching method based on visual codebook is proposed to calculate the similarity of copy videos. Finally, a hierarchical fusion scheme is used to refine the detection results. Experiments on TRECVID dataset show that the proposed framework gives better results than the average results of CBCD task in TRECVID 2008.

A novel hierarchical fingerprint matching algorithm with low complexity is proposed, which is suitable for resource-constrained applications. By analyzing different support levels from coarse to fine matching based on the non-fixed local minutiae spatial relationships, the matching score is chosen from the sparse matrix representing the similarity. Experimental results show that the performance of the proposed algorithm is efficient and that it can be used in real resource-constrained platforms. We have implemented our fingerprint verification system using the proposed preprocessing and matching algorithms on the embedded platform including ARM920T (SUMSUNG S3C2410) and DSP (TMS320VC5509A).

Content-based video copy detection aims at deciding whether there is a common segment between the query video and the video in the database. In this paper, a copy detection system is proposed based on local features that can deal with most video transformations and realize video searching in the database by using inverted file. Local features are first extracted and then clustered to visual words as index of the inverted file. The voting strategy makes use of the property of temporal consistence. The experimental results indicate that these visual features are robust and the searching in database is feasible.

In this paper, a novel framework for semantic-based video retrieval is proposed. 15 low-level visual features on different levels are extracted and a supervised SVM classifier is trained for each feature. We have explored early fusion schemes between SIFT and SURF, and evaluated 4 kinds of later fusion strategies. Experiments on TRECVID dataset show that the proposed system is effective and stable.

We present our findings on using coherent anti-Stokes Raman scattering (CARS) microscopy to image brain tissue slices. Compared with other modalities such as confocal and two-photon laser scanning microscopy, CARS microscopy offers chemical selectivity with high sensitivity without the need for any labeling agents. CARS microscopy uses two laser frequencies, whose energy difference is tuned to target a specific molecular vibration. This creates a vibrational coherence that, when probed, can give rise to a substantial chemically-selective signal. CARS overcomes the drawback of weak inelastic scattering of conventional Raman spectroscopy. As a modality that uses the intrinsic chemical selectivity to image specimen, CARS avoids the photobleaching problem and perturbations to cell functions induced by fluorescent proteins. It can acquire three-dimensional images with high resolution, in addition to the high sensitivity and chemical selectivity. In this work, we demonstrate the performance of using CARS to acquire images of mouse brain tissues and compare it with standard histology images

This paper describes a new method for shape detection based on the radial symmetry nature and direction-discriminated voting. Multiple shapes including circles, regular and non-regular polygons can be detected under a general framework. The novelty of our approach is that different shapes can be simultaneously located and classified. It is implemented by taking account of both voting accumulations and voting directions. We show that the approach can reduce false detection and computation burden compared to some existing methods. Moreover, by modeling a shape based on its partial radial symmetry characteristics and the geometrical relationship among the symmetry centers, our approach is extended to detect non-regular polygons. Experiments on traffic sign detection demonstrate good performance of our method.

A robotic tele-drill system is constructed based on a robotic telesurgery system. The system is in client/ server structure. Client part includes main control interface and video, audio interface and predictive display interface. Server part includes robot control server and video, audio server. For applying into teleoperation, a virtual reality environment of robotic system developed by using JAVA, JAVA 3D and PRO/ E etc is finished. The geometry and kinematics model of serial robot MOTOMAN sv3x, parallel robot, C-type arm and X-ray machine, surgery bed and its work environment are fulfilled in it. Simulation engine and its simulation syntax are finished, which made the environment controllable. This environment is used as a new type predictive display interface in the telerobotics in order to tackling the problem in visual feedback as ambiguous or time delay. Experiments that verified feasible of the system have been done.

Safety undoubtedly is important in telesurgery systems. Force feedback is assumed to enhance performance in robotic surgery and even render some tasks feasible but it is hard to control. Huge and unpredictable time delay over the Internet may induce close-loop control to be unstable. The use of QOS (Quality of Service) networks and protocols is proved to be an effective means in ensuring the safety of telesurgery, but current Internet is unstable, for example, variable time delay and package losses. TCP is a more reliable protocol than UDP, but UDP is proved to be the better protocol used in real-time case than TCP. So using different protocol for different data is necessary in telesurgery. Supervisory control and predictor display are proved to be effective methods in time delay compensation. In order to ensure force-feedback system stable, event-based and wave variable method are used. Based on such rules above, prototype of a bonesetting system is constructed.