Search

Pancreatic cancer is projected to become the second leading cause of cancer-related mortality in the United States by 2030. This is in part due to the paucity of reliable screening and diagnostic options for early detection. Amongst known pre-malignant pancreatic lesions, pancreatic intraepithelial neoplasia (PanIN) and intraductal papillary mucinous neoplasms (IPMNs) are the most prevalent. The current standard of care for the diagnosis and classification of pancreatic cystic lesions (PCLs) involves cross-sectional imaging studies and endoscopic ultrasound (EUS) and, when indicated, EUS-guided fine needle aspiration and cyst fluid analysis. However, this is suboptimal for the identification and risk stratification of PCLs, with accuracy of only 65–75% for detecting mucinous PCLs. Artificial intelligence (AI) is a promising tool that has been applied to improve accuracy in screening for solid tumors, including breast, lung, cervical, and colon cancer. More recently, it has shown promise in diagnosing pancreatic cancer by identifying high-risk populations, risk-stratifying premalignant lesions, and predicting the progression of IPMNs to adenocarcinoma. This review summarizes the available literature on artificial intelligence in the screening and prognostication of precancerous lesions in the pancreas, and streamlining the diagnosis of pancreatic cancer.

Object segmentation in three-dimensional (3-D) point clouds is a critical task for robots capable of 3-D perception. Despite the impressive performance of deep learning-based approaches on object segmentation in 2-D images, deep learning has not been applied nearly as successfully for 3-D point cloud segmentation. Deep networks generally require large amounts of labeled training data, which are readily available for 2-D images but are difficult to produce for 3-D point clouds. In this letter, we present Label Diffusion Lidar Segmentation (LDLS), a novel approach for 3-D point cloud segmentation, which leverages 2-D segmentation of an RGB image from an aligned camera to avoid the need for training on annotated 3-D data. We obtain 2-D segmentation predictions by applying Mask-RCNN to the RGB image, and then link this image to a 3-D lidar point cloud by building a graph of connections among 3-D points and 2-D pixels. This graph then directs a semi-supervised label diffusion process, where the 2-D pixels act as source nodes that diffuse object label information through the 3-D point cloud, resulting in a complete 3-D point cloud segmentation. We conduct empirical studies on the KITTI benchmark dataset and on a mobile robot, demonstrating wide applicability and superior performance of LDLS compared with the previous state of the art in 3-D point cloud segmentation, without any need for either 3-D training data or fine tuning of the 2-D image segmentation model., Comment: Accepted for publication in IEEE Robotics and Automation Letters with presentation at IROS 2019

Objective To examine the robustness of the published machine learning models in the prediction of extraction vs non-extraction for a diverse US sample population seen by multiple providers. Setting and sample population Diverse group of 838 patients (208 extraction, 630 non-extraction) were consecutively enrolled. Materials and methods Two sets of input features (117 and 22) including clinical and cephalometric variables were identified based on previous studies. Random forest (RF) and multilayer perception (MLP) models were trained using these feature sets on the sample population and evaluated using measures including accuracy (ACC) and balanced accuracy (BA). A technique to identify incongruent data was used to explore underlying characteristics of the data set and split all samples into 2 groups (G1 and G2) for further model training. Results Performance of the models (75%-79% ACC and 72%-76% BA) on the total sample population was lower than in previous research. Models were retrained and evaluated using G1 and G2 separately, and individual group MLP models yielded improved accuracy for G1 (96% ACC and 94% BA) and G2 (88% ACC and 85% BA). RF feature ranking showed differences between top features for G1 (maxillary crowding, mandibular crowding and L1-NB) and G2 (age, mandibular crowding and lower lip to E-plane). Conclusions An incongruent data pattern exists in a consecutively enrolled patient population. Future work with incongruent data segregation and advanced artificial intelligence algorithms is needed to improve the generalization ability to make it ready to support clinical decision-making.

Background and Aims EUS-guided needle-based confocal laser endomicroscopy (EUS-nCLE) can differentiate high-grade dysplasia/adenocarcinoma (HGD-Ca) in intraductal papillary mucinous neoplasms (IPMNs) but requires manual interpretation. We sought to derive predictive computer-aided diagnosis (CAD) and artificial intelligence (AI) algorithms to facilitate accurate diagnosis and risk stratification of IPMNs. Methods A post hoc analysis of a single-center prospective study evaluating EUS-nCLE (2015-2019; INDEX study) was conducted using 15,027 video frames from 35 consecutive patients with histopathologically proven IPMNs (18 with HGD-Ca). We designed 2 CAD-convolutional neural network (CNN) algorithms: (1) a guided segmentation-based model (SBM), where the CNN-AI system was trained to detect and measure papillary epithelial thickness and darkness (indicative of cellular and nuclear stratification), and (2) a reasonably agnostic holistic-based model (HBM) where the CNN-AI system automatically extracted nCLE features for risk stratification. For the detection of HGD-Ca in IPMNs, the diagnostic performance of the CNN-CAD algorithms was compared with that of the American Gastroenterological Association (AGA) and revised Fukuoka guidelines. Results Compared with the guidelines, both n-CLE-guided CNN-CAD algorithms yielded higher sensitivity (HBM, 83.3%; SBM, 83.3%; AGA, 55.6%; Fukuoka, 55.6%) and accuracy (SBM, 82.9%; HBM, 85.7%; AGA, 68.6%; Fukuoka, 74.3%) for diagnosing HGD-Ca, with comparable specificity (SBM, 82.4%; HBM, 88.2%; AGA, 82.4%; Fukuoka, 94.1%). Both CNN-CAD algorithms, the guided (SBM) and agnostic (HBM) models, were comparable in risk stratifying IPMNs. Conclusion EUS-nCLE-based CNN-CAD algorithms can accurately risk stratify IPMNs. Future multicenter validation studies and AI model improvements could enhance the accuracy and fully automatize the process for real-time interpretation.

Reliable and accurate 3D object detection is a necessity for safe autonomous driving. Although LiDAR sensors can provide accurate 3D point cloud estimates of the environment, they are also prohibitively expensive for many settings. Recently, the introduction of pseudo-LiDAR (PL) has led to a drastic reduction in the accuracy gap between methods based on LiDAR sensors and those based on cheap stereo cameras. PL combines state-of-the-art deep neural networks for 3D depth estimation with those for 3D object detection by converting 2D depth map outputs to 3D point cloud inputs. However, so far these two networks have to be trained separately. In this paper, we introduce a new framework based on differentiable Change of Representation (CoR) modules that allow the entire PL pipeline to be trained end-to-end. The resulting framework is compatible with most state-of-the-art networks for both tasks and in combination with PointRCNN improves over PL consistently across all benchmarks -- yielding the highest entry on the KITTI image-based 3D object detection leaderboard at the time of submission. Our code will be made available at https://github.com/mileyan/pseudo-LiDAR_e2e., Comment: Accepted to 2020 Conference on Computer Vision and Pattern Recognition (CVPR 2020)

In the domain of autonomous driving, deep learning has substantially improved the 3D object detection accuracy for LiDAR and stereo camera data alike. While deep networks are great at generalization, they are also notorious to over-fit to all kinds of spurious artifacts, such as brightness, car sizes and models, that may appear consistently throughout the data. In fact, most datasets for autonomous driving are collected within a narrow subset of cities within one country, typically under similar weather conditions. In this paper we consider the task of adapting 3D object detectors from one dataset to another. We observe that naively, this appears to be a very challenging task, resulting in drastic drops in accuracy levels. We provide extensive experiments to investigate the true adaptation challenges and arrive at a surprising conclusion: the primary adaptation hurdle to overcome are differences in car sizes across geographic areas. A simple correction based on the average car size yields a strong correction of the adaptation gap. Our proposed method is simple and easily incorporated into most 3D object detection frameworks. It provides a first baseline for 3D object detection adaptation across countries, and gives hope that the underlying problem may be more within grasp than one may have hoped to believe. Our code is available at https://github.com/cxy1997/3D_adapt_auto_driving., Accepted to 2020 Conference on Computer Vision and Pattern Recognition (CVPR 2020)

In this work, we introduce VQA 360, a novel task of visual question answering on 360 images. Unlike a normal field-of-view image, a 360 image captures the entire visual content around the optical center of a camera, demanding more sophisticated spatial understanding and reasoning. To address this problem, we collect the first VQA 360 dataset, containing around 17,000 real-world image-question-answer triplets for a variety of question types. We then study two different VQA models on VQA 360, including one conventional model that takes an equirectangular image (with intrinsic distortion) as input and one dedicated model that first projects a 360 image onto cubemaps and subsequently aggregates the information from multiple spatial resolutions. We demonstrate that the cubemap-based model with multi-level fusion and attention diffusion performs favorably against other variants and the equirectangular-based models. Nevertheless, the gap between the humans' and machines' performance reveals the need for more advanced VQA 360 algorithms. We, therefore, expect our dataset and studies to serve as the benchmark for future development in this challenging task. Dataset, code, and pre-trained models are available online., Accepted to WACV 2020

In this work, we consider the problem of searching people in an unconstrained environment, with natural language descriptions. Specifically, we study how to systematically design an algorithm to effectively acquire descriptions from humans. An algorithm is proposed by adapting models, used for visual and language understanding, to search a person of interest (POI) in a principled way, achieving promising results without the need to re-design another complicated model. We then investigate an iterative question-answering (QA) strategy that enable robots to request additional information about the POI's appearance from the user. To this end, we introduce a greedy algorithm to rank questions in terms of their significance, and equip the algorithm with the capability to dynamically adjust the length of human-robot interaction according to model's uncertainty. Our approach is validated not only on benchmark datasets but on a mobile robot, moving in a dynamic and crowded environment., Comment: 8 pages, 12 figures, Published in IEEE Robotics and Automation Letters (RA-L), "Dataset at: https://github.com/vikshree/QA_PersonSearchLanguageData" , Video attachment at: https://www.youtube.com/watch?v=Yyxu8uVUREE&feature=youtu.be

Person re-identification aims to identify a person from an image collection, given one image of that person as the query. There is, however, a plethora of real-life scenarios where we may not have a priori library of query images and therefore must rely on information from other modalities. In this paper, an attribute-based approach is proposed where the person of interest (POI) is described by a set of visual attributes, which are used to perform the search. We compare multiple algorithms and analyze how the quality of attributes impacts the performance. While prior work mostly relies on high precision attributes annotated by experts, we conduct a human-subject study and reveal that certain visual attributes could not be consistently described by human observers, making them less reliable in real applications. A key conclusion is that the performance achieved by non-expert attributes, instead of expert-annotated ones, is a more faithful indicator of the status quo of attribute-based approaches for person re-identification., Comment: Accepted by RO-MAN 2019, 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). IEEE, 2019

Research in computer-aided diagnosis (CAD) and the application of artificial intelligence (AI) in the endoscopic evaluation of the gastrointestinal tract is novel. Since colonoscopy and detection of polyps can decrease the risk of colon cancer, it is recommended by multiple national and international societies. However, the procedure of colonoscopy is performed by humans where there are significant interoperator and interpatient variations, and hence, the risk of missing detection of adenomatous polyps. Early studies involving CAD and AI for the detection and differentiation of polyps show great promise. In this appraisal, we review existing scientific aspects of AI in CAD of colon polyps and discuss the pitfalls and future directions for advancing the science. This review addresses the technical intricacies in a manner that physicians can comprehend to promote a better understanding of this novel application.

We investigate the problem of cross-dataset adaptation for visual question answering (Visual QA). Our goal is to train a Visual QA model on a source dataset but apply it to another target one. Analogous to domain adaptation for visual recognition, this setting is appealing when the target dataset does not have a sufficient amount of labeled data to learn an "in-domain" model. The key challenge is that the two datasets are constructed differently, resulting in the cross-dataset mismatch on images, questions, or answers. We overcome this difficulty by proposing a novel domain adaptation algorithm. Our method reduces the difference in statistical distributions by transforming the feature representation of the data in the target dataset. Moreover, it maximizes the likelihood of answering questions (in the target dataset) correctly using the Visual QA model trained on the source dataset. We empirically studied the effectiveness of the proposed approach on adapting among several popular Visual QA datasets. We show that the proposed method improves over baselines where there is no adaptation and several other adaptation methods. We both quantitatively and qualitatively analyze when the adaptation can be mostly effective., Accepted at CVPR 2018

3D object detection is an essential task in autonomous driving. Recent techniques excel with highly accurate detection rates, provided the 3D input data is obtained from precise but expensive LiDAR technology. Approaches based on cheaper monocular or stereo imagery data have, until now, resulted in drastically lower accuracies --- a gap that is commonly attributed to poor image-based depth estimation. However, in this paper we argue that it is not the quality of the data but its representation that accounts for the majority of the difference. Taking the inner workings of convolutional neural networks into consideration, we propose to convert image-based depth maps to pseudo-LiDAR representations --- essentially mimicking the LiDAR signal. With this representation we can apply different existing LiDAR-based detection algorithms. On the popular KITTI benchmark, our approach achieves impressive improvements over the existing state-of-the-art in image-based performance --- raising the detection accuracy of objects within the 30m range from the previous state-of-the-art of 22% to an unprecedented 74%. At the time of submission our algorithm holds the highest entry on the KITTI 3D object detection leaderboard for stereo-image-based approaches. Our code is publicly available at https://github.com/mileyan/pseudo_lidar., Comment: Accepted by CVPR 2019

Leveraging class semantic descriptions and examples of known objects, zero-shot learning makes it possible to train a recognition model for an object class whose examples are not available. In this paper, we propose a novel zero-shot learning model that takes advantage of clustering structures in the semantic embedding space. The key idea is to impose the structural constraint that semantic representations must be predictive of the locations of their corresponding visual exemplars. To this end, this reduces to training multiple kernel-based regressors from semantic representation-exemplar pairs from labeled data of the seen object categories. Despite its simplicity, our approach significantly outperforms existing zero-shot learning methods on standard benchmark datasets, including the ImageNet dataset with more than 20,000 unseen categories., ICCV2017 camera-ready

This paper provides a new age estimation approach, which distinguishes itself with the following three contributions. First, we combine distance metric learning and dimensionality reduction to better explore the connections between facial features and age labels. Second, to exploit the intrinsic ordinal relationship among human ages and overcome the potential data imbalance problem, a label-sensitive concept and several imbalance treatments are introduced in the system training phase. Finally, an age-oriented local regression is presented to capture the complicated facial aging process for age determination. The simulation results show that our approach achieves the lowest estimation error against existing methods.

Video summarization has unprecedented importance to help us digest, browse, and search today's ever-growing video collections. We propose a novel subset selection technique that leverages supervision in the form of human-created summaries to perform automatic keyframe-based video summarization. The main idea is to nonparametrically transfer summary structures from annotated videos to unseen test videos. We show how to extend our method to exploit semantic side information about the video's category/genre to guide the transfer process by those training videos semantically consistent with the test input. We also show how to generalize our method to subshot-based summarization, which not only reduces computational costs but also provides more flexible ways of defining visual similarity across subshots spanning several frames. We conduct extensive evaluation on several benchmarks and demonstrate promising results, outperforming existing methods in several settings., CVPR 2016 camera ready

We propose a novel supervised learning technique for summarizing videos by automatically selecting keyframes or key subshots. Casting the task as a structured prediction problem, our main idea is to use Long Short-Term Memory (LSTM) to model the variable-range temporal dependency among video frames, so as to derive both representative and compact video summaries. The proposed model successfully accounts for the sequential structure crucial to generating meaningful video summaries, leading to state-of-the-art results on two benchmark datasets. In addition to advances in modeling techniques, we introduce a strategy to address the need for a large amount of annotated data for training complex learning approaches to summarization. There, our main idea is to exploit auxiliary annotated video summarization datasets, in spite of their heterogeneity in visual styles and contents. Specifically, we show that domain adaptation techniques can improve learning by reducing the discrepancies in the original datasets’ statistical properties.

Given semantic descriptions of object classes, zero-shot learning aims to accurately recognize objects of the unseen classes, from which no examples are available at the training stage, by associating them to the seen classes, from which labeled examples are provided. We propose to tackle this problem from the perspective of manifold learning. Our main idea is to align the semantic space that is derived from external information to the model space that concerns itself with recognizing visual features. To this end, we introduce a set of "phantom" object classes whose coordinates live in both the semantic space and the model space. Serving as bases in a dictionary, they can be optimized from labeled data such that the synthesized real object classifiers achieve optimal discriminative performance. We demonstrate superior accuracy of our approach over the state of the art on four benchmark datasets for zero-shot learning, including the full ImageNet Fall 2011 dataset with more than 20,000 unseen classes.

In this paper, a robust illuminant estimation algorithm for color constancy is proposed. Considering the drawback of the well-known max-RGB algorithm, which regards only pixels with the maximum image intensities, we explore the representative pixels from an image for illuminant estimation: The representative pixels are determined via the intensity bounds corresponding to a certain percentage value in the normalized accumulative histograms. To achieve the suitable percentage, an iterative algorithm is presented by simultaneously neutralizing the chromaticity distribution and preventing overcorrection. The experimental results on the benchmark databases provided by Simon Fraser University and Microsoft Research Cambridge, as well as several web images, demonstrate the effectiveness of our approach.

Clinical examination of Congenital Muscular Torticollis (CMT) is often carried out by ultrasound equipments. However, a variety of subjective factors during diagnosis may result in wrong decision. Thus, we propose an image processing algorithm to derive the objective judgment on the healthiness of muscle in this paper. We first apply image segmentation technique, such as the fast scanning algorithm, for ultrasonic muscle image segmentation. Then, the proposed algorithms are applied to determine the healthiness of muscle fibers. We furthermore propose a score criterion to evaluate the degree of injury. The experimental results show that the injury score measured by the proposed methods can successfully determine whether the muscle is hurt and infer the extent of fibrosis.

In this paper, a new video retargeting method based on temporal information and seam carving is presented. Two video energy functions, motion weight prediction and pixel-based optimization, are proposed to take the temporal information into account and make dynamic programming available during the process of retargeting. The motion weight prediction exploits both the block-based motion estimation and Gaussian masks to predict the coarse location of seams in the current frame and reduce the search range of dynamic programming. The pixel-based optimization then utilizes the concept of pixel-based optical flow to explore better temporal relations between the current frame and previous frames in the reduced search range. The experimental results show that combining these two video energy functions as well as dynamic programming, the proposed method could achieve content-aware and temporal smoothing retargeting results with less computational complexity.

The Fourier descriptor is an efficient and effective way to describe a closed boundary. However, for a non-closed segment, since the two non-adjacent end points result in signal discontinuity, after eliminating the high-frequency part, the reconstructed segment has large error near the two ends. In this paper, we propose a warping method to connect the two ends and perform odd-symmetric extension to smooth the warped segment around them. With these modifications, the high-frequency components near the two ends can be much reduced and we can obtain the reconstructed segment with accurate end-point locations even when only the low frequency coefficients are preserved. This method could also be used for a closed boundary with a pre-segmentation process, and the experimental result shows that with the same boundary compression rate, our method has better reconstruction quality than directly extracting Fourier descriptors on the closed boundary.

The aims of this study were to investigate (i) if and when the blood pressure would rise or fall and (ii) the associated changes of human heart rate variability (HRV) by manual stimulation of the Neiguan (PC 6) acupuncture site. In this paper, two groups of six healthy male volunteers with ranges of ages 20-56 and 20-55 and with no neurological diseases participated in this study. In order to minimize artefacts, the electrocardiogram (ECG) and radial arterial pulse pressure wave were collected with the subjects alert but eyes closed before, during, and after sham/manual acupuncture. No statistically significant changes (P0.05) were found in the sham acupuncture group. As for the manual acupuncture group, the needle was inserted into the PC 6 acupoint and manually stimulated about 15 to 30 seconds to achieve De Qi sensation. Needles were left in place for 30 min and then removed. Analysis of the data due to acupuncture was then compared with the baseline values. Results indicate that the blood pressures of different subject can either rise (P0.01) or fall (P0.01). To further determine the indicator for one subject who exhibited both rise and fall of blood pressures, 7 more trials were given conducted with the same protocol until statistically significant results were obtained (P0.01). We found that his change of blood pressure was highly correlated (p = -0.94 and -0.99 for rise and fall, respectively) with the ratio of the magnitude of pulse pressure to that of the dicrotic notch in the local radial pulse wave (P0.01). As to the heart rate variability (HRV) spectra, significant changes in the low frequency (LF) and very low frequency (VLF) ranges were also detected. These results indicate that the autonomic innervations of heart have been modified. However, the information on the power of LF, high frequency (HF), and LF/HF of HRV are not conclusive to statistically differentiate the sympathetic contribution from that of the parasympathetic nervous systems at present stage.