Search

Automatic cell tracking in dense environments is plagued by inaccurate correspondences and misidentification of parent-offspring relationships. In this paper, we introduce a novel cell tracking algorithm named DenseTrack, which integrates deep learning with mathematical model-based strategies to effectively establish correspondences between consecutive frames and detect cell division events in crowded scenarios. We formulate the cell tracking problem as a deep learning-based temporal sequence classification task followed by solving a constrained one-to-one matching optimization problem exploiting the classifier's confidence scores. Additionally, we present an eigendecomposition-based cell division detection strategy that leverages knowledge of cellular geometry. The performance of the proposed approach has been evaluated by tracking densely packed cells in 3D time-lapse image sequences of bacterial biofilm development. The experimental results on simulated as well as experimental fluorescence image sequences suggest that the proposed tracking method achieves superior performance in terms of both qualitative and quantitative evaluation measures compared to recent state-of-the-art cell tracking approaches.

This paper presents a novel spatiotemporal transformer network that introduces several original components to detect actions in untrimmed videos. First, the multi-feature selective semantic attention model calculates the correlations between spatial and motion features to model spatiotemporal interactions between different action semantics properly. Second, the motion-aware network encodes the locations of action semantics in video frames utilizing the motion-aware 2D positional encoding algorithm. Such a motion-aware mechanism memorizes the dynamic spatiotemporal variations in action frames that current methods cannot exploit. Third, the sequence-based temporal attention model captures the heterogeneous temporal dependencies in action frames. In contrast to standard temporal attention used in natural language processing, primarily aimed at finding similarities between linguistic words, the proposed sequence-based temporal attention is designed to determine both the differences and similarities between video frames that jointly define the meaning of actions. The proposed approach outperforms the state-of-the-art solutions on four spatiotemporal action datasets: AVA 2.2, AVA 2.1, UCF101-24, and EPIC-Kitchens., Comment: IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

Coherent imaging systems, such as medical ultrasound and synthetic aperture radar (SAR), are subject to corruption from speckle due to sub-resolution scatterers. Since speckle is multiplicative in nature, the constituent image regions become corrupted to different extents. The task of denoising such images requires algorithms specifically designed for removing signal-dependent noise. This paper proposes a novel image denoising algorithm for removing signal-dependent multiplicative noise with diffusion models, called Speckle Denoising Diffusion Probabilistic Models (SDDPM). We derive the mathematical formulations for the forward process, the reverse process, and the training objective. In the forward process, we apply multiplicative noise to a given image and prove that the forward process is Gaussian. We show that the reverse process is also Gaussian and the final training objective can be expressed as the Kullback Leibler (KL) divergence between the forward and reverse processes. As derived in the paper, the final denoising task is a single step process, thereby reducing the denoising time significantly. We have trained our model with natural land-use images and ultrasound images for different noise levels. Extensive experiments centered around two different applications show that SDDPM is robust and performs significantly better than the comparative models even when the images are severely corrupted., Comment: 12 pages, 12 figures

This paper proposes a novel multi-modal transformer network for detecting actions in untrimmed videos. To enrich the action features, our transformer network utilizes a new multi-modal attention mechanism that computes the correlations between different spatial and motion modalities combinations. Exploring such correlations for actions has not been attempted previously. To use the motion and spatial modality more effectively, we suggest an algorithm that corrects the motion distortion caused by camera movement. Such motion distortion, common in untrimmed videos, severely reduces the expressive power of motion features such as optical flow fields. Our proposed algorithm outperforms the state-of-the-art methods on two public benchmarks, THUMOS14 and ActivityNet. We also conducted comparative experiments on our new instructional activity dataset, including a large set of challenging classroom videos captured from elementary schools.

This paper proposes a novel age estimation algorithm, the Temporally-Aware Adaptive Graph Convolutional Network (TAA-GCN). Using a new representation based on graphs, the TAA-GCN utilizes skeletal, posture, clothing, and facial information to enrich the feature set associated with various ages. Such a novel graph representation has several advantages: First, reduced sensitivity to facial expression and other appearance variances; Second, robustness to partial occlusion and non-frontal-planar viewpoint, which is commonplace in real-world applications such as video surveillance. The TAA-GCN employs two novel components, (1) the Temporal Memory Module (TMM) to compute temporal dependencies in age; (2) Adaptive Graph Convolutional Layer (AGCL) to refine the graphs and accommodate the variance in appearance. The TAA-GCN outperforms the state-of-the-art methods on four public benchmarks, UTKFace, MORPHII, CACD, and FG-NET. Moreover, the TAA-GCN showed reliability in different camera viewpoints and reduced quality images.

This study proposes an efficient neural network with convolutional layers to classify significantly class-imbalanced clinical data. The data are curated from the National Health and Nutritional Examination Survey (NHANES) with the goal of predicting the occurrence of Coronary Heart Disease (CHD). While the majority of the existing machine learning models that have been used on this class of data are vulnerable to class imbalance even after the adjustment of class-specific weights, our simple two-layer CNN exhibits resilience to the imbalance with fair harmony in class-specific performance. In order to obtain significant improvement in classification accuracy under supervised learning settings, it is a common practice to train a neural network architecture with a massive data and thereafter, test the resulting network on a comparatively smaller amount of data. However, given a highly imbalanced dataset, it is often challenging to achieve a high class 1 (true CHD prediction rate) accuracy as the testing data size increases. We adopt a two-step approach: first, we employ least absolute shrinkage and selection operator (LASSO) based feature weight assessment followed by majority-voting based identification of important features. Next, the important features are homogenized by using a fully connected layer, a crucial step before passing the output of the layer to successive convolutional stages. We also propose a training routine per epoch, akin to a simulated annealing process, to boost the classification accuracy. Despite a 35:1 (Non-CHD:CHD) ratio in the NHANES dataset, the investigation confirms that our proposed CNN architecture has the classification power of 77% to correctly classify the presence of CHD and 81.8% the absence of CHD cases on a testing data, which is 85.70% of the total dataset. ( (<1920 characters)Please check the paper for full abstract), Comment: Accepted in Expert Systems with Applications

Gait recognition using noninvasively acquired data has been attracting an increasing interest in the last decade. Among various modalities of data sources, it is experimentally found that the data involving skeletal representation are amenable for reliable feature compaction and fast processing. Model-based gait recognition methods that exploit features from a fitted model, like skeleton, are recognized for their view and scale-invariant properties. We propose a model-based gait recognition method, using sequences recorded by a single flash lidar. Existing state-of-the-art model-based approaches that exploit features from high quality skeletal data collected by Kinect and Mocap are limited to controlled laboratory environments. The performance of conventional research efforts is negatively affected by poor data quality. We address the problem of gait recognition under challenging scenarios, such as lower quality and noisy imaging process of lidar, that degrades the performance of state-of-the-art skeleton-based systems. We present GlidarCo to attain high accuracy on gait recognition under the described conditions. A filtering mechanism corrects faulty skeleton joint measurements, and robust statistics are integrated to conventional feature moments to encode the dynamic of the motion. As a comparison, length-based and vector-based features extracted from the noisy skeletons are investigated for outlier removal. Experimental results illustrate the efficacy of the proposed methodology in improving gait recognition given noisy low resolution lidar data.

Gait recognition is a leading remote-based identification method, suitable for real-world surveillance and medical applications. Model-based gait recognition methods have been particularly recognized due to their scale and view-invariant properties. We present the first model-based gait recognition methodology, $\mathcal{G}$lidar3DJ using a skeleton model extracted from sequences generated by a single flash lidar camera. Existing successful model-based approaches take advantage of high quality skeleton data collected by Kinect and Mocap, for example, are not practicable for applications outside the laboratory. The low resolution and noisy imaging process of lidar negatively affects the performance of state-of-the-art skeleton-based systems, generating a significant number of outlier skeletons. We propose a rule-based filtering mechanism that adopts robust statistics to correct for skeleton joint measurements. Quantitative measurements validate the efficacy of the proposed method in improving gait recognition., Comment: This paper is accepted to be published in: 2019 IEEE International Conference on Image Processing, Sept 22-25, 2019, Taipei, Taiwan

Automatic 3D reconstruction of glia morphology is a powerful tool necessary for investigating the role of microglia in neurological disorders in the central nervous system. Current glia skeleton reconstruction techniques fail to capture an accurate tracing of the processes over time, useful for the study of the microglia motility and morphology in the brain during healthy and diseased states. We propose Hieroglyph, a fully automatic temporal 3D skeleton reconstruction algorithm for glia imaged via 3D multiphoton microscopy. Hieroglyph yielded a 21% performance increase compared to state of the art automatic skeleton reconstruction methods and outperforms the state of the art in different measures of consistency on datasets of 3D images of microglia. The results from this method provide a 3D graph and digital reconstruction of glia useful for a myriad of morphological analyses that could impact studies in brain immunology and disease., Comment: submitted to IEEE International Conference on Image Processing, 2019

The shape and connectivity of a neuron determine its function. Modern imaging methods have proven successful at extracting such information. However, in order to analyze this type of data, neuronal morphology needs to be encoded in a graph-theoretic method. This encoding enables the use of high throughput informatic methods to extract and infer brain function. The application of graph-theoretic methods to neuronal morphological representation comes with certain difficulties. Here we report a novel, effective method to accomplish this task. The morphology of a neuron, which consists of its overall size, global shape, local branch patterns, and cell-specific biophysical properties, can vary significantly with the cell's identity, location, as well as developmental and physiological state. Various algorithms have been developed to customize shape based statistical and graph related features for quantitative analysis of neuromorphology, followed by the classification of neuron cell types using the features. Unlike the classical feature extraction based methods from imaged or 3D reconstructed neurons, we propose a model based on the rooted path decomposition from the soma to the dendrites of a neuron and extract morphological features on each path. We hypothesize that measuring the distance between two neurons can be realized by minimizing the cost of continuously morphing the set of all rooted paths of one neuron to another. To validate this claim, we first establish the correspondence of paths between two neurons using a modified Munkres algorithm. Next, an elastic deformation framework that employs the square root velocity function is established to perform the continuous morphing, which, in addition, provides an effective visualization tool. We experimentally show the efficacy of NeuroPath2Path, NeuroP2P, over the state of the art., Comment: Submitted to Neuroinformatics

In this paper, we present a novel region-based segmentation method for cortical spreading depressions in 2-photon microscopy images. Fluorescent microscopy has become an important tool in neuroscience, but segmentation approaches are challenged by the opaque properties and structures of brain tissue. These challenges are made more extreme when segmenting events such as cortical spreading depressions, where low signal-to-noise ratios and intensity inhomogeneity dominate images. The method we propose uses a local intensity similarity measure that takes advantage of normalized Euclidean and geodesic distance maps of the image. This method provides a smooth segmentation boundary which is robust to the noise and inhomogeneity within cortical spreading depression images. Experimental results yielded a DICE index of 0.9859, an increase of 6% over the current state-of-the-art, and a reduction of root mean square error by 79.9%.

Bacterial biofilm segmentation poses significant challenges due to lack of apparent structure, poor imaging resolution, limited contrast between conterminous cells and high density of cells that overlap. Although there exist bacterial segmentation algorithms in the existing art, they fail to delineate cells in dense biofilms, especially in 3D imaging scenarios in which the cells are growing and subdividing in a complex manner. A graph-based data clustering method, LCuts, is presented with the application on bacterial cell segmentation. By constructing a weighted graph with node features in locations and principal orientations, the proposed method can automatically classify and detect differently oriented aggregations of linear structures (represent by bacteria in the application). The method assists in the assessment of several facets, such as bacterium tracking, cluster growth, and mapping of migration patterns of bacterial biofilms. Quantitative and qualitative measures for 2D data demonstrate the superiority of proposed method over the state of the art. Preliminary 3D results exhibit reliable classification of the cells with 97% accuracy., Comment: v1: Submitted to IEEE International Conference on Image Processing (ICIP) 2019; v2: Minor edits, updated reference and co-authors; v3: Accepted to be published in 2019 IEEE International Conference on Image Processing, Sep 22-25, 2019, Taipei. IEEE Copyright notice added. Minor changes for camera-ready version. (updated May. 6, 2019)

Transform-domain least mean squares (LMS) adaptive filters encompass the class of algorithms where the input data are subjected to a data-independent unitary transform followed by a power normalization stage as preprocessing steps. Because conventional transformations are not data-dependent, this preconditioning procedure was shown theoretically to improve the convergence of the LMS filter only for certain classes of input data. However, in reality if the class of input data is not known beforehand, it is difficult to decide which transformation to use. Thus, there is a need to devise a learning framework to obtain such a preconditioning transformation using input data prior to applying on the input data. It is hypothesized that the underlying topology of the data affects the selection of the transformation. With the input modeled as a weighted graph that mimics neuronal interactions, PrecoG obtains the desired transform by recursive estimation of the graph Laplacian matrix. Additionally, we show the efficacy of the transform as a generalized split preconditioner on a linear system of equations and in Hebb-LMS settings. In terms of the improvement of the condition number after applying the transformation, PrecoG performs significantly better than the existing state-of-the-art techniques that involve unitary and non-unitary transforms., Comment: Preprint, in submission

Robust and accurate scale estimation of a target object is a challenging task in visual object tracking. Most existing tracking methods cannot accommodate large scale variation in complex image sequences and thus result in inferior performance. In this paper, we propose to incorporate a novel criterion called the average peak-to-correlation energy into the multiresolution translation filter framework to obtain robust and accurate scale estimation. The resulting system is named SITUP: Scale Invariant Tracking using Average Peak-to-Correlation Energy. SITUP effectively tackles the problem of fixed template size in standard discriminative correlation filter based trackers. Extensive empirical evaluation on the publicly available tracking benchmark datasets demonstrates that the proposed scale searching framework meets the demands of scale variation challenges effectively while providing superior performance over other scale adaptive variants of standard discriminative correlation filter based trackers. Also, SITUP obtains favorable performance compared to state-of-the-art trackers for various scenarios while operating in real-time on a single CPU., Comment: This work has been submitted to the IEEE for possible publication

In the study of neurons, morphology influences function. The complexity in the structure of neurons poses a challenge in the identification and analysis of similar and dissimilar neuronal cells. Existing methodologies carry out structural and geometrical simplifications, which substantially change the morphological statistics. Using digitally-reconstructed neurons, we extend the work of Path2Path as ElasticPath2Path, which seamlessly integrates the graph-theoretic and differential-geometric frameworks. By decomposing a neuron into a set of paths, we derive graph metrics, which are path concurrence and path hierarchy. Next, we model each path as an elastic string to compute the geodesic distance between the paths of a pair of neurons. Later, we formulate the problem of finding the distance between two neurons as a path assignment problem with a cost function combining the graph metrics and the geodesic deformation of paths. ElasticPath2Path is shown to have superior performance over the state of the art., Comment: This paper is submitted to IEEE International Conference on Image Processing, 2018

Reliable cell counting and segmentation of oligodendrocyte progenitor cells (OPCs) are critical image analysis steps that could potentially unlock mysteries regarding OPC function during pathology. We propose a saliency-based method to detect OPCs and use a marker-controlled watershed algorithm to segment the OPCs. This method first implements frequency-tuned saliency detection on separate channels to obtain regions of cell candidates. Final detection results and internal markers can be computed by combining information from separate saliency maps. An optimal saliency level for OPCs (OSLO) is highlighted in this work. Here, watershed segmentation is performed efficiently with effective internal markers. Experiments show that our method outperforms existing methods in terms of accuracy., Comment: v1: Submitted to ICIP 2018; v2: Accepted to be published in 2018 IEEE International Conference on Image Processing, Oct 7-10, 2018, Athens, Greece. IEEE Copyright notice added. Minor changes for camera-ready version

A shape filter is presented to repair segmentation results obtained in calcium imaging of neurons in vivo. This post-segmentation algorithm can automatically smooth the shapes obtained from a preliminary segmentation, while precluding the cases where two neurons are counted as one combined component. The shape filter is realized using a square-root velocity to project the shapes on a shape manifold in which distances between shapes are based on elastic changes. Two data-driven weighting methods are proposed to achieve a trade-off between shape smoothness and consistency with the data. Intuitive comparisons of proposed methods via projection onto Cartesian maps demonstrate the smoothing ability of the shape filter. Quantitative measures also prove the superiority of our methods over models that do not employ any weighting criterion., Comment: v1: Submitted as a conference contribution to ICIP 2018 (pre-print version Feb. 14, 2018); v2: Accepted to be published in 2018 IEEE International Conference on Image Processing, Oct 7-10, 2018, Athens, Greece. IEEE Copyright notice added. Minor changes for camera-ready version (updated Jun. 1, 2018)

We propose an automatic 3D segmentation algorithm for multiphoton microscopy images of microglia. Our method is capable of segmenting tubular and blob-like structures from noisy images. Current segmentation techniques and software fail to capture the fine processes and soma of the microglia cells, useful for the study of the microglia role in the brain during healthy and diseased states. Our coupled tubularity flow field (TuFF)-blob flow field (BFF) method evolves a level set toward the object boundary using the directional tubularity and blobness measure of 3D images. Our method found a 20% performance increase against state of the art segmentation methods on a dataset of 3D images of microglia even in images with intensity heterogeneity throughout the object. The coupled TuFF-BFF segmentation results also yielded 40% improvement in accuracy for the ramification index of the processes, which displays the efficacy of our method., Comment: submitted as a conference contribution to International Conference on Image Processing 2018

Measurement of lung ventilation is one of the most reliable techniques of diagnosing pulmonary diseases. The time consuming and bias prone traditional methods using hyperpolarized H${}^{3}$He and ${}^{1}$H magnetic resonance imageries have recently been improved by an automated technique based on multiple active contour evolution. Mapping results from an equivalent thermodynamic model, here we analyse the fundamental dynamics orchestrating the active contour (AC) method. We show that the numerical method is inherently connected to the universal scaling behavior of a classical nucleation-like dynamics. The favorable comparison of the exponent values with the theoretical model render further credentials to our claim., Comment: 4 pages, 4 figures