Search

The stripe fixed pattern noise (FPN) of infrared images significantly corrupts image quality, so that most infrared imaging systems suffer from the degradation of visibility and detectability during operation. Therefore, the FPN de-striping method, which eliminates stripe patterns without substantial loss of image information, remains a core technology in the field of infrared image processing. In this article, we propose the dual-branch structure based FPN de-striping deep convolutional neural network (DBS-DCN) to effectively extract structural features of FPN and preserve the image details in a single infrared image. In addition, we have established the parametric FPN model through the diagnostic experiments of infrared images based on the physical principle of an infrared detector and its signal response. We have optimized each parameter of the FPN model using measured data, which acquired on a wide range of detector temperatures. Further, we generate the training data using our FPN model to ensure stable learning performance against various stripe patterns. We performed comparative experiments with state-of-the-art methods using artificially corrupted infrared images and real corrupted infrared data, and our proposed method achieved outstanding de-striping results in both qualitative and quantitative evaluation compared to existing methods.

The success of multi-sensor data fusions in deep learning appears to be attributed to the use of complementary information among multiple sensor datasets. Compared to their predictive performance, relatively less attention has been devoted to the adversarial robustness of multi-sensor data fusion models. To achieve adversarial robust multi-sensor data fusion networks, we propose here a novel robust training scheme called Multi-Sensor Cumulative Learning (MSCL). The motivation behind the MSCL method is based on the way human beings learn new skills. The MSCL allows the multi-sensor fusion network to learn robust features from individual sensors, and then learn complex joint features from multiple sensors just as people learn to walk before they run. The step wise framework of MSCL enables the network to incorporate pre-trained knowledge of robustness with new joint information from multiple sensors. Extensive experimental evidence validated that the MSCL outperforms other multi-sensor fusion training in defending against adversarial examples.

Our work addresses long-term motion context issues for predicting future frames. To predict the future precisely, it is required to capture which long-term motion context (e.g., walking or running) the input motion (e.g., leg movement) belongs to. The bottlenecks arising when dealing with the long-term motion context are: (i) how to predict the long-term motion context naturally matching input sequences with limited dynamics, (ii) how to predict the long-term motion context with high-dimensionality (e.g., complex motion). To address the issues, we propose novel motion context-aware video prediction. To solve the bottleneck (i), we introduce a long-term motion context memory (LMC-Memory) with memory alignment learning. The proposed memory alignment learning enables to store long-term motion contexts into the memory and to match them with sequences including limited dynamics. As a result, the long-term context can be recalled from the limited input sequence. In addition, to resolve the bottleneck (ii), we propose memory query decomposition to store local motion context (i.e., low-dimensional dynamics) and recall the suitable local context for each local part of the input individually. It enables to boost the alignment effects of the memory. Experimental results show that the proposed method outperforms other sophisticated RNN-based methods, especially in long-term condition. Further, we validate the effectiveness of the proposed network designs by conducting ablation studies and memory feature analysis. The source code of this work is available., CVPR 2021 (Oral)

It is well-known that identity-unrelated variations (e.g., viewpoint or illumination) degrade the performances of face recognition methods. In order to handle this challenge, a robust method for disentangling the identity and view representations has drawn an attention in the machine learning area. However, existing methods learn discriminative features which require a manual supervision of such factors of variations. In this paper, we propose a novel disentangling framework through modeling three representations of identity, viewpoint, and residues (i.e., identity and pose unrelated) which do not require supervision of the variations. By jointly modeling the three representations, we enhance the disentanglement of each representation and achieve robust face recognition performance. Further, the learned viewpoint representation can be utilized for pose estimation or editing of a posed facial image. Extensive quantitative and qualitative evaluations verify the effectiveness of our proposed method which disentangles identity, viewpoint, and residues of facial images.

The performance of recent deep neural networks in various computer vision areas such as object detection has increased significantly. Along with such advances, attempts to visualize and interpret the networks have been made in order to understand how a network predicts a certain result. However, there is a lack of research on ways to improve the interpretability of networks’ features. In this paper, we propose a spatial relation reasoning (SRR) framework to encode interpretable networks’ features, especially an object detector, by mimicking the human visual cognition system. The SRR consists of the spatial feature encoder (SFE) and the graph-based spatial relation encoder (GSRE) to consider spatial relationships between different parts of an object. So that, object detectors can encode spatially-related object features enabling humanlike visual interpretation. We verified the proposed framework with general object detectors on public datasets-PAS-CAL VOC and MS COCO.

Despite the success of the deep neural networks (DNNs), in case of incremental learning, DNNs are known to suffer from catastrophic forgetting problems which are the phenomenon of entirely forgetting previously learned task information upon learning current task information. To alleviate this problem, we propose a novel knowledge distillation-based class incremental learning method with a task-selective autoencoder (TsAE). By learning the TsAE to reconstruct the feature map of each task, the proposed method effectively memorizes not only the classes of the current task but also the classes of previously learned tasks. Since the proposed TsAE has a simple but powerful architecture, it can be easily generalized to other knowledge distillation-based class incremental learning methods. Our experimental results on various datasets, including iCIFAR-100 and iILSVRC-small, demonstrated that the proposed method achieves higher classification accuracy and less forgetting compared to the stateof-the-art methods.

To combat against adversarial attacks, autoencoder structure is widely used to perform denoising which is regarded as gradient masking. In this paper, we revisit the role of autoencoders in adversarial settings. Through the comprehensive experimental results and analysis, this paper presents the inherent property of adversarial robustness in the autoencoders. We also found that autoencoders may use robust features that cause inherent adversarial robustness. We believe that our discovery of the adversarial robustness of the autoencoders can provide clues to the future research and applications for adversarial defense., Accepted at ICIP 2020

Deep learning-based video facial authentication has limitations when it comes to real-world applications, due to large mode variations such as illumination, pose, and eyeglasses variations in real-life situations. Many of existing mode-invariant facial authentication methods need labels of each mode. However, the label information could not be always available in practice. To alleviate this problem, we develop an unsupervised mode disentangling method for video facial authentication. By matching both disentangled identity features and dynamic features of two facial videos, our proposed method shows significant face verification and identification performances on three publicly available datasets, KAIST-MPMI, UVA-NEMO, and YTF.

Few-shot classification is defined as a task where the network aims to classify unseen classes given only a few samples. Recent approaches, especially metric-based methods, have great progress in few-shot classification. However, the existing metric-based methods have a limitation in deploying discriminative features for elaborate comparison. They usually extract features from the embedding network without direct consideration of the relationship between support and query sets. To address the relationship, we propose a novel architecture, Style Correlated Module (SCM) to learn style correlation between support and query sets for few-shot classification. The proposed module leads support and query feature maps to focus on significant style correlated features and encourage the metric network to conduct an elaborate comparison. Furthermore, the proposed module can be generally applied to the existing metric-based approaches by adding the SCM behind the embedding network. We evaluate our proposed method with comprehensive experiments on two publicly available datasets and demonstrate its effectiveness with comparable results.

Recent development of Virtual Reality (VR) technology provides more realistic experience for viewers with a variety of contents. While the viewing safety of the viewers is one of the important issues in VR industry, the necessity of VR sickness estimation has been drawing attentions. Inspired by the observations that camera shake in VR videography is one of the major causes of VR sickness, we propose a novel deep network that predicts VR sickness level of individuals caused by camera shake. The proposed method is designed to comprehensively identify changes in direction and speed of the VR video scenes with camera shake. Sparse selection of optical flow maps with different intervals allows the proposed network to efficiently extract stimulus features with a variety of camera shake patterns. We built a new benchmark database for the evaluation of the proposed method that consists of 360-degree videos including various camera shake movements, physiological signals, and Simulation Sickness Questionnaires (SSQ) scores of the experimental participants. Experimental results of the sickness prediction show the effectiveness of the proposed method on the built benchmark database.

DeepFake synthesizes realistic fake videos that could be used maliciously such as manipulation and harassment. In order to prevent such malicious usages, detecting fake videos is immediately needed. In this paper, we propose a novel fake video detection method by adopting predictive uncertainty in detection. We devise the certainty-based attention network which guides to focus certainty-key frames in detecting fake videos. In addition, certainty-based attention is proposed for refining the features with consideration for frame-level certainty. Experiments are performed to validate the effectiveness of the proposed method by comparing the existing methods on Celeb-DF, the latest DeepFake dataset.

In this paper, we propose a novel image segmentation method to tackle two critical problems of medical image, which are (i) ambiguity of structure boundary in the medical image domain and (ii) uncertainty of the segmented region without specialized domain knowledge. To solve those two problems in automatic medical segmentation, we propose a novel structure boundary preserving segmentation framework. To this end, the boundary key point selection algorithm is proposed. In the proposed algorithm, the key points on the structural boundary of the target object are estimated. Then, a boundary preserving block (BPB) with the boundary key point map is applied for predicting the structure boundary of the target object. Further, for embedding experts’ knowledge in the fully automatic segmentation, we propose a novel shape boundary-aware evaluator (SBE) with the ground-truth structure information indicated by experts. The proposed SBE could give feedback to the segmentation network based on the structure boundary key point. The proposed method is general and flexible enough to be built on top of any deep learning-based segmentation network. We demonstrate that the proposed method could surpass the state-of-the-art segmentation network and improve the accuracy of three different segmentation network models on different types of medical image datasets.

In this paper, we propose a novel video frame interpolation method via exceptional motion-aware synthesis, in which accurate optical flow could be estimated even with exceptional motion patterns. Specifically, we devise two deep learning modules: exceptional motion detection and frame interpolation with refined flow. The motion detection module detects the position and intensity of exceptional motion patterns in current frame given the past frame sequence. The flow refinement module refines the pre-estimated optical flow for synthesizing the intermediate frame using the information of exceptional motion. The proposed modules improve the quality of the synthesized intermediate frame by making the optical flow robust against exceptional case of motion. Experimental results showed that the proposed method outperforms the state-of-the-art methods qualitatively and quantitatively.

Object detection performs two tasks (classification and localization) simultaneously. Two tasks share a similarity: they need robust features that effectively represent the visual appearance of the objects. However, two tasks also have different properties. First, classification mainly requires features from discriminative parts of an object to determine the object category, whereas localization mainly requires features from the entire object regions for localizing by drawing a bounding box. Second, classification has a translation invariant property, whereas localization has a translation variant property. In order to increase the efficiency of object detection, it is necessary to design a network in consideration of the commonalities and differences of two tasks. In this work, we simply modified layers of the existing object detection networks into three parts by considering such characteristics: lower-layer feature sharing part, layer separation part, and feature fusion part. As a result, the performance of the proposed method was noticeably improved by properly sharing, separating, and fusing layers of the existing object detection networks.

In recent years, it is verified that the deep learning network is able to process not only images but also time-series information. Since breast image analysis plays a big role in the diagnosis of breast cancer, there have been a large number of attempts to apply the deep learning method for an accurate diagnosis. With the advance of deep learning approaches, the possibility of using medical reports (in natural language) has been increased. However, there is no public medical report dataset associated with the breast image. Instead, in the conventional public breast mammography datasets, the characteristics of breast cancer are annotated according to the standardized term (Breast Imaging-Reporting and Data System). In this study, a breast-sentence dataset is proposed to investigate its usefulness in computer-aided diagnosis. Based on the conventional breast mammography datasets, we annotated sentences in the natural language according to the standardized terms (defined in Breast Imaging-Reporting and Data System) in conventional breast mammography datasets. In the experiments, we show three use cases to verify the usefulness of the breast-sentence dataset: 1) CAD framework with radiologist's input, 2) the use of sentence dataset in training a CAD, and 3) visual pointing guided by sentence.

Despite the rapid progress of deep learning research in recent years, interpreting deep network is still quite challenging. Interpreting deep networks is essential to both end-users and developers since it gives confidence in the usage of the deep network. This paper deals with a method for interpreting deep networks, especially visual interpretation. In order to get visual interpretation from a target deep network, we propose a ProbeNet that provides a decomposed visual interpretation of the target deep network. The ProbeNet decomposes the feature representations of the point of the target deep network into human interpretable units. Furthermore, the ProbeNet provides kernel-level analysis about the target deep network. In experiments, visual interpretation of two different target deep networks showed the usefulness of the ProbeNet to interpret target deep networks.

Realistic image synthesis is to generate an image that is perceptually indistinguishable from an actual image. Generating realistic looking images with large variations (e.g., large spatial deformations and large pose change), however, is very challenging. Handing large variations as well as preserving appearance needs to be taken into account in the realistic looking image generation. In this paper, we propose a novel realistic looking image synthesis method, especially in large change demands. To do that, we devise generative guiding blocks. The proposed generative guiding block includes realistic appearance preserving discriminator and naturalistic variation transforming discriminator. By taking the proposed generative guiding blocks into generative model, the latent features at the layer of generative model are enhanced to synthesize both realistic looking- and target variation- image. With qualitative and quantitative evaluation in experiments, we demonstrated the effectiveness of the proposed generative guiding blocks, compared to the state-of-the-arts., This work is accepted in ICIP 2019

Object detection became one of the major fields in computer vision. In object detection, object classification and object localization tasks are conducted. Previous deep learning-based object detection networks perform with feature maps generated by completely shared networks. However, object classification focuses on the most discriminative object part of the feature map. Whereas, object localization requires a feature map that is focused on the entire area of the object. In this paper, we propose a novel object detection network by considering the difference between the two tasks. The proposed deep learning-based network mainly consists of two parts; 1) Attention network part where task-specific attention maps are generated, 2) Layer separation part where layers for estimating two tasks are separated. Comprehensive experimental results based on PASCAL VOC dataset and MS COCO dataset showed that proposed object detection network outperformed the state-of-the-art methods.

This paper proposes a novel deep learning framework for age and gender classification with the adversarial spatial frequency domain critic. In the proposed framework, the encoder-generator synthesizes realistic facial images with real images and corresponding age and gender label. An adversarial critic is devised to make generated images more proper for age and gender classification. In particular, we analyze the characteristic of age and gender attributes in the spatial frequency domain. Based on our investigation, we devise the spatial frequency domain critic network for considering the specific frequency bands which are dominant on age and gender attributes. Our discriminator is designed to simultaneously perform age and gender classification tasks. For this purpose, alternating learning is performed for multi-task classification. Experimental results showed that the proposed method outperformed other state-of-the-art methods in age and gender classifications.

Recent state-of-the-art systems for human action recognition are computationally intensive due to the use of full-length videos and complex network structures. This study aims to develop sampling strategies as well as simple network structure to boost inference time of recognition. Especially, auto-correlation sequence, which shows the similarity between a video and a lagged version of itself, is adopted to extract the most essential segment of the video without information loss. The proposed method considerably reduces inference time while keeping comparable recognition accuracy.

Object detection in a road scene has received a significant attention from research fields of developing autonomous vehicle and automatic road monitoring systems. However, object occlusion problems frequently occur in generic road scenes. Due to such occlusion problems, previous object detection methods have limitations of not being able to detect objects accurately. In this paper, we propose a novel object detection network which is robust in occlusions. For effective object detection even with occlusion, the proposed network mainly consists of two parts; 1) Object detection framework, 2) Multiple object bounding box (OBB)-Critic network for predicting a BB map which estimates both ob-j ect region and occlusion region. Comprehensive experimental results on a KITTI Vision Benchmark Suite dataset showed that the proposed object detection network outperformed the state-of-the-art methods.

This paper proposes a novel visual comfort assessment (VCA) for stereoscopic images using deep learning. To predict visual discomfort of human visual system in stereoscopic viewing, we devise VCA deep networks to latently encode perceptual cues, which are visual differences between stereoscopic images and human attention-based disparity magnitude and gradient information. To extract the visual difference features from left and right views, a Siamese network is employed. In addition, human attention region-based disparity magnitude and gradient maps are fed to two individual deep convolutional neural networks (DCNNs) for disparity-related features based on human visual system (HVS). Finally, by aggregating these perceptual features, the proposed method directly predicts the final visual comfort score. Extensive and comparative experiments have been conducted on IEEE-SA dataset. Experimental results show that the proposed method can yield excellent correlation performance compared to existing methods.

In the past few decades, automatic face recognition has been an important vision task. In this paper, we exploit the spatial relationships of facial local regions by using a novel deep network. In the proposed method, face is spatially scanned with spatial long short-term memory (LSTM) to encode the spatial correlation of facial regions. Moreover, with facial regions of various scales, the complementary information of the multi-scale facial features is encoded. Experimental results on public database showed that the proposed method outperformed the conventional methods by improving the face recognition accuracy under illumination variation.

Automatic understanding of the content of a reference image and natural language questions is needed in Visual Question Answering (VQA). Generating a visual attention map that focuses on the regions related to the context of the question can improve performance of VQA. In this paper, we propose adaptive attention-based VQA network. The proposed method utilizes the complementary information from the attention maps depending on three levels of word embedding (word level, phrase level, and question level embedding), and adaptively fuses the information to represent the image-question pair appropriately. Comparative experiments have been conducted on the public COCO-QA database to validate the proposed method. Experimental results have shown that the proposed method outperforms previous methods in terms of accuracy.

In this paper, we propose a novel medical image segmentation using iterative deep learning framework. We have combined an iterative learning approach and an encoder-decoder network to improve segmentation results, which enables to precisely localize the regions of interest (ROIs) including complex shapes or detailed textures of medical images in an iterative manner. The proposed iterative deep convolutional encoder-decoder network consists of two main paths: convolutional encoder path and convolutional decoder path with iterative learning. Experimental results show that the proposed iterative deep learning framework is able to yield excellent medical image segmentation performances for various medical images. The effectiveness of the proposed method has been proved by comparing with other state-of-the-art medical image segmentation methods., Comment: accepted at EMBC 2017

Facial expression recognition is increasingly gaining importance in emerging affective computing applications. In practice, achieving accurate facial expression recognition is still challenging due to environmental variations. In this paper, we propose a color channel-wise recurrent facial feature learning. The proposed method adopts recurrent neural network to learn expression features sequentially along color channels. The proposed network preserves discriminative expression feature through a long short-term memory for the sequence of color spatial features. Comprehensive experiments have been conducted on the publically available CMU Multi-PIE dataset under illumination variations. Experimental results showed that the proposed method achieved higher recognition rates compared to the state-of-the-art methods.

We propose a bilateral hemiface feature representation learning via convolutional neural networks (CNNs) for pose robust facial expression recognition. The proposed method considers two characteristics of facial expressions. First, features from local patches are more robust to pose variations. Second, human faces are bilaterally symmetrical on left and right hemifaces. To incorporate those characteristics, a CNN is devised to learn feature representations from local patches. Then, feature representations are learned from each hemiface separately. To reduce the effect of self-occlusion, a shared feature representation is learned by combining both hemiface feature representations. The shared feature representation adaptively learns to utilize the hemiface feature representations according to the head pose. Experiments conducted on the Multi-PIE dataset showed that the proposed bilateral hemiface feature representation is pose robust and compares favorably to state-of-the-art methods.

In this paper, we propose a new facial landmarks detection method based on deep learning with facial contour and facial components constraints. The proposed deep convolutional neural networks (DCNNs) for facial landmark detection consists of two deep networks: one DCNN is to detect landmarks constrained on the facial contour and the other is to detect landmarks constrained on facial components. A novel DCNN structure for the landmarks detection with facial component constraints is proposed, which branches the network at higher layers in order to capture the intricate local facial components features. Moreover, a novel learning strategy is proposed to learn the DCNN for detecting the landmarks on the facial contour by exploiting the relationship between facial contour landmarks and those on facial components. Experimental results have shown that the proposed method outperforms the state-of-the-art FLD methods.

Facial color is known as playing an important role in face recognition. Color face recognition has been investigated in the last decade. Recently, deep learning has attracted considerable attention due to their high performance in face recognition. The importance of the color in a deep learning framework is not fully investigated yet. In this paper, we have conducted experiments to investigate the effectiveness of facial color in face recognition with deep learning. Through experimental results, we have demonstrated that facial color is helpful for enhancing the recognition performance in deep learning and color space selection is crucial to achieve high performance. Moreover, by fusing features from multiple color spaces, the face recognition accuracy has been considerably improved.

For human identification, facial motion is useful in representing specific dynamic signature. In this paper, we present an effective spatio-temporal representation from facial motion as well as appearance by devising a 3D convolutional neural network (CNN). To maintain the intra-class invariance with limited number of training samples, a multi-task learning approach with human attributes, which are high-level semantic descriptions for identity, has been proposed. Identity-related human attributes can be leveraged to learn the 3D CNN. Comparative experiment has showed that the proposed method improves the performance of the face-based authentication system compared to conventional methods by effectively encoding facial appearance and motions with identity-related human attributes.

According to the supplementary information hypothesis in psychology, facial motion benefits the perception of identity for human. In this study, we propose a face authentication framework which exploits facial dynamics with appearance to effectively improve the authentication performance. In our face authentication scenario, users are guided to make smile expression and the identity behind smile dynamics has been utilized. In order to model the facial dynamics, the recurrent neural network with long short-term memory cells is adopted and the facial dynamics from onset to offset duration is encoded. Comparative experiment has showed that the combination of facial dynamic features with appearance features improves the accuracy of the face authentication system compared to conventional appearance features and spatio-temporal features by effectively capturing facial dynamic and appearance features.

This paper proposes a new temporal consistency measure for quality assessment of synthesized video. Disocclusion regions appear hole regions of the synthesized video at virtual viewpoints. Filling hole regions could be problematic when the synthesized video is perceived through multi-view displays. In particular, the temporal inconsistency caused by hole filling process in view synthesis could affect the perceptual quality of the synthesized video. In the proposed method, we extract excessive flicker regions between consecutive frames and quantify the perceptual effects of the temporal inconsistency on them by measuring the structural similarity. We have demonstrated the validity of the proposed quality measure by comparisons of subjective ratings and existing objective metrics. Experimental results have shown that the proposed temporal inconsistency measure is highly correlated with the overall quality of the synthesized video.

HTTP Adaptive Streaming (HAS) has become a popular trend for multimedia delivery nowadays. Because of throughput variations, video adaptation methods are needed to avoid buffer underflows. In this context, it is also important to evaluate the overall video quality of a session. In this paper, we investigate a quality model that can evaluate a session quality as well as different adaptation strategies in real time. We use the histogram of segment quality values and the histogram of quality gradients in a session to model the overall video quality. Then, our quality model is employed to evaluate, for the first time, the cumulative quality of typical adaptation methods in real time. It is found that, to provide a high quality level, the client should avoid changing versions frequently and drastically.

In clinical studies of breast cancer, masses appear as asymmetric densities between the left and the right breasts, which show different breast tissue structures. For classifying breast masses, most researchers have developed hand-crafted bilateral features by extracting the asymmetric information in 2-D mammograms. In digital breast tomosynthesis (DBT), which has 3D volume data, effective bilateral features are needed to detect masses. In this paper, we propose latent bilateral feature representation with 3-D multi-view deep convolutional neural network (DCNN) in the DBT reconstructed volume. The proposed DCNN is designed to discover hidden or latent bilateral feature representation of masses in self-taught learning. Experimental results show that the proposed latent bilateral feature representation outperforms conventional hand-crafted features by achieving a high area under the receiver operating characteristic curve.

Automatic face recognition (FR) under uncontrolled environments has attracted considerable research attention. In the uncontrolled environments, pose variation is known as one of the crucial factors that influences FR performance. In this paper, we propose a discriminative and pose-robust feature representation using the multi-task learning in deep convolutional neural networks (ConvNet). We introduce four tasks (i.e., maximizing inter-class variation, minimizing intraclass variation, minimizing intra-pose variation, and preserving pose continuity) to learn the ConvNet. Moreover, two-stage learning strategy is proposed to minimize the error functions in learning the deep ConvNet. The extensive experimental results (with the challenging CMU MultiPIE dataset containing pose variations) show that the proposed method outperform stateof-the-art in terms of FR accuracy. Furthermore, the proposed method shows significant improvement even for the face images whose poses are not included in training set.

Recently, high-performance face recognition has attracted research attention in real-world scenarios. Thanks to the advances in sensor technology, face recognition system equipped with multiple sensors has been widely researched. Among them, face recognition system with near-infrared imagery has been one important research topic. In this paper, complementary effect resided in face images captured by nearinfrared and visible rays is exploited by combining two distinct spectral images (i.e., face images captured by near-infrared and visible rays). We propose a new texture feature (i.e., multispectral texture feature) extraction method with synthesized face images to achieve high-performance face recognition with illumination-invariant property. The experimental results show that the proposed method enhances the discriminative power of features thanks the complementary effect.

In this paper, we propose bilateral features for classifying breast masses by extracting the asymmetric information of both the left and the right breasts in the digital breast tomo-synthesis (DBT) reconstructed volume. Clinically, it is known that the left and the right breast of the same patient tend to present a high degree of symmetry of internal structures over broad areas. On the other hand, masses appear as asymmetric densities which show different breast tissue structures between the left and the right breasts. Based on that clinical fact, bilateral features are proposed to measure the dissimilarity of texture or intensity characteristics between volumes-of-interest (VOIs) in a given breast, and the corresponding VOIs in the bilateral breast. Experimental results show that the proposed bilateral features in conjunction with single-view mass features can achieve higher level of classification performance in terms of the area under the receiver operating characteristic (ROC) curve (AUC) compared to the performance of the single-view features only.

This paper presents a new temporally consistent hole filling method based on global optimization for a synthesized 3D vid-eo.1 For the temporal consistency, the proposed method adaptively utilizes the already filled region in a previous frame under the guidance of motion vectors to fill a hole region in a current frame (i.e., target frame to-be-filled). In addition, when filling the hole region in the target frame, reliable labels are stored and propagated to a next target frame in order to reduce the computational cost of the global optimization. Experimental results show that the proposed method can achieve the temporal consistency and a high computational gain than existing hole filling methods.

Human action recognition (HAR) has been attracting much attention in the computer vision arena. In particular, many research efforts were dedicated for developing discriminative feature extraction methods for improving the HAR performance. Among them, trajectory-based features have shown state-of-the-art performance. However, the time-variance of trajectory-based feature and the large number of indistinctive trajectories describing the human action can limit their performance. In this paper, we propose extracting human action features from a distinctive subset of trajectories, namely key-trajectories. Moreover, the key-trajectories are extracted in a time-invariant manner, so that they are able to represent human action regardless of the time at which the action occurs. With publically available and challenging datasets, comparative experiments have been conducted. Results show that the proposed key-trajectory feature extraction improves the HAR performance thanks to their distinctive and time-invariant characteristics.

Digital breast tomosynthesis (DBT) is an emerging 3D x-ray imaging modality in breast cancer screening. Clinical studies have been reported that sensitivity of breast cancer detection can be increased by using two ipsilateral DBT views. Matching corresponding regions in the ipsilateral DBT views is important to achieve high detection sensitivity. In this paper, we propose a novel and effective region matching method based on local structure information in the ipsilateral DBT views. In the proposed method, for a given query region, we find corresponding region on target slices by using local descriptor and associated similarity. Experimental results showed that the proposed region matching method achieved the improvement in accuracy of region matching compared with the existing breast compression model-based region matching method.