Search

Recent temporal action proposal generation approaches have suggested integrating segment- and snippet score-based methodologies to produce proposals with high recall and accurate boundaries. In this paper, different from such a hybrid strategy, we focus on the potential of the snippet score-based approach. Specifically, we propose a new snippet score-based method, named Snippet Relatedness-based Generator (SRG), with a novel concept of "snippet relatedness". Snippet relatedness represents which snippets are related to a specific action instance. To effectively learn this snippet relatedness, we present "pyramid non-local operations" for locally and globally capturing long-range dependencies among snippets. By employing these components, SRG first produces a 2D relatedness score map that enables the generation of various temporal intervals reliably covering most action instances with high overlap. Then, SRG evaluates the action confidence scores of these temporal intervals and refines their boundaries to obtain temporal action proposals. On THUMOS-14 and ActivityNet-1.3 datasets, SRG outperforms state-of-the-art methods for temporal action proposal generation. Furthermore, compared to competing proposal generators, SRG leads to significant improvements in temporal action detection., To appear in TCSVT

Precision medicine is a new approach to understanding health and disease based on patient-specific data such as medical diagnoses; clinical phenotype; biologic investigations such as laboratory studies and imaging; and environmental, demographic, and lifestyle factors. The importance of machine learning techniques in healthcare has expanded quickly in the last decade owing to the rising availability of vast multi-modality data and developed computational models and algorithms. Reinforcement learning is an appealing method for developing efficient policies in various healthcare areas where the decision-making process is typically defined by a long period or a sequential process. In our research, we leverage the power of reinforcement learning and electronic health records of South Koreans to dynamically recommend treatment prescriptions, which are personalized based on patient information of hypertension. Our proposed reinforcement learning-based treatment recommendation system decides whether to use mono, dual, or triple therapy according to the state of the hypertension patients. We evaluated the performance of our personalized treatment recommendation model by lowering the occurrence of hypertension-related complications and blood pressure levels of patients who followed our model’s recommendation. With our findings, we believe that our proposed hypertension treatment recommendation model could assist doctors in prescribing appropriate antihypertensive medications.

In this paper, we propose a new framework for detecting the unauthorized dumping of garbage in real‐world surveillance camera. Although several action/behavior recognition methods have been investigated, these studies are hardly applicable to real‐world scenarios because they are mainly focused on well‐refined datasets. Because the dumping actions in the real‐world take a variety of forms, building a new method to disclose the actions instead of exploiting previous approaches is a better strategy. We detected the dumping action by the change in relation between a person and the object being held by them. To find the person‐held object of indefinite form, we used a background subtraction algorithm and human joint estimation. The person‐held object was then tracked and the relation model between the joints and objects was built. Finally, the dumping action was detected through the voting‐based decision module. In the experiments, we show the effectiveness of the proposed method by testing on real‐world videos containing various dumping actions. In addition, the proposed framework is implemented in a real‐time monitoring system through a fast online algorithm.

Semantic image segmentation has a wide range of applications. When it comes to medical image segmentation, its accuracy is even more important than those of other areas because the performance gives useful information directly applicable to disease diagnosis, surgical planning, and history monitoring. The state-of-the-art models in medical image segmentation are variants of encoder-decoder architecture, which is called U-Net. To effectively reflect the spatial features in feature maps in encoder-decoder architecture, we propose a spatially adaptive weighting scheme for medical image segmentation. Specifically, the spatial feature is estimated from the feature maps, and the learned weighting parameters are obtained from the computed map, since segmentation results are predicted from the feature map through a convolutional layer. Especially in the proposed networks, the convolutional block for extracting the feature map is replaced with the widely used convolutional frameworks: VGG, ResNet, and Bottleneck Resent structures. In addition, a bilinear up-sampling method replaces the up-convolutional layer to increase the resolution of the feature map. For the performance evaluation of the proposed architecture, we used three data sets covering different medical imaging modalities. Experimental results show that the network with the proposed self-spatial adaptive weighting block based on the ResNet framework gave the highest IoU and DICE scores in the three tasks compared to other methods. In particular, the segmentation network combining the proposed self-spatially adaptive block and ResNet framework recorded the highest 3.01% and 2.89% improvements in IoU and DICE scores, respectively, in the Nerve data set. Therefore, we believe that the proposed scheme can be a useful tool for image segmentation tasks based on the encoder-decoder architecture.

Moving object detection in a video plays an important role in many vision applications. Recently, moving object detection using appearance modeling based on a convolutional neural network has been actively developed. However, the CNN-based methods usually require the user's supervision of the first frame so that it becomes highly dependent on the training dataset. In contrast, the method of finding a foreground, which models a background occupying a large proportion in an image, can detect a moving object efficiently in an unsupervised manner. However, existing methods based on background modeling in a pan-tilt-zoom (PTZ) camera suffer many false positives or loss of moving objects due to the estimation error of camera motion. To overcome the aforementioned limitations, we propose a moving object detection method for a PTZ camera through two background models. In an unsupervised way, our method builds the two background models that have different roles: 1) a coarse background model for detecting large changes, and 2) a fine background model for detecting small changes. In more detail, the coarse background model builds a block-based Gaussian model, and the fine model builds a sample consensus model. Both models are adaptively updated according to the estimated camera motion in the video recorded by a PTZ camera. Then, each foreground result from two background models is incorporated to fill the moving object region. Through experiments, the proposed method achieves better performance than the state-of-the-art methods and operates in real-time without parallel processing. In addition, we showed the effectiveness of the proposed model through improved results of moving object detection through combination with the latest supervised method.

From a streaming video, online action detection aims to identify actions in the present. For this task, previous methods use recurrent networks to model the temporal sequence of current action frames. However, these methods overlook the fact that an input image sequence includes background and irrelevant actions as well as the action of interest. For online action detection, in this paper, we propose a novel recurrent unit to explicitly discriminate the information relevant to an ongoing action from others. Our unit, named Information Discrimination Unit (IDU), decides whether to accumulate input information based on its relevance to the current action. This enables our recurrent network with IDU to learn a more discriminative representation for identifying ongoing actions. In experiments on two benchmark datasets, TVSeries and THUMOS-14, the proposed method outperforms state-of-the-art methods by a significant margin. Moreover, we demonstrate the effectiveness of our recurrent unit by conducting comprehensive ablation studies., To appear in CVPR 2020

For video understanding, namely analyzing who did what in a video, actions along with objects are primary elements. Most studies on actions have handled recognition problems for a well‐trimmed video and focused on enhancing their classification performance. However, action detection, including localization as well as recognition, is required because, in general, actions intersect in time and space. In addition, most studies have not considered extensibility for a newly added action that has been previously trained. Therefore, proposed in this paper is an extensible hierarchical method for detecting generic actions, which combine object movements and spatial relations between two objects, and inherited actions, which are determined by the related objects through an ontology and rule based methodology. The hierarchical design of the method enables it to detect any interactive actions based on the spatial relations between two objects. The method using object information achieves an F‐measure of 90.27%. Moreover, this paper describes the extensibility of the method for a new action contained in a video from a video domain that is different from the dataset used.

A novel model for trajectories and semantic regions (sest-hiHMM) is proposed.A sticky version of sest-hiHMMs is proposed for reducing redundant semantic regions.An extended definition of semantic regions covers actual regions, not sets of points.Our models concern the temporal dependency of observations in a trajectory.Our models retrieve reasonable semantic regions from a real trajectory dataset. With an increasing attempt of finding latent semantics in a video dataset, trajectories have become key components since they intrinsically include concise characteristics of object movements. An approach to analyze a trajectory dataset has concentrated on semantic region retrieval, which extracts some regions in which have their own patterns of object movements. Semantic region retrieval has become an important topic since the semantic regions are useful for various applications, such as activity analysis. The previous literatures, however, have just revealed semantically relevant points, rather than actual regions, and have less consideration of temporal dependency of observations in a trajectory. In this paper, we propose a novel model for trajectory analysis and semantic region retrieval. We first extend the meaning of semantic regions that can cover actual regions. We build a model for the extended semantic regions based on a hierarchically linked infinite hidden Markov model, which can capture the temporal dependency between adjacent observations, and retrieve the semantic regions from a trajectory dataset. In addition, we propose a sticky extension to diminish redundant semantic regions that occur in a non-sticky model. The experimental results demonstrate that our models well extract semantic regions from a real trajectory dataset.

In this study, the authors consider a multi-target tracking (MTT) problem in a cluttered environment. Due to the difficulty of the problem, the methods relying only on spatial information such as range, bearing and Doppler velocity can be unreliable. To overcome this, they additionally exploit the amplitude information, commonly provided by radar and sonar, for MTT. However, the usage of amplitude information is not straightforward because the signal-to-noise ratio (SNR) should be known in advance or estimated at the same time. To this end, they first propose a novel SNR estimation algorithm based on a maximum a posteriori approach, which helps the tracker to exploit the amplitude information effectively. Based on the estimated SNR, they then propose a complete framework for MTT, which is mainly composed of data association and track state update parts. They extensively evaluate the proposed system in a series of challenging scenarios, and the experimental results verify the effectiveness and robustness of the authors’ methods.

As DenseNet conserves intermediate features with diverse receptive fields by aggregating them with dense connection, it shows good performance on the object detection task. Although feature reuse enables DenseNet to produce strong features with a small number of model parameters and FLOPs, the detector with DenseNet backbone shows rather slow speed and low energy efficiency. We find the linearly increasing input channel by dense connection leads to heavy memory access cost, which causes computation overhead and more energy consumption. To solve the inefficiency of DenseNet, we propose an energy and computation efficient architecture called VoVNet comprised of One-Shot Aggregation (OSA). The OSA not only adopts the strength of DenseNet that represents diversified features with multi receptive fields but also overcomes the inefficiency of dense connection by aggregating all features only once in the last feature maps. To validate the effectiveness of VoVNet as a backbone network, we design both lightweight and large-scale VoVNet and apply them to one-stage and two-stage object detectors. Our VoVNet based detectors outperform DenseNet based ones with 2x faster speed and the energy consumptions are reduced by 1.6x - 4.1x. In addition to DenseNet, VoVNet also outperforms widely used ResNet backbone with faster speed and better energy efficiency. In particular, the small object detection performance has been significantly improved over DenseNet and ResNet., CVPR2019 CEFRL Workshop

In visual surveillance systems, it is necessary to recognize the behavior of people handling objects such as a phone, a cup, or a plastic bag. In this paper, to address this problem, we propose a new framework for recognizing object-related human actions by graph convolutional networks using human and object poses. In this framework, we construct skeletal graphs of reliable human poses by selectively sampling the informative frames in a video, which include human joints with high confidence scores obtained in pose estimation. The skeletal graphs generated from the sampled frames represent human poses related to the object position in both the spatial and temporal domains, and these graphs are used as inputs to the graph convolutional networks. Through experiments over an open benchmark and our own data sets, we verify the validity of our framework in that our method outperforms the state-of-the-art method for skeleton-based action recognition., Accepted in WACV 2019

Online action detection aims to detect a current action from an untrimmed, streaming video, where only current and past frames are available. Recent methods for online action detection have focused on how to model discriminative representations from temporally partial information. However, they overlook the fact that the input video contains background as well as actions. To overcome this problem, in this paper, we propose a novel approach, named Temporal Filtering Network, to distinguish between relevant and irrelevant information from a partially observed, untrimmed video. Specifically, we present a filtering module to learn relevance scores indicating how relevant the information is to a current action. Our filtering module emphasizes the relevant information to a current action, while it filters out the information of background and unrelated actions. We conduct extensive experiments on THUMOS-14 and TVSeries datasets. On these datasets, the proposed method outperforms state-of-the-art methods by a large margin. We also show the effectiveness of the filtering module through comprehensive ablation studies.

With the advent of the era of digital big data, the Hadoop platform has become widely used in various fields. However, the Hadoop MapReduce framework suffers from problems related to the increase of the name node's main memory and map tasks for the processing of large number of small files. In addition, a method for running C++-based tasks in the MapReduce framework is required in order to conjugate GPUs supporting hardware-based data parallelism in the MapReduce framework. Therefore, in this paper, we present a face detection system that generates a sequence file for images to process big data for images in the Hadoop platform. The system also deals with tasks for GPU-based face detection in the MapReduce framework using Hadoop Pipes. We demonstrate a performance increase of around 6.8-fold as compared to a single CPU process.

We propose a new prototype training dataset, GOD(Generic Objects Dataset), for object detection systems. Our dataset was constructed to fit a bird’s-eye view, not a general eye-level view. We analyzed the training dataset for the eye-level view such as Pascal VOC(Visual Objects Classes), ImageNet, Microsoft COCO(Common Objects in Context), Caltech, and SUN, which were created by many researchers and AMTs(Amazon Mechanical Turk). We then developed a prototype of the proposed GOD, taking into account the statistical properties gained from them. Our prototype GOD has both 16,830 images based on ‘bird’s-eye view’ and 126,510 bounding boxes annotated objects for 10 object categories. This paper confirms statistical properties similar to those of Microsoft COCO.

In this paper, we propose an adaptive transformation engine that provides smart devices with optimal user interfaces and media contents to their characteristics so as to conveniently control home networks remotely using smart devices in the wired and wireless home networks which various kinds of smart devices and home network devices are connected. The proposed engine dynamically transforms UI contents and transfers them using device profiles and UI transformation policies to make it suitable for smart devices. Moreover, the proposed engine offers suitable media contents for smart devices through capabilities of real-time media transcoding and streaming considering device and media profiles.

The welding of zirconium alloy components is one of the most critical processes in the fabrication of nuclear fuel rods used in pressurized water reactors. For this, various welding processes, such as gas tungsten arc welding, electron beam welding, laser beam welding, and resistance pressure welding (RPW), are used around the world. In Korea, the RPW process is being used to fabricate nuclear fuel assembly fuel rods. This study investigated changes in the weldment shape owing to welding conditions such as welding current, welding force, and overlapping. The welding soundness of the weldment was evaluated by hydraulic burst test. The welding temperature of the weld zone was measured using a thermal infrared method. Discontinuous black spots in the weld line, regarded as a non-bonding defect, were confirmed as spots caused by the carbide precipitation of zirconium during welding.