Search

Numerous sleep disorders are characterised by movement during sleep, these include rapid-eye movement sleep behaviour disorder (RBD) and periodic limb movement disorder. The process of diagnosing movement related sleep disorders requires laborious and time-consuming visual analysis of sleep recordings. This process involves sleep clinicians visually inspecting electromyogram (EMG) signals to identify abnormal movements. The distribution of characteristics that represent movement can be diverse and varied, ranging from brief moments of tensing to violent outbursts. This study proposes a framework for automated limb-movement detection by fusing data from two EMG sensors (from the left and right limb) through a Dirichlet process mixture model. Several features are extracted from 10 second mini-epochs, where each mini-epoch has been classified as 'leg-movement' or 'no leg-movement' based on annotations of movement from sleep clinicians. The distributions of the features from each category can be estimated accurately using Gaussian mixture models with the Dirichlet process as a prior. The available dataset includes 36 participants that have all been diagnosed with RBD. The performance of this framework was evaluated by a 10-fold cross validation scheme (participant independent). The study was compared to a random forest model and outperformed it with a mean accuracy, sensitivity, and specificity of 94\%, 48\%, and 95\%, respectively. These results demonstrate the ability of this framework to automate the detection of limb movement for the potential application of assisting clinical diagnosis and decision-making.

A key factor that makes action detection in videos different from general video classification is human-guided clues, especially motion signals. Since not all the pixels in a video are informative for action recognition, the irrelevant and redundant parts can lead to a lot of noise and be burdensome for both feature extraction and classifier training. This encourages the researchers to seek out the design of the attentive model that can dynamically focus computations on the key spatiotemporal volumes. In this paper, we propose a motion-centric attention model for action detection in videos which imitates the human perception of saccade and fixation procedures while detecting actions in a video. Specifically, we first present a strategy to generate motion-centric locations based on the density peak of motion signals, providing reliable candidates around which actions have high possibilities to occur. Then, we introduce an attention model that conducts the saccade and fixation procedures on these candidates to observe local spatiotemporal visual information, preserve internal comprehension, and produce the action proposals on temporal bounds. Afterward, a classifier with several variants is prepared to classify the action proposals and decide which one to fixate and generate the final predictions. We show how to efficiently train our model to produce fast and accurate action detection, by scanning only a small fraction of locations in a video. The extensive experiments on three challenging datasets show promising results with both accuracy and speed.

With the rapid increase in the amount of multimedia data, video classification has become a demanding and challenging research topic. Compared with image classification, video classification requires mapping a video that contains hundreds of frames to semantic tags, which poses many challenges to the direct use of advanced models originally designed for image-oriented tasks. On the other hand, continuous frames in a video also give us more visual clues that we can leverage to achieve better classification. One of the most important clues is the context in the spatiotemporal domain. In this paper, we introduce the multiscale deep alternative neural network (DANN), a novel architecture combining the strengths of both convolutional neural network and recurrent neural networks to achieve a deep network that can collect rich context hierarchies for video classification. In particular, the DANN is stacked with alternative layers, each of which consists of a volumetric convolutional layer followed by a recurrent layer. The former acts as a local feature learner, whereas the latter is used to collect contexts. Compared with popular deep feed-forward neural networks, the DANN learns local features and their contexts from the very beginning. This setting enables preserving context evolutions, which we show to be essential for improving the accuracy of video classification. To release the full potential of the DANN, we develop a deeper version with stochastic-layer skip-connections and construct a multiscale DANN to incorporate contexts at different scales. We show how to apply the multiscale DANN for video classification with carefully designed configurations in terms of both input–output settings and training–testing methods. The DANN is shown to be robust to not only human-centric videos, but also natural videos. As there are few large-scale natural disaster video datasets, we construct a new large-scale one and make it publicly available. Experiments on four datasets show the effectiveness of our method for both human actions and natural events.

Purpose: To study the anesthetic and hemodynamic effects of etomidate-remifentanil combination treatment in laparoscopic surgery.Methods: Patients scheduled for gynecological laparoscopic surgery (n = 120) were assigned to test and control groups (60 patients each). Etomidate combined with remifentanil anesthesia was used in the test group, while propofol-remifentanil combination anesthesia was used in the control group. The effect of anesthesia on awakening time, extubation time, pain relief time, systolic blood pressure (SBP), diastolic blood pressure (DBP), and heart rate (HR) of patients before and after the extubation were observed and recorded for the two groups.Results: Excellent rating was 98.0 % in the test group, and was superior to the corresponding rating of 86.0 % in the control group. Anesthesia time, awakening time, extubation time and pain relief time were markedly shorter in the test group than in controls (p < 0.05). However, there were no significant differences in SBP, DBP and HR of patients with tracheal intubation between the two groups (p > 0.05). The results were similar in patients with laparoscopic placement. After laparoscopic placement and tracheal extubation, significant decreases in SBP and HR in the test group were seen, relative to control patients (p < 0.05).Conclusion: The anesthetic effect of etomidate combined with remifentanil is superior to that of propofol and remifentanil, and ensures stability of hemodynamic parameters such as SBP, DBP and HR during the period of anesthesia.Keywords: Etomidate, Remifentanil, Propofol, Laparoscopic surgery, Hemodynamic parameters

Good ecological Environment and Atmospheric Environment is an important part of human healthy growth and survival. With the acceleration of the urbanization process, bringing people a lot of material and spiritual enjoyment, but also increased water, soil and atmospheric environment and other serious pollution problems. Under the influence of greenhouse effect, global warming further aggravates the harm to people’s physical and mental health. In this paper, based on ZigBee and GIS technology, the content of real-time dynamic monitoring of urban atmospheric environment is deeply analyzed, and the positioning and targeted warning of pollution sources of solid pollution particles and pollution gases in the city are carried out to further prevent the spread of pollution accidents.

Person re-identification aims at identifying the same person from different non-overlapping camera views, in which one of the fundamental issues is to have a robust feature under various conditions. In order to deal with the misaligned problem, most works incline to fuse the feature of less associated patches together. Such strategy might result in the loss of their relative location information and hinder the better performance. Therefore, in this paper we introduce aligned local descriptors to preserve the information of patches’ relative location and design hierarchical global features to improve the robustness of image representation for person re-identification. We attempt to apply affine transformation to our framework and find it effective for resolving the viewpoint and pose changes. Experiments are implemented on three challenging datasets VIPeR, QMUL GRID and CUHK Campus. We obtain competitive or superior performance compared to state-of-the-art methods.

Convolutional neural networks have been verified to be exceptionally powerful on extracting semantic features, which contribute to a great progress in computer vision. However, focusing too much on the superiority, researchers seem to pay less attention to exploring CNNs' potential in other aspects, e.g. the ability to discriminate the difference. In this work we try to dig into the discriminative power of CNNs and introduce a deep discriminative network with inception module (DDN-IM) for person re-identification. Without individual feature extraction as prerequisite, input images from two different non-overlapping camera views are concatenated in depth at the beginning, followed by series of convolutional and nonlinear operations, etc. to predict their similarity. In addition, inception module is embedded in our network to boost the performance. We validate our proposal on several person re-identification datasets, CUHK01, QMUL GRID and PRID2011 included. We obtain competitive or superior performance compared to the state-of-the-art methods.

This paper considers the challenging task of long-term video interpolation. Unlike most existing methods that only generate few intermediate frames between existing adjacent ones, we attempt to speculate or imagine the procedure of an episode and further generate multiple frames between two non-consecutive frames in videos. In this paper, we present a novel deep architecture called bidirectional predictive network (BiPN) that predicts intermediate frames from two opposite directions. The bidirectional architecture allows the model to learn scene transformation with time as well as generate longer video sequences. Besides, we make attempts to extend our model to predict multiple possible procedures by sampling different noise vectors. A joint loss composed of clues in image and feature spaces and adversarial loss is designed to train our model. We demonstrate the advantages of BiPN on two benchmarks Moving 2D Shapes and UCF101 and report competitive results to recent approaches.

This paper considers the person verification problem in video surveillance systems. The goal is to verify whether or not a given pair of human body images belong to the same identity. For this purpose, we propose a method of collaborative networks which contains two kinds of novel agents. Specifically, one is implemented by an improved siamese network (iSN) and the other is employed as a deep discriminative network (DDN). The iSN explores the commonness and difference properties of pairwise feature vectors to enhance the robustness for person verification. Instead, the DDN learns to discriminate the difference of input images from the original difference space, without individual feature extraction. Both of the networks capture the correlation of the input and determine whether they are the same or not. Moreover, we introduce a collaborative learning strategy to fuse them into a unified architecture. Extensive experiments are conducted on four person verification datasets, including CUHK01, CUHK03, PRID2011 and QMUL GRID. We obtain competitive or superior performance compared to state-of-the-art methods.

In this work, we focus on a challenging task: synthesizing multiple imaginary videos given a single image. Major problems come from high dimensionality of pixel space and the ambiguity of potential motions. To overcome those problems, we propose a new framework that produce imaginary videos by transformation generation. The generated transformations are applied to the original image in a novel volumetric merge network to reconstruct frames in imaginary video. Through sampling different latent variables, our method can output different imaginary video samples. The framework is trained in an adversarial way with unsupervised learning. For evaluation, we propose a new assessment metric RIQA. In experiments, we test on 3 datasets varying from synthetic data to natural scene. Our framework achieves promising performance in image quality assessment. The visual inspection indicates that it can successfully generate diverse five-frame videos in acceptable perceptual quality.

In this paper, we introduce a joint model that learns to directly localize the temporal bounds of actions in untrimmed videos as well as precisely classify what actions occur. Most existing approaches tend to scan the whole video to generate action instances, which are really inefficient. Instead, inspired by human perception, our model is formulated based on a recurrent neural network to observe different locations within a video over time. And, it is capable of producing temporal localizations by only observing a fixed number of fragments, and the amount of computation it performs is independent of input video size. The decision policy for determining where to look next is learned by REINFORCE which is powerful in non-differentiable settings. In addition, different from relevant ways, our model runs localization and classification serially, and possesses a strategy for extracting appropriate features to classify. We evaluate our model on ActivityNet dataset, and it greatly outperforms the baseline. Moreover, compared with a recent approach, we show that our serial design can bring about 9% increase in detection performance.

Conventional pedestrian detection methods construct models based on hand-crafted features or deep learning. They are powerful but limited due to finite capabilities of single classifiers. Ensemble models escape these problems by assembling multiple classifiers using some man-made criteria which synthetically utilize information from all combined models. However, these criteria lack theoretical support. Therefore, we propose a novel ensemble deep model called collaborative deep networks where multiple deep networks are meaningfully combined in a fully-connected network. For maximizing the abilities of these deep models, we incorporate a resampling process to prepare diverse datasets and pre-train them using these resampled data. Finally, a collaborative learning method is presented to train the entire model. Experimental results prove that our approach can improve the performance of single classifiers and outperform state-of-the-art methods both on Caltech dataset and ETH dataset.

Motion information is a key factor for action recognition and has been eagerly pursued for decades. How to effectively learn motion features in Convolutional Networks (ConvNets) remains an open issue. Prevalent ConvNets often take several full frames of video as input at a time, which can be a heavy burden for network training. In this paper, we introduce a novel framework called Tube ConvNets, by substituting action tubes for full frames to reduce this burden. Tube ConvNets focus on the regions of interest (ROI) where key motions occur, and thus eliminate the distraction of irrelevant objects. Each action tube is a fraction of spatiotemporal volumes, generated by the techniques of object detection and clustering algorithm. We demonstrate the effectiveness of Tube ConvNets for action classification on UCF-101 dataset, and illustrate its potential to support fine-grained localization on UCF-Sports dataset. Source code is available at https://github.com/wangjinzhuo/tubecnn.

This paper proposes to employ deep learning model to encode local descriptors for image classification. Previous works using deep architectures to obtain higher representations are often operated from pixel level, which lack the power to be generalized to large-size and complex images due to computational burdens and internal essence capture. Our method slips the leash of this limitation by starting from local descriptors to leverage more semantical inputs. We investigate to use two layers of Restricted Boltzmann Machines (RBMs) to encode different local descriptors with a novel group sparse learning (GSL) inspired by the recent success of sparse coding. Besides, unlike the most existing pure unsupervised feature coding strategies, we use another RBM corresponding to semantic labels to perform supervised fine-tuning which makes our model more suitable for classification task. Experimental results on Caltech-256 and Indoor-67 datasets demonstrate the effectiveness of our method.

This paper presents a compact shot representation for video semantic indexing (SIN). The proposed representation consists of visual cues from only two frames, i.e., key frame (KF) and difference frame (DF), which are both constructed with spatial pyramid. The KF describes static information while the generated DF captures non-static information. Each region of DF is derived from the same location in a selected frame, which has the most salient difference compared with the key frame in that region. We introduce a variation of DF to further enhance our model. Experimental results on TRECVID SIN demonstrate that our method obtains better accuracy than the state-of-the-art, while requiring less storage space and consuming time.

Spatial pyramid (SP) representation is an extension of bag-of-feature model which embeds spatial layout information of local features by pooling feature codes over pre-defined spatial shapes. However, the uniform style of spatial pooling shapes used in standard SP is an ad-hoc manner without theoretical motivation, thus lacking the generalization power to adapt to different distribution of geometric properties across image classes. In this paper, we propose a data-driven approach to adaptively learn class-specific pooling shapes (CSPS). Specifically, we first establish an over-complete set of spatial shapes providing candidates with more flexible geometric patterns. Then the optimal subset for each class is selected by training a linear classifier with structured sparsity constraint and color distribution cues. To further enhance the robust of our model, the representations over CSPS are compressed according to the shape importance and finally fed to SVM with a multi-shape matching kernel for classification task. Experimental results on three challenging datasets (Caltech-256, Scene-15 and Indoor-67) demonstrate the effectiveness of the proposed method on both object and scene images.