Search

The increasing availability of wireless access points (APs) is leading toward human sensing applications based on Wi-Fi signals as support or alternative tools to the widespread visual sensors, where the signals enable to address well-known vision-related problems such as illumination changes or occlusions. Indeed, using image synthesis techniques to translate radio frequencies to the visible spectrum can become essential to obtain otherwise unavailable visual data. This domain-to-domain translation is feasible because both objects and people affect electromagnetic waves, causing radio and optical frequencies variations. In the literature, models capable of inferring radio-to-visual features mappings have gained momentum in the last few years since frequency changes can be observed in the radio domain through the channel state information (CSI) of Wi-Fi APs, enabling signal-based feature extraction, e.g. amplitude. On this account, this paper presents a novel two-branch generative neural network that effectively maps radio data into visual features, following a teacher–student design that exploits a cross-modality supervision strategy. The latter conditions signal-based features in the visual domain to completely replace visual data. Once trained, the proposed method synthesizes human silhouette and skeleton videos using exclusively Wi-Fi signals. The approach is evaluated on publicly available data, where it obtains remarkable results for both silhouette and skeleton videos generation, demonstrating the effectiveness of the proposed cross-modality supervision strategy.

This paper proposes the use of the FASSD-Net model for semantic segmentation of human silhouettes, these silhouettes can later be used in various applications that require specific characteristics of human interaction observed in video sequences for the understanding of human activities or for human identification. These applications are classified as high-level task semantic understanding. Since semantic segmentation is presented as one solution for human silhouette extraction, it is concluded that convolutional neural networks (CNN) have a clear advantage over traditional methods for computer vision, based on their ability to learn the representations of appropriate characteristics for the task of segmentation. In this work, the FASSD-Net model is used as a novel proposal that promises real-time segmentation in high-resolution images exceeding 20 FPS. To evaluate the proposed scheme, we use the Cityscapes database, which consists of sundry scenarios that represent human interaction with its environment (these scenarios show the semantic segmentation of people, difficult to solve, that favors the evaluation of our proposal), To adapt the FASSD-Net model to human silhouette semantic segmentation, the indexes of the 19 classes traditionally proposed for Cityscapes were modified, leaving only two labels: One for the class of interest labeled as person and one for the background. The Cityscapes database includes the category “human” composed for “rider” and “person” classes, in which the rider class contains incomplete human silhouettes due to self-occlusions for the activity or transport used. For this reason, we only train the model using the person class rather than human category. The implementation of the FASSD-Net model with only two classes shows promising results in both a qualitative and quantitative manner for the segmentation of human silhouettes.

Human action is recognized directly from the video sequences. The objective of this work is to recognize various human actions like run, jump, walk etc. Human action recognition requires some prior knowledge about actions namely, the motion estimation, foreground and background estimation. Region of interest (ROI) is extracted to identify the human in the frame. Then, optical flow technique is used to extract the motion vectors. Using the extracted features similarity measure based classification is done to recognize the action. From experimentations upon the Weizmann database, it is found that the proposed method offers a high accuracy., {"references":["A. Bobick and J. Davis. The recognition of human movement using temporal templates. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(3):257–267, 2002.","Antonios Oikonomopoulos, Ioannis Patras and Maja Pantic. \"Spatiotemporal Salient Points For visual Recognition of Human actions.\" IEEE Transactions on Image Processing Vol. 36, no. No. 3. (2006).","Antonios Oikonomopoulos, Maja Pantic, Ioannis Patras \"Sparse B-spline polynomial descriptors for human activity recognition\" in 2009.","Danielweinland, RemiRonfard and Edmond Boyer, \"A Survey of Vision- Based Methods for Action Representation, Segmentation and Recognition\", October 18, 2010.","Droogenbroeck, O. Barnich and M. Van. \"ViBe: A universal background subtraction algorithm for video sequences.\" IEEE Transactions on Image Processing, June 2011. 20(6):1709-1724.","Enric Meinhardt-Llopis, Javier Sanchez, Daniel Kondermann , \"Horn–Schunck Optical Flow with a Multi-Scale Strategy\", Image Processing On Line, 3 (2013), pp. 151–172.","I. Laptev, M. Marszalek, C. Schmid, and B. Rozenfeld. Learning realistic human actions from movies. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1–8, 2008.","J. K. Aggarwal, Q. Cai, W. Liao, and B. Sabata, Articulated and elastic non-rigid motion: a review, in Workshop on Motion of Non-Rigid and Articulated Objects, Austin, TX, 1994, pp. 2–14.","J. Yamato, J. Ohya, and K. Ishii. Recognizing human action in time-sequential images using hidden markov model. In CVPR, 1992.\n[10]\tK. Schindler and L. van Gool. Action snippets: How many frames does human action recognition require? In CVPR, 2008.\n[11]\tKeigo Takahara, Takashi Toriu and Thi Thi Zin. \"Making Background Subtraction Robustto Various Illumination Changes.\" IJCSNS International Journal of Computer Science and Network Security, March 2011: VOL.11 No.3.\n[12]\tM. Blank, L. Gorelick, E. Shechtman, M. Irani, and R. Basri. \"Actions as space-time shapes.\" In ICCV, 2005.\n[13]\tM. Rodriguez, J. Ahmed, and M. Shah. Action MACH a spatio-temporal Maximum Average Correlation Height filter for action recognition. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1–8, 2008.\n[14]\tMaja Pantic, Alexpentland and Thomas Huang, \"Human Computing and Machine Understanding of Human Behaviour\", ICMI'06, November 2-4, 2006.\n[15]\tN.A. Deepak and U.N.Sinha, \"Silhouette Based Human Motion Detection and Recognising their Actions from the captured Video Streams\", IntJ. Advanced Networking and Applications Volume: 02, Issue: 05, Pages: 817-823(2011).\n[16]\tR. Polana and R. Nelson. Low level recognition of human motion. In IEEE Workshop onNonrigid and Articulate Motion, 1994."]}

Human silhouette extraction has been a primary step to estimate human poses or classify activities from videos. While the accuracy of human silhouettes has great impact on the follow-on human pose/gait estimation, it has been important to guarantee the highly-accurate extraction of human silhouettes. However, traditional methods such as motion segmentation can be fragile due to the complexity of real-world environment. In this paper, we propose an automated human silhouette extraction algorithm to attain this highly-demanded task. In our proposed scheme, the initial motion segmentation of foreground objects was roughly computed by Stauffer's background subtraction using Gaussian mixtures, and then refined by the proposed Laplacian fitting scheme. In our method, the candidate regions of human objects are taken as the initial input, their Laplacian matrices are constructed, and Eigen mattes are then obtained by minimizing on Laplacian matrices. RANSAC algorithm is then applied to fit the Eigen mattes iteratively with inliers of the initially estimated motion blob. Finally, the foreground human silhouettes are obtained from the optimized matte fitting. Experimental results on a number of test videos validated that the proposed Laplacian fitting scheme enhances the accuracy in automated human silhouette extraction, exhibiting a potential use of our Laplacian fitting algorithm in many silhouette-based applications such as human pose estimation. Scopus

International audience; The union of Histograms of Oriented Gradients based descriptors and of a Support Vector Machine classifier is a classic and efficient method for human detection in the images. Conversely, as often in detection, accurate segmentation of these persons is not performed. Many applications however need it. this paper tackles the problem of giving rise to information that will guide the final segmentation step. It presents a method which uses the union mention above to relate to each contour segment a likelihood degree of being part of a human silhouette.

Cataloged from PDF version of article. Most of the approaches to human action recognition tend to form complex models which require lots of parameter estimation and computation time. In this study, we show that, human actions can be simply represented by pose without dealing with the complex representation of dynamics. Based on this idea, we propose a novel pose descriptor which we name as Histogram-of-Oriented-Rectangles (HOR) for representing and recognizing human actions in videos. We represent each human pose in an action sequence by oriented rectangular patches extracted over the human silhouette. We then form spatial oriented histograms to represent the distribution of these rectangular patches. We make use of several matching strategies to carry the information from the spatial domain described by the HOR descriptor to temporal domain. These are (i) nearest neighbor classification, which recognizes the actions by matching the descriptors of each frame, (ii) global histogramming, which extends the idea of Motion Energy Image proposed by Bobick and Davis to rectangular patches, (iii) a classifier-based approach using Support Vector Machines, and (iv) adaptation of Dynamic Time Warping on the temporal representation of the HOR descriptor. For the cases when pose descriptor is not sufficiently strong alone, such as to differentiate actions "jogging" and "running", we also incorporate a simple velocity descriptor as a prior to the pose based classification step. We test our system with different configurations and experiment on two commonly used action datasets: the Weizmann dataset and the KTH dataset. Results show that our method is superior to other methods on Weizmann dataset with a perfect accuracy rate of 100%, and is comparable to the other methods on KTH dataset with a very high success rate close to 90%. These results prove that with a simple and compact representation, we can achieve robust recognition of human actions, compared to complex representations. (C) 2009 Elsevier B.V. All rights reserved.