Search

In this paper a novel eye-tracking device designed which uses tiny CCD cameras to capture the eye image from the screen with the help of a video capture card. In the head gesture control mode, a light source projector is turned on and the CCD camera detects the position of the light source. The locations of the spots on the screen and on the image pupil of the eye image are calculated, compared with the previous point and are subsequently mapped to the point on the screen. The movement increment-coordinate control is also discussed which could improve the ease of use of the computer. We investigate the use of non-rigid head fixation using a helmet that constrains only general head orientation and allows some freedom of movement. Device results simulated with the help of software which achieves excellent timing performance due to the use of live data streaming, instead of the traditionally employed data storage mode for processing analogous eye position data. [ABSTRACT FROM AUTHOR]

Hand gesture control is an efficient modality for telerobot control because of gesture expressiveness and naturalness. This paper discusses a collaborative cybernetic system, where telerobots are controlled simultaneously by a group of distributed operators over the network to accomplish a task. Computer vision algorithms for hand gesture recognition are introduced to facilitate the human–robot interface. The gestures are converted into commands that are delivered to robots for dexterous task completion. Commands from multiple operators are aggregated by a collaboration protocol into a single control stream. The aggregation is updated according to operators’ performance. A distributed conflict and error detection–prediction network is designed and applied to a case study of collaborative control for a robotic nuclear decommissioning task. Operators use hand gestures to command telerobots to disassemble facilities in a contaminated area. The hypothesis is tested that collaborative control is more effective and less susceptible to conflicts/errors than the standard single-operator control. During collaboration, operators performed gesture commands simultaneously to control a set of robots. The system can reliably recognise operators’ hands with a 96% accuracy in cluttered backgrounds. Collaboration between expert and novice operators can reduce the time to complete a multi-step task by 45% on average. [ABSTRACT FROM PUBLISHER]

This study proposed a method to enable a humanoid robot to step up onto a stair by imitating the step-up motion of a human and to accomplish a lift and carry event in HuroCup of Federation of International RoboSports Association. The step-up motion, divided into five states, was captured by a Kinect sensor, and the human joints corresponded to the humanoid robot joints. Selected servomotors and their angle variation were matched with that of human joint numbers by a designed fuzzy inference system on the basis between the human and the humanoid robot joints. Then, the rest of the robot motors were adjusted by the zero moment point obtained from force-sensing registers to maintain stability. Next, two intermediate transition states were added between each state of the humanoid robot step-up to maintain its balance and reduce motor damage. Finally, to be applied in a real lift and carry event, a vision system was integrated to recognize the edge of a color board and determine a suitable site for the step-up. With these functions integrated, the robot under the proposed method was verified to successfully achieve the task of the lift and carry event without losing its balance or falling. [ABSTRACT FROM AUTHOR]

Biometric authentication technologies are rapidly gaining popularity, and hand gestures are emerging as a promising biometric trait due to their rich physiological and behavioral characteristics. Hand gesture authentication can be categorized as defined hand gesture authentication and random hand gesture authentication. Unlike defined hand gesture authentication, random hand gesture authentication is not constrained to specific hand gesture types, allowing users to perform hand gestures randomly during enrollment and verification, thus more flexible and friendly. However, in random hand gesture authentication, the model needs to extract more generalized physiological and behavioral features from different viewpoints and positions without gesture templates, which is more challenging. In this paper, we present a novel efficient Temporal-Segment-Set-Network (TS2N) that directly extracts both behavioral and physiological features from a single RGB video to further enhance the performance of random hand gesture authentication. Our method adopts a new motion pseudo-modality and leverages a set-based representation to capture behavioral characteristics online. Additionally, we propose a channel-spatial attention mechanism, Contextual Squeeze-and-Excitation Network (CoSEN), to better abstract and understand physiological characteristics by explicitly modeling the channel-spatial interdependence, thereby adaptively recalibrating channel-specific and spatial-specific responses. Extensive experiments on the largest public hand gesture authentication dataset SCUT-DHGA demonstrate TS2N's superiority against 21 state-of-the-art models in terms of EER (5.707% for full version and 6.664% for lite version) and computational cost (98.9022G for full version and 46.3741G for lite version). The code is available at https://github.com/SCUT-BIP-Lab/TS2N. [ABSTRACT FROM AUTHOR]

In an era where man-machine interaction increasingly uses remote sensing, gesture recognition through use of the micro-Doppler (mD) effect is an emerging application which has attracted great interest. It is a sensible solution and here the authors show its potential for detecting aperiodic human movements. In this study, the authors classify ten hand gestures with a set of handcrafted features using simulated mD signatures generated from Kinect skeleton data. Data augmentation in the form of synthetic minority oversampling technique has been applied to create synthetic samples and classified with the support vector machine and K-nearest neighbour classifier with classification rate of 71.1 and 51% achieved. Finally, using weights generated by an action pair based one vs. one classification layer improves classification accuracy by 24.7 and 28.4%. [ABSTRACT FROM AUTHOR]

In this paper, we present the sensor fusion of a quad rotor aircraft. There are two main applications examined in this study, namely, gesture control and object detection. In gesture control, we can use hand and gesture to control the quadcopter instead of a traditional joystick. In the second application, the quadcopter can be applied to detecting an object surface automatically. The quadcopter can fly along an object to identify its color and circle around the object from a high altitude to a low altitude. If an abnormal area of the object is detected, the quadcopter will hover at that position, and the inspector can determine whether the object needs to be repaired through the images transmitted from the quadcopter. In addition, a voice broadcast system is integrated in the quadcopter, which can report the current status of the quadcopter. The inspector does not need to pay attention to the quadcopter and only needs to listen to the speaker. Ultrasonic sensors with a fuzzy logic system are applied to automatic circling and descending flight control. [ABSTRACT FROM AUTHOR]

A study of methods that determine surface curvature in real and synthetic range images is considered here. Four new determination methods are introduced (two convolution-based and two spline-based). These new methods, as well as a number of existing determination methods, are comparatively evaluated in terms of their (1) accuracy and (2) computational performance (i.e., run time). Their behavior in two common but challenging multimedia tasks (surface rendering and object discrimination based in whole or part on curvature determination) is also considered. Our evaluations also include an analysis of (1) which methods are most suitable for application in particular scenarios and (2) behavior of the methods with respect to parameter settings. One of the considered use cases includes application to consumer-grade Kinect sensor data. [ABSTRACT FROM AUTHOR]

With advances in technology, motion sensing has been applied in a wide range of devices. The system used for motion sensing captures images via a camera and analyzes the acquired images, also known as human-computer interaction, wherein gesture-controlled machines are considered as the most convenient kind and are most commonly used. However, the existing gesture recognition algorithm often has a long computation time and uses a huge amount of memory, preventing the algorithm from being implemented in embedded systems. To alleviate these problems, in this paper, we propose to reduce the computation time of the system by employing the lifting-based discrete wavelet transform (DWT). The different frequency bands featured in a lifting-based discrete wavelet can swiftly distinguish the face region from the hand region, reduce the image resolution, and thus reduce memory usage. In addition to recognizing gestures in a swift and accurate manner, the gesture recognition approach proposed in this study is also compatible with embedded systems. Our experimental results suggest that the proposed approach can reduce the execution time by up to 66% while achieving a high identification rate. [ABSTRACT FROM AUTHOR]

Gesture recognition has raised wide attention in computer vision owing to its many applications. However, the task of video-based large-scale gesture recognition yet faces many challenges, since many gesture-irrelevant factors like the background may disturb the recognition accuracy. To better recognize gestures with large-scale videos, we propose a method based on RGB-D data in this paper, where the “RGB-D” means RGB and depth data captured simultaneously by specific devices like Kinect. To learn gesture details better, we first use an adaptive frame unification strategy to unify the frame number of inputs, and then the RGB and depth data are sent to the C3D model to extract spatiotemporal features, respectively. In order to alleviate the interference of gesture-irrelevant factors, the saliency theory is also employed to generate auxiliary data. Next the features of these data are combined to boost the performance, which can also avoid unreasonable synthetic data, since the dimension of C3D features is uniform. Finally the performances of several classifiers are tested and the best one of SVM classifier is selected to output the ultimate accuracy. Our approach achieves 52.04% and 59.43% accuracy on the validation and testing subset of the Chalearn LAP IsoGD, respectively, both of which outperform our results in the chalearn LAP Large-scale Gesture Recognition Challenge as reported in ICPR 2016. [ABSTRACT FROM AUTHOR]

In this paper, we study one-shot learning gesture recognition on RGB-D data recorded from Microsoft’s Kinect. To this end, we propose a novel bag of manifold words (BoMW)-based feature representation on symmetric positive definite (SPD) manifolds. In particular, we use covariance matrices to extract local features from RGB-D data due to its compact representation ability as well as the convenience of fusing both RGB and depth information. Since covariance matrices are SPD matrices and the space spanned by them is the SPD manifold, traditional learning methods in the Euclidean space, such as sparse coding, cannot be directly applied to them. To overcome this problem, we propose a unified framework to transfer the sparse coding on SPD manifolds to the one on the Euclidean space, which enables any existing learning method to be used. After building BoMW representation on a video from each gesture class, a nearest neighbor classifier is adopted to perform the one-shot learning gesture recognition. Experimental results on the ChaLearn gesture data set demonstrate the outstanding performance of the proposed one-shot learning gesture recognition method compared against the state-of-the-art methods. The effectiveness of the proposed feature extraction method is also validated on a new RGB-D action recognition data set. [ABSTRACT FROM AUTHOR]

The Kinect device is being increasingly used in conjunction with rehabilitative actions. However, the use of Kinect as a skeletal tracking system requires several further modifications and technological breakthroughs. This study used inertial measurement units (IMUs) to complement skeletal tracking with the Kinect. The IMUs were used to compensate for errors in calculating shoulder and elbow joint angles detected by the Kinect device while the patients performed rehabilitation movements. Thirty normal participants were recruited, and their shoulder and elbow joint angles were recorded and calculated using the Kinect and IMUs while they moved during movement games. If movement with a larger measuring error was detected, the measurement was directed to the IMU to calculate the angle and calibrate the angles measured by the Kinect device. The mean percent errors of the Kinect measurements with respect to the IMU measurement at the shoulder joint during shoulder flexion and rotation at 90° of shoulder flexion were 15.08 ± 4.13 and 26.00 ± 7.41%, respectively. The mean percent errors of the Kinect measurements with respect to the IMU measurements at the elbow joint during shoulder flexion, shoulder rotation at 90° of shoulder abduction, and shoulder rotation at 90° of shoulder flexion were 12.92 ± 2.43, 17.75 ± 4.91, and 23.3 ± 7.01%, respectively. The mean percent errors for the participants’ shoulders in Game 2 and Game 3 were 15.47 ± 4.88 and 28.13 ± 8.51%, respectively, and the mean percent errors of the participants’ elbows in Game 3 were 55.62 ± 13.74%. The proposed method to calibrate the angles detected using the Kinect have a greater mean accuracy rate (84.58%) and a higher processing rate (10 ms/frame) than traditional methods that use only Kinect or IMUs. The proposed system increases the accuracy of movement detected by the Kinect device, and this increases the processing rate of the IMUs, thereby improving clinical practicality. [ABSTRACT FROM AUTHOR]

We present a new bare-hand gesture interface for large-screen interaction in which multiple users can participate simultaneously and interact with virtual content directly. To better reflect the intent of our new interface, we have created a new type of hardware system with a large hybrid display, named ThunderPunch. Unlike the conventional method, which involves positioning the camera in front, the cameras are mounted on the ceiling so that they avoid covering the large screen. To achieve bare-hand interaction in this hardware structure, we propose real-time algorithms that detect multiple body poses and recognize punching and touching gestures from top-view depth images. A pointing and touching test shows that the proposed algorithm is usable and that it outperforms other algorithms. In addition, we created a game to make the best use of the proposed system. [ABSTRACT FROM AUTHOR]

Hand posture estimation is a building block in hand gesture detection systems. One of the most popular techniques in this field is to generate a 3D hand model and find its optimal structure using an optimisation algorithm. The main advantages of such methods are flexibility to model complex hand postures, ability to better address occlusions, and estimating both discrete and contentious poses. However, rendering and finding optimal structure for a 3D hand model are computationally expensive. Popular models in the literature are made of mesh or simple components. This work first analyses a number of hand models in the literature. Then, a new hand model with simple components is proposed. The performance of the proposed model is benchmarked on 50 poses extracted from four standard datasets. The results show that the proposed 3D model shows superior results compared to the best models in the literature. [ABSTRACT FROM AUTHOR]

Gesture-based systems allow users to interact with a virtual reality application in a natural way. Visual feedback for the gesture-based interaction technique has an impact on the performance and the hand instability making the manipulation of the object less precise. This paper investigated two new interaction techniques in a virtual environment. It describes the influence of natural and non-natural virtual feedback in the selection process using the GITDVR-G interaction technique, which consists of a grasping visual feedback. The GITDVR-G was evaluated in a virtual knee surgery training system. The results showed that it was effective in terms of the task completion time, and that the participants preferred the natural grasping visual feedback. Besides that, the precise manipulation in a newly-designed interaction technique (Precise GITDVR-G) was evaluated. The Precise GITDVR-G includes a normal manipulation mode and a precise manipulation mode that can be triggered by hand gestures. During the precise manipulation mode, an inset view will appear and move with the selected object to provide a better view to users, while the movements of the virtual hand are scaled down to improve the precision. Four different configurations of the precise manipulation technique were evaluated, and the results showed that the unimanual control method with an inset view performed better in terms of the task performance time and the subjective feedback. The finding suggested that the realistic virtual grasping visual feedback can be applied in a virtual hand interaction technique, and that the inset view feature is helpful in the precise manipulation. [ABSTRACT FROM AUTHOR]

In industrial human-robot collaboration, variability commonly exists in the operation environment and the components, which induces uncertainty and error that require frequent manual intervention for rectification. Conventional teach pendants can be physically demanding to use and require user training prior to operation. Thus, a more effective control interface is required. In this paper, the design and evaluation of a contactless gesture control system using Leap Motion is described. The design process involves the use of RULA human factor analysis tool. Separately, an exploratory usability test was conducted to compare three usability aspects between the developed gesture control system and an off-the-shelf conventional touchscreen teach pendant. This paper focuses on the user-centred design methodology of the gesture control system. The novelties of this research are the use of human factor analysis tools in the human-centred development process, as well as the gesture control design that enable users to control industrial robot’s motion by its joints and tool centre point position. The system has potential to use as an input device for industrial robot control in a human-robot collaboration scene. The developed gesture control system was targeting applications in system recovery and error correction in flexible manufacturing environment shared between humans and robots. The system allows operators to control an industrial robot without the requirement of significant training. [ABSTRACT FROM AUTHOR]

Technical advances in movement sensing and processing offer musicians greater scope to perform in ways that reflect their individual body signature. However, there is currently an asymmetry between innovations in gestural system design and the development of movement skills and awareness among performers. In particular, stagecraft for musicians to master and perform effectively with gesture-controlled instruments is underdeveloped. Few formal guidelines exist to equip musicians with the physical skills to calibrate whole body movements spatially in order to achieve precise and nuanced control of sound when using non-tactile controllers. This paper firstly examines the relevance of embodied interaction design approaches and movement awareness disciplines in addressing this challenge. An outline of the work Bodyscapes follows, composed for a novel gestural instrument, the Telechord, voice and drums. The piece explores the influence of metaphor-based mapping strategies on gesture-controlled performance experiences and skills. Analysis of autoethnographic insights that emerge from the work illuminates ways in which musicians inhabit their bodies to cultivate new forms of musicianship when performing with gestural instruments, drawing on techniques that include movement improvisation and kinaesthetic imagery. [ABSTRACT FROM AUTHOR]

The gesture recognition in computer vision on life, work and the application of technology products occupy an important position. In this paper, we use the finger valleys, distance mapping and triangular method (FVDMTM) to precisely recognize the fingers. FVDMTM adopt three novel ideas: first, we use the finger valleys to distinguish each finger. It is robust against intentional deformation of the fingers. Second, we employ a distance mapping method effectively to detect the valleys between the fingers. Third: we use the center-of-gravity of the palm as the original point for angle calculation of each finger by the triangular method. This let us to get precise finger angles of the test image. The experimental results demonstrate that our scheme is an effective and correct method for finger recognition. [ABSTRACT FROM AUTHOR]

With the thriving of different natural interaction paradigms—such as gesture-based interfaces—it becomes important to understand how these novel interfaces can influence users’ performance when it comes to their age. Recent advances made in human-computer interaction allow us to manipulate digital contents more intuitively; however, no work has yet been reported that systematically evaluates how gestural interfaces may influence the performance of different user groups. Different optical sensors, which allow human body acquisition with reliable accuracy, have been released, and with the appearance of such controllers for gesture recognition, it becomes important to understand if different age-related groups display similar performance levels concerning gestural interaction, or, on the other hand, if specific sensors could induce better results than others when dealing with users of different age brackets. In this article, we compare two gesture-sensing devices (Microsoft Kinect and Leap Motion) using the Fitts’ law model to evaluate target acquisition performance, with relation to three user groups: children, young adults and older adults. This case study involved 60 participants that were asked to perform a simple continuous selection task as quickly and accurately as possible using one of the devices for gestural recognition. Indeed, performance results showed statistically significant differences among the age groups in the selection task accomplished. However, when considering the users’ performance with regard to both input devices compared side by side, there were no significant differences in each group of users. We believe this situation could imply that the device itself might not have influenced the users’ performance, but actually the users’ age might. The participants feedback was interesting on account of their behaviors and preferences: Although there are no significant differences in performance, there could be when it comes to user preference. [ABSTRACT FROM AUTHOR]

A new control system of a hand gesture-controlled wheelchair (EWC) is proposed. This smart control device is suitable for a large number of patients who cannot manipulate a standard joystick wheelchair. The movement control system uses a camera fixed on the wheelchair. The patient’s hand movements are recognized using a visual recognition algorithm and artificial intelligence software; the derived corresponding signals are thus used to control the EWC in real time. One of the main features of this control technique is that it allows the patient to drive the wheelchair with a variable speed similar to that of a standard joystick. The designed device “hand gesture-controlled wheelchair” is performed at low cost and has been tested on real patients and exhibits good results. Before testing the proposed control device, we have created a three-dimensional environment simulator to test its performances with extreme security. These tests were performed on real patients with diverse hand pathologies in Mohamed Kassab National Institute of Orthopedics, Physical and Functional Rehabilitation Hospital of Tunis, and the validity of this intelligent control system had been proved. [ABSTRACT FROM AUTHOR]

The use of cutting-edge technologies such as wearable devices to control reactive audiovisual systems are rarely applied in more conventional stage performances, such as opera performances. This work reports a cross-disciplinary approach for the research and development of the WMTSensor-Glove, a data-glove used in an opera performance to control audiovisual elements on stage through gestural movements. A system architecture of the interaction between the wireless wearable device and the different audiovisual systems is presented, taking advantage of the Open Sound Control (OSC) protocol. The developed wearable system was used as audiovisual controller in “As sete mulheres de Jeremias Epicentro”, a portuguese opera by Quarteto Contratempus, which was premiered in September 2017. [ABSTRACT FROM AUTHOR]

Bimanual gestures are of the utmost importance for the study of motor coordination in humans and in everyday activities. A reliable detection of bimanual gestures in unconstrained environments is fundamental for their clinical study and to assess common activities of daily living. This paper investigates techniques for a reliable, unconstrained detection and classification of bimanual gestures. The work assumes the availability of inertial data originating from the two hands/arms, builds upon a previously developed technique for gesture modeling based on Gaussian Mixture Modeling (GMM) and Gaussian Mixture Regression (GMR), and compares different modeling and classification techniques, which are based on a number of assumptions inspired by literature about how bimanual gestures are represented and modeled in the brain. Experiments show results related to 5 everyday bimanual activities, which have been selected on the basis of three main parameters: (not) constraining the two hands by a physical tool, (not) requiring a specific sequence of single-hand gestures, being recursive (or not). In the best performing combination of modeling approach and classification technique, we achieve overall accuracy, precision, recall and F1-score above 80 % . [ABSTRACT FROM AUTHOR]

This paper presents a new and an effective sensing and actuating approach to control a mobile robotic arm using gestures and body postures. This makes human-machine interaction and sensing more receptive and intuitive for the user. Programming robots to perform human-like tasks can be hard and time consuming. However, controlling a robot by gestures and body postures is a simple method that offers a simpler and direct approach to perform these human-like tasks. Therefore, the overall goal of this paper is to design and implement a mobile robotic arm that is commanded by human gestures, which are captured by a skeletal motion sensing device and processed through a digital controller to manage the movement of the mobile robotic arm. A complete system with integrated hardware and software subsystems were designed, constructed, and tested. Copyright © 2018 Praise Worthy Prize S.r.l. - All rights reserved. [ABSTRACT FROM AUTHOR]

This paper proposes a novel human-computer interaction system exploiting gesture recognition. It is based on the combined usage of an head-mounted display and a multi-modal sensor setup including also a depth camera. The depth information is used both to seamlessly include augmented reality elements into the real world and as input for a novel gesture-based interface. Reliable gesture recognition is obtained through a real-time algorithm exploiting novel feature descriptors arranged in a multi-dimensional structure fed to an SVM classifier. The system has been tested with various augmented reality applications including an innovative human-computer interaction scheme where virtual windows can be arranged into the real world observed by the user. [ABSTRACT FROM AUTHOR]

One of the key technologies that lays behind the human-machine interaction and human motion diagnosis is the limbs motion tracking. To make the limbs tracking efficient, it must be able to estimate a precise and unambiguous position of each tracked human joint and resulting body part pose. In recent years, body pose estimation became very popular and broadly available for home users because of easy access to cheap tracking devices. Their robustness can be improved by different tracking modes data fusion. The paper defines the novel approach--orientation based data fusion--instead of dominating in literature position based approach, for two classes of tracking devices: depth sensors (i.e., Microsoft Kinect) and inertial measurement units (IMU). The detailed analysis of their working characteristics allowed to elaborate a new method that let fuse more precisely limbs orientation data from both devices and compensates their imprecisions. The paper presents the series of performed experiments that verified the method's accuracy. This novel approach allowed to outperform the precision of position-based joints tracking, the methods dominating in the literature, of up to 18%. [ABSTRACT FROM AUTHOR]

Copyright of Technical Transactions / Czasopismo Techniczne is the property of Sciendo and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In recent years, depth cameras (such as Microsoft Kinect and ToF cameras) have gained much popularity in computer graphics, visual computing and virtual reality communities due to their low price and easy availability. While depth cameras (e.g. Microsoft Kinect) provide RGB images along with real-time depth information at high frame rate, the depth images often suffer from several artifacts due to inaccurate depth measurement. These artifacts highly degrade the visual quality of the depth frames. Most of these artifacts originate from two main sources—the missing/invalid depth values and fluctuating valid depth values on the generated contents. In this paper, we propose a new depth image enhancement method, for the contents of depth cameras, which addresses these two main sources of artifacts. We introduce a robust 1D Least Median of Squares (1D LMedS) approach to estimate the depth values of those pixels which have missing/invalid depth values. We use a sequence of frames to look for invalid depth values (considered as outliers), and finally, replace those values with stable and more plausible depth values. By doing so, our approach improves the unstable nature of valid depth values in captured scenes that is perceived as flickering. We use self-recorded and reference datasets along with reference methods to evaluate the performance of our proposed 1D LMedS. Experimental results show improvements both for static and moving parts of a scene. [ABSTRACT FROM AUTHOR]

With the increasing popularity of augmented reality (AR) services, providing seamless human-computer interactions in the AR setting has received notable attention in the industry. Gesture control devices have recently emerged to be the next great gadgets for AR due to their unique ability to enable computer interaction with day-to-day gestures. While these AR devices are bringing revolutions to our interaction with the cyber world, it is also important to consider potential privacy leakages from these always-on wearable devices. Specifically, the coarse access control on current AR systems could lead to possible abuse of sensor data. Although the always-on gesture sensors are frequently quoted as a privacy concern, there has not been any study on information leakage of these devices. In this article, we present our study on side-channel information leakage of the most popular gesture control device, Myo. Using signals recorded from the electromyography (EMG) sensor and accelerometers on Myo, we can recover sensitive information such as passwords typed on a keyboard and PIN sequence entered through a touchscreen. EMG signal records subtle electric currents of muscle contractions. We design novel algorithms based on dynamic cumulative sum and wavelet transform to determine the exact time of finger movements. Furthermore, we adopt the Hudgins feature set in a support vector machine to classify recorded signal segments into individual fingers or numbers. We also apply coordinate transformation techniques to recover fine-grained spatial information with low-fidelity outputs from the sensor in keystroke recovery. We evaluated the information leakage using data collected from a group of volunteers. Our results show that there is severe privacy leakage from these commodity wearable sensors. Our system recovers complex passwords constructed with lowercase letters, uppercase letters, numbers, and symbols with a mean success rate of 91%. [ABSTRACT FROM AUTHOR]

Multimodal interactions provide users with more natural ways to manipulate virtual 3D objects than using traditional input methods. An emerging approach is gaze modulated pointing, which enables users to perform object selection and manipulation in a virtual space conveniently through the use of a combination of gaze and other interaction techniques (e.g., mid-air gestures). As gaze modulated pointing uses different sensors to track and detect user behaviours, its performance relies on the user's perception on the exact spatial mapping between the virtual space and the physical space. An underexplored issue is, when the spatial mapping differs with the user's perception, manipulation errors (e.g., out of boundary errors, proximity errors) may occur. Therefore, in gaze modulated pointing, as gaze can introduce misalignment of the spatial mapping, it may lead to user's misperception of the virtual environment and consequently manipulation errors. This paper provides a clear definition of the problem through a thorough investigation on its causes and specifies the conditions when it occurs, which is further validated in the experiment. It also proposes three methods (Scaling, Magnet and Dual-gaze) to address the problem and examines them using a comparative study which involves 20 participants with 1040 runs. The results show that all three methods improved the manipulation performance with regard to the defined problem where Magnet and Dual-gaze delivered better performance than Scaling. This finding could be used to inform a more robust multimodal interface design supported by both eye tracking and mid-air gesture control without losing efficiency and stability. [ABSTRACT FROM AUTHOR]

Hand detection and gestures recognition have become very popular in recent human-computer interaction systems. Although several methods of hand detection have been proposed in the literature, they exist few methods that use the wrist as a factor of detection, others impose constraints on the length of the sleeves and on the orientation of the hand. In this work, we present a new two-stage algorithm of wrist localization designed for hand detection and gestures recognition systems. The first stage of the algorithm consists in separating the skin region containing the hand from the background, and in the second stage, the wrist is localized from the resulted skin mask. The main contribution of the proposed method is based on the analysis of corners along the contour of the skin masks to localize the wrist emplacement. Based on an evaluation on 437 color images with their ground-truth and three sets of skin masks, we compared our method with other efficient methods of literature and the results obtained were very satisfactory. [ABSTRACT FROM AUTHOR]

We show that the Kinematic Theory produces synthesized stroke gestures that 'look and feel' the same and hold the same statistical characteristics as human-generated gestures. Previous research in this vein has conducted such comparison from the classification accuracy performance, which is a legitimate though indirect measure. In this article, we synthesized two well-known public data sets comprising unistroke and multistroke gestures. We then compared geometric, kinematic and articulation aspects of human and synthetic gestures, and found no practical differences between both populations. We also conducted an online survey involving 236 participants and found that it is very difficult to tell human and synthetic gestures apart. We can finally be confident that synthesized gestures are actually reflective of how users produce stroke gestures. In sum, this work enables a deeper understanding of synthetic gestures' production, which can inform the design of better gesture sets and development of more accurate recognizers. [ABSTRACT FROM AUTHOR]

In this research, we conducted an elicitation study with 54 participants to evaluate user-defined mobile gestures given three types of operations, handheld postures, and interaction spaces, respectively, to investigate the effects of these factors on mental model. The results of the elicitation study revealed that each of the three factors significantly affected the users' mental models, which could be reflected by the characteristics (e.g., nature, number of steps, and spatial utility) of the user-defined mobile gestures. In addition, we found that users tended to assign different roles to mid-air and on-screen gestures based on their motor ability given a handheld gesture and their interpretation of the characteristics and requirements of an operation. We further derived a set of gestures commonly defined in the elicitation study for a list of representative operations, and we share our insights into users' mental models and the implications on future mobile-gesture design in different contexts. [ABSTRACT FROM AUTHOR]

This article identifies, catalogues, and discusses factors that are responsible for causing visual impairment of either a pathological or situational nature for touch and gesture input on smart mobile devices. Because the vast majority of interactions with touchscreen devices are highly visual in nature, any factor that prevents a clear, direct view of the mobile device's screen can have potential negative implications on the effectiveness and efficiency of the interaction. This work presents the first overview of such factors, which are grouped in a catalogue of users, devices, and environments. The elements of the catalogue (e.g., psychological factors that relate to the user, or the social acceptability of mobile device use in public that relates to the social environment) are discussed in the context of current eye pathology classification from medicine and the recent literature in human-computer interaction on mobile touch and gesture input for people with visual impairments, for which a state-of-the-art survey is conducted. The goal of this work is to help systematize research on visual impairments and mobile touchscreen interaction by providing a catalogue-based view of the main causes of visual impairments affecting touch and gesture input on smart mobile devices. [ABSTRACT FROM AUTHOR]

In the last few years, the use of gestural data has become a key enabler for human-computer interaction (HCI) applications. The growing diffusion of low-cost acquisition devices has thus led to the development of a class of middleware aimed at ensuring a fast and easy integration of such devices within the actual HCI applications. The purpose of this paper is to present a modular middleware for gestural data and devices management. First, we describe a brief review of the state of the art of similar middleware. Then, we discuss the proposed architecture and the motivation behind its design choices. Finally, we present a use case aimed at demonstrating the potential uses as well as the limitations of our middleware. [ABSTRACT FROM AUTHOR]

Weight training leads to muscle injury for inexperience exercise. The deadlift form is one of high risk posture of weight training. Therefore, the prevention of incorrect deadlift form needs an expert trainer. Some people are not be able to effort the private trainer cost. This research presents the deadlift form analysis system using Microsoft Kinect. Microsoft Kinect can be applied to detect the deadlift form with its full body motion capability the Chaffin’s biomechanical modeling. The Chaffin’s model is used to calculate two forces, compression force and shear force, on lumbosacral disc (L5/S1). Results of this research can help the people to practice the corrected deadlift form and reducing injuries on lumbosacral disc (L5/S1) using Microsoft Kinect. The accuracy of this system is 80.9%. [ABSTRACT FROM AUTHOR]

We propose a function-based temporal pooling method that captures the latent structure of the video sequence data - e.g., how frame-level features evolve over time in a video. We show how the parameters of a function that has been fit to the video data can serve as a robust new video representation. As a specific example, we learn a pooling function via ranking machines. By learning to rank the frame-level features of a video in chronological order, we obtain a new representation that captures the video-wide temporal dynamics of a video, suitable for action recognition. Other than ranking functions, we explore different parametric models that could also explain the temporal changes in videos. The proposed functional pooling methods, and rank pooling in particular, is easy to interpret and implement, fast to compute and effective in recognizing a wide variety of actions. We evaluate our method on various benchmarks for generic action, fine-grained action and gesture recognition. Results show that rank pooling brings an absolute improvement of 7-10 average pooling baseline. At the same time, rank pooling is compatible with and complementary to several appearance and local motion based methods and features, such as improved trajectories and deep learning features. [ABSTRACT FROM PUBLISHER]

Modern cars continue to offer more and more functionalities due to which they need a growing number of commands. As the driver tries to monitor the road and the graphic user interface simultaneously, his/her overall efficiency is reduced. In order to reduce the visual attention necessary for monitoring, a gesture-based user interface is very important. In this paper, gesture recognition for a vehicle through impulse radio ultra-wideband (IR-UWB) radar is discussed. The gestures can be used to control different electronic devices inside a vehicle. The gestures are based on human hand and finger motion. We have implemented a real-time version using only one radar sensor. Studies on gesture recognition using IR-UWB radar have rarely been carried out, and some studies are merely simple methods using the magnitude of the reflected signal or those whose performance deteriorates largely due to changes in distance or direction. In this study, we propose a new hand-based gesture recognition algorithm that works robustly against changes in distance or direction while responding only to defined gestures by ignoring meaningless motions. We used three independent features, i.e., variance of the probability density function (pdf) of the magnitude histogram, time of arrival (TOA) variation and the frequency of the reflected signal, to classify the gestures. A data fitting method is included to differentiate between gesture signals and unintended hand or body motions. We have used the clustering technique for the classification of the gestures. Moreover, the distance information is used as an additional input parameter to the clustering algorithm, such that the recognition technique will not be vulnerable to distance change. The hand-based gesture recognition proposed in this paper would be a key technology of future automobile user interfaces. [ABSTRACT FROM AUTHOR]

Gesture controlled mouse is a device in which we can move the mouse by giving the direction from our wrist and turn the direction of mouse through it. After using a accelerometer we can able to move and change the direction of our mouse and by the help of accelerometer we can also change the speed of the mouse. This is a new technology for controlling any device through our wrist movement. After upgrading it we can make many type of other devices which can be very useful for the human being can create a turn in our life. It can be used at many places and also be the part of the future. The gesture will be main part as the movements of our body will play the important role for operations. A stationary point will be taken as the directions will be in x-axis, y-axis and as well as the z-axis. An accelerometer will be used as the direction indicators. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel approach for human action and gesture recognition using dual-complementary tensors. In particular, the proposed method constructs a compact and yet discriminative representation by normalizing the input video volume into dual tensors. One tensor is obtained from the raw video volume data and the other one is obtained from the histogram of oriented gradients (HOG) features. Each tensor is converted to factored matrices and the similarity between factored matrices is evaluated using canonical correlation analysis (CCA). We, furthermore, propose an information fusion method to combine the resulting similarity scores. The proposed fusion strategy can effectively enhance discriminability between different action categories and lead to better recognition accuracy. We have conducted several experiments on two publicly available databases (UCF sports and Cambridge-Gesture). The results show that our proposed method achieves comparable recognition accuracy as the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

We present a method to recognize gestures made by Chinese traffic police based on the static and dynamic descriptor fusion for driver assistance systems and intelligent vehicles. Gesture recognition is made possible by combining the extracted static and dynamic features. First, the point cloud data of human upper body in each frame of input video is obtained to estimate the static descriptor with 2.5D gesture model. Then, the dynamic descriptor is estimated by computing the motion history image of the input RGB video sequence. Finally, the above two descriptors are fused and the mean structural similarity index is used to recognize the gestures made by Chinese traffic police. A comparative study and qualitative evaluation are proposed with other gesture recognition methods, which demonstrate that better recognition results can be obtained using the proposed method on a number of video sequences. [ABSTRACT FROM AUTHOR]

This study is based on speech recognition and interaction, augmented reality, the movement of the head gesture interaction interactive technology, human-computer interaction system is designed to meet the power equipment installation, operation and maintenance operations, to enhance formation of reality for the show, with gestures and head movements as the main way of interaction, with voice commands and touch as a unified interactive model of secondary interaction the way. [ABSTRACT FROM AUTHOR]

The work presented here suggests new ways to tackle exergames for physical rehabilitation and to improve the players' immersion and involvement. The primary (but not exclusive) purpose is to increase the motivation of children and adolescents with severe physical impairments, for doing their required exercises while playing. The proposed gaming environment is based on the Kinect sensor and the Blender Game Engine. A middleware has been implemented that efficiently transmits the data from the sensor to the game. Inside the game, different newly proposed mechanisms have been developed to distinguish pure exercise-gestures from other movements used to control the game (e.g., opening a menu). The main contribution is the amplification of weak movements, which allows the physically impaired to have similar gaming experiences as the average population. To test the feasibility of the proposed methods, four mini-games were implemented and tested by a group of 11 volunteers with different disabilities, most of them bound to a wheelchair. Their performance has also been compared to that of a healthy control group. Results are generally positive and motivating, although there is much to do to improve the functionalities. There is a major demand for applications that help to include disabled people in society and to improve their life conditions. This work will contribute towards providing them with more fun during exercise. [ABSTRACT FROM AUTHOR]

Our society may enhance and create new services in a people-centric IoT context through the exchange of information with sensor devices. Unfortunately, communication and services may be compromised due to a number of factors including unreliable communication, complexity, and security threats like spoofing. Within the technologies involved in the IoT paradigm, passive RFID allows the inventorying of simple objects toward wireless communication with a low-cost investment. We present a solution to increase the personalization of IoT applications and services (e.g., accessing a restricted area with a contact- less card) by detecting people-object gestures with an accelerometer-enabled passive RFID tag. We demonstrate the feasibility of our proposal by achieving a precision of 85 percent in people-object gestures classification. [ABSTRACT FROM AUTHOR]

A complete gesture recognition system should localize and classify each gesture from a given gesture vocabulary, within a continuous video stream. In this work, we compare two approaches: a method that performs the tasks of temporal segmentation and classification simultaneously with another that performs the tasks sequentially. The first method trains a single random forest model to recognize gestures from a given vocabulary, as presented in a training dataset of video plus 3D body joint locations, as well as out-of-vocabulary (non-gesture) instances. The second method employs a cascaded approach, training a binary random forest model to distinguish gestures from background and a multi-class random forest model to classify segmented gestures. Given a test input video stream, both frameworks are applied using sliding windows at multiple temporal scales. We evaluated our formulation in segmenting and recognizing gestures from two different benchmark datasets: the NATOPS dataset of 9600 gesture instances from a vocabulary of 24 aircraft handling signals, and the ChaLearn dataset of 7754 gesture instances from a vocabulary of 20 Italian communication gestures. The performance of our method compares favorably with state-of-the-art methods that employ Hidden Markov Models or Hidden Conditional Random Fields on the NATOPS dataset. We conclude with a discussion of the advantages of using our model for the task of gesture recognition and segmentation, and outline weaknesses which need to be addressed in the future. [ABSTRACT FROM AUTHOR]