Search

Introduction: Clinicians typically observe and describe abnormal head postures (AHPs) and may also measure them. Depth cameras have been suggested as a reliable measurement device for measuring head position using face-tracking technology. This study compared a depth camera (Microsoft Kinect) to a gold standard electromagnetic tracking system (Polhemus device) to measure head position.\ud \ud \ud \ud Method: Twenty healthy volunteers (mean age 21 years) had their head position simultaneously recorded using the depth camera (Kinect) and the electromagnetic tracking system (Polhemus). Participants were asked to make 30-degree head movements into chin up, chin down, head turn and head tilt positions. The head movement made and the stability of the head at each position were recorded and analysed.\ud \ud \ud \ud Results: Compared to the electromagnetic tracking system (Polhemus), the depth camera (Kinect) always measured a smaller head movement. Measurements with the two devices were not statistically significantly different for turn right (P = 0.3955, p > 0.05), turn left (P = 0.4749, p > 0.05), tilt right (P = 0.7086, p > 0.05) and tilt left (P = 0.4091, p > 0.05) head movements. However, the smaller depth camera measurement of chin up and chin down head movements were statistically significant, chin up (P = 0.0001, p < 0.01) and chin down (P = 0.0005, p < 0.001). At each eccentric position, the depth camera (Kinect) recordings were more variable than the electromagnetic tracking system (Polhemus).\ud \ud \ud \ud Conclusions: Compared to the electromagnetic tracking system (Polhemus), the depth camera (Kinect) was comparable for measuring head turns and tilts but was less accurate at measuring chin up and chin down head positions. Further research is needed before the depth cameras are considered for clinical recordings of head position.

A novel, semi-autonomous radiological scanning system for inspecting irregularly shaped and radiologically uncharacterised objects in various orientations is presented. The system utilises relatively low cost, commercial-off-the-shelf (COTS) electronic components, and is intended for use within relatively low to medium radioactive dose environments. To illustrate the generic concepts, the combination of a low-cost COTS vision system, a six DoF manipulator and a gamma radiation spectrometer are investigated. Three modes of vision have been developed, allowing a remote operator to choose the most appropriate algorithm for the task. The robot arm subsequently scans autonomously across the selected object, determines the scan positions and enables the generation of radiological spectra using the gamma spectrometer. These data inform the operator of any likely radioisotopes present, where in the object they are located and thus whether the object should be treated as LLW, ILW or HLW.

Accurate gait recognition is of high significance for numerous industrial and consumer applications, including video surveillance, virtual reality, on-line games, medical rehabilitation, collaborative space exploration, and others. This paper proposes a new architecture designed using deep learning neural network for a highly accurate and robust Kinect-based gait recognition. Two new geometric features: joint relative cosine dissimilarity and joint relative triangle area are introduced. Both of the proposed features are view and pose invariant, thus enhancing recognition performance. The proposed neural network model is trained using the feature vector of dynamic joint relative cosine dissimilarity and joint relative triangle area. Subsequent application of Adam optimization method minimizes the loss of the objective function iteratively. The performance of the proposed deep learning neural network architecture is evaluated on two publicly available 3D skeleton-based gait datasets recorded with the Microsoft Kinect sensor. It is experimentally proven that the accuracy, precision, recall, and F-score of the proposed neural network architecture, trained using introduced dynamic geometric features, is superior to other state-of-the-art methods for Kinect skeleton-based gait recognition.

The rising interest in collaborative robotics leads to research solutions in order to increase robot interaction with the environment. The development of methods that permit robots to recognize and track human motion is relevant for safety and collaboration matters. A large quantity of data can be measured in real time by Microsoft Kinect®, a well-known low-cost depth sensor, able to recognize human presence and to provide postural information by extrapolating a skeleton. However, the Kinect sensor tracks motion with relatively low accuracy and jerky behavior. For this reason, the effective use in industrial applications in which the measurement of arm velocity is required can be unsuitable. The present work proposes a filtering method that allows the measurement of more accurate velocity values of human arm, based on row data provided by the Kinect sensor. The estimation of arm motion is achieved by a Kalman filter based on a kinematic model and by the imposition of fixed lengths for the skeleton links detected by the sensor. The development of the method is supported by experimental tests. The achieved results suggest the practical applicability of the developed algorithms.

In the last years, the advent of low-cost markerless motion capture systems fostered their use in several research fields, such as healthcare and sport. Any system presents benefits and drawbacks that have to be considered to design a Mocap solution providing a proper motion acquisition for a specific context. In order to evaluate low-cost technology, this research work focuses on the evaluation of the accuracy of two categories of devices: the RGB active cameras and the RGB-D, or depth sensors devices. In particular, GoPro Hero 6 active cameras and Microsoft Kinect v2 devices have been selected as representative of the two categories. In particular, this work evaluates and compares the performances of the two systems used to track the position of human articulations. The two devices have been chosen among those available on the market after a state of the art has been completed. Before starting with the campaign of acquisition, the number of sensors and their layout have been designed to optimize the acquisition with both mark-less Mocap systems. Their comparison is based on a list of specific movements of upper and lower limbs. Each movement has been acquired simultaneously, to guarantee the same test conditions. The results have been organized, compared and discussed by evaluating performances and limitations of both solutions related to specific context of use. Conclusions highlight the best candidate technology.

This study targets 2D articulated human pose estimation (i.e. localisation of body limbs) in stereo videos. Although in recent years depth-based devices (e.g. Microsoft Kinect) have gained popularity, as they perform very well in controlled indoor environments (e.g. living rooms, operating theatres or gyms), they suffer clear problems in outdoor scenarios and, therefore, human pose estimation is still an interesting unsolved problem. The authors propose here a novel approach that is able to localise upper-body keypoints (i.e. shoulders, elbows, and wrists) in temporal sequences of stereo image pairs. The authors' method starts by locating and segmenting people in the image pairs by using disparity and appearance information. Then, a set of candidate body poses is computed for each view independently. Finally, temporal and stereo consistency is applied to estimate a final 2D pose. The authors' validate their model on three challenging datasets: `stereo human pose estimation dataset', `poses in the wild' and `INRIA 3DMovie'. The experimental results show that the authors' model not only establishes new state-of-the-art results on stereo sequences, but also brings improvements in monocular sequences.

[EN] RGBD capture devices have been proven as an ICT realistic approach for clinical prevention of falls. RGBD devices facilitate the capture of human movement and are known because of its low cost. According to that, its use is widespread and has been validated in different interactive applications for balance rehabilitation. In this type of rehabilitation, it is very important to have information on clinical patient outcomes. Moreover, it would be helpful to use RGBD devices in case the patient performs the rehabilitation treatment at home because the specialist could use the RGBD devices to assess the balance. This paper demonstrates that the Microsoft Kinect device is reliable and adequate to calculate the standard functional reach test (FRT), one of the most widely used balance clinical measurement. To do so, an experiment was performed on 14 healthy users to compare the FRT calculation manually and using a RGBD device. The results show an average absolute difference of 2.84 cm (± 2.62), and there are no statistically significant differences applying a paired t-student test for the data., [ES] Los dispositivos de captura RGBD han demostrado ser un enfoque tecnológico realista para la prevención terapéutica de caídas. Estos dispositivos RGBD facilitan la captura del movimiento humano y son conocidos gracias a su bajo coste. Por esta razón, su uso se ha generalizado y han sido validados en diferentes aplicaciones interactivas para la rehabilitación motora del equilibrio. En este tipo de rehabilitación es muy importante tener información sobre la evolución terapéutica del paciente. Además, en los casos en que el paciente realiza el tratamiento de rehabilitación en el domicilio y cada cierto tiempo recibe una visita del fisioterapeuta, si éste pudiese utilizar el dispositivo RGBD para valorar el equilibrio con un test estándar le simplificaría mucho el trabajo al no tener que desplazar ningún instrumento de medida. En este trabajo se demuestra que el dispositivo Microsoft Kinect es fiable y adecuado para calcular el test estándar de alcance funcional (FRT), uno de los más utilizados para medir terapéuticamente el equilibrio. Para ello, se ha realizado un experimento donde se ha comparado el cálculo del FRT de forma manual y utilizando un dispositivo RGBD sobre 14 usuarios sanos. Los resultados muestran una diferencia absoluta media de 2.84 cm (±2.62) y la aplicación de un test t-student pareado sobre los datos indica que no hay diferencias estadísticamente significativas., This work was partially funded by European commission under Alyssa Program (ERASMUS-MUNDUS action 2 lot 6), by the Project TIN2012-35427 of the Spanish Government, with FEDER support.

Gait has been recently proposed as a biometric feature that, with respect to other human characteristics, can be captured at a distance without requiring the collaboration of the observed subject. Therefore, it turns out to be a promising approach for people identification in several scenarios, e.g. access control and forensic applications. In this paper, we propose an automatic gait recognition system based on a set of features acquired using the 3D skeletal tracking provided by the popular Kinect sensor. Gait features are defined in terms of distances between selected sets of joints and their vertical and lateral sway with respect to walking direction. Moreover we do not rely on any geometrical assumptions on the position of the sensor. The effectiveness of the defined gait features is shown in the case of person identification based on supervised classification, using the principal component analysis and the support vector machine. A rich set of experiments is provided in two scenarios: a controlled identification setup and a classical video-surveillance setting, respectively. Moreover, we investigate if gait can be considered invariant over time for an individual, at least in a time interval of few years, by comparing gait samples of several subjects three years apart. Our experimental analysis shows that the proposed method is robust to acquisition settings and achieves very competitive identification accuracy with respect to the state of the art.

OpenPTrack is an open source software for multi-camera calibration and people tracking in RGB-D camera networks. It allows to track people in big volumes at sensor frame rate and currently supports a heterogeneous set of 3D sensors. In this work, we describe its user-friendly calibration procedure, which consists of simple steps with real-time feedback that allow to obtain accurate results in estimating the camera poses that are then used for tracking people. On top of a calibration based on moving a checkerboard within the tracking space and on a global optimization of cameras and checkerboards poses, a novel procedure which aligns people detections coming from all sensors in a x - y - t i m e space is used for refining camera poses. While people detection is executed locally, in the machines connected to each sensor, tracking is performed by a single node which takes into account detections from all over the network. Here we detail how a cascade of algorithms working on depth point clouds and color, infrared and disparity images is used to perform people detection from different types of sensors and in any indoor light condition. We present experiments showing that a considerable improvement can be obtained with the proposed calibration refinement procedure that exploits people detections and we compare Kinect v1, Kinect v2 and Mesa SR4500 performance for people tracking applications. OpenPTrack is based on the Robot Operating System and the Point Cloud Library and has already been adopted in networks composed of up to ten imagers for interactive arts, education, culture and human–robot interaction applications.

Biometrics is currently an area that is both very interesting as well as rapidly growing. Among various types of biometrics the human gait recognition seems to be one of the most intriguing. However, one of the greatest problems within this field of biometrics is the change in gait caused by footwear. A change of shoes results in a significant lowering of accuracy in recognition of people. The following work presents a method which uses data gathered by two sensors: force plates and Microsoft Kinect v2 to reduce this problem. Microsoft Kinect is utilized to measure the body height of a person which allows the reduction of the set of recognized people only to those whose height is similar to that which has been measured. The entire process is preceded by identifying the type of footwear which the person is wearing. The research was conducted on data obtained from 99 people (more than 3400 strides) and the proposed method allowed us to reach a Correct Classification Rate (CCR) greater than 88% which, in comparison to earlier methods reaching CCR’s of

One of the key technologies that lays behind the human–machine interaction and human motion diagnosis is the limbs motion tracking. To make the limbs tracking efficient, it must be able to estimate a precise and unambiguous position of each tracked human joint and resulting body part pose. In recent years, body pose estimation became very popular and broadly available for home users because of easy access to cheap tracking devices. Their robustness can be improved by different tracking modes data fusion. The paper defines the novel approach—orientation based data fusion—instead of dominating in literature position based approach, for two classes of tracking devices: depth sensors (i.e., Microsoft Kinect) and inertial measurement units (IMU). The detailed analysis of their working characteristics allowed to elaborate a new method that let fuse more precisely limbs orientation data from both devices and compensates their imprecisions. The paper presents the series of performed experiments that verified the method’s accuracy. This novel approach allowed to outperform the precision of position-based joints tracking, the methods dominating in the literature, of up to 18%.

Image labeling tools help to extract objects within images to be used as ground truth for learning and testing in object detection processes. The inputs for such tools are usually RGB images. However with new widely available low-cost sensors like Microsoft Kinect it is possible to use depth images in addition to RGB images. Despite many existing powerful tools for image labeling, there is a need for RGB-depth adapted tools. We present a new interactive labeling tool that partially automates image labeling, with two major contributions. First, the method extends the concept of image segmentation from RGB to RGB-depth using Fuzzy C-Means clustering, connected component labeling and superpixels, and generates bounding pixels to extract the desired objects. Second, it minimizes the interaction time needed for object extraction by doing an efficient segmentation in RGB-depth space. Very few clicks are needed for the entire procedure compared to existing, tools. When the desired object is the closest object to the camera, which is often the case in robotics applications, no clicks at all are required to accurately extract the object.

The paper presents a calibration model suitable for software-based calibration of Kinect-type RGB-D sensors. Additionally, it describes a two-step calibration procedure assuming a use of only a simple checkerboard pattern. Finally, the paper presents a calibration case study, showing that the calibration may improve sensor accuracy 3 to 5 times, depending on the anticipated use of the sensor. The results obtained in this study using calibration models of different levels of complexity reveal that depth measurement correction is an important component of calibration as it may reduce by 50% the errors in sensor reading.

In this paper, the problem of 3D body registration using a single RGB-D sensor is approached. It has been guided by three main requirements: low-cost, unconstrained movement and accuracy. In order to fit them, an iterative registration method for accurately aligning data from single RGB-D sensor is proposed. The data is acquired while a person rotates in front of the camera, without the need of any external marker or constraint about its pose. The articulated alignment is carried out in a model-free approach in order to be more consistent with the real data. The iterative method is divided in stages, contributing to each other by the refinement of a specific part of the acquired data. The exploratory results validate the proposed method that is able to feed on itself in each iteration improving the final result by a progressive iteration, with the required precision under the conditions of affordability and unconstrained movement acquisition. This work has been supported by a grant from the University of Alicante project GRE16-28.

Sensörler ile donatılmış derinlik kamera cihazlarının maliyetlerinin ekonomik olması nedeniyle, günümüzde kullanım alanları artmakta ve yaygınlaşmaktadır. Bu çalışmada bu tür cihazların en çok kullanılanlarından biri olan Kinect cihazından elde edilen veriler üzerinde, Ağırlıklı Dinamik Zaman Bükmesi ve Sembolik Birleştirme Yaklaşımı yöntemleri birlikte kullanılarak yeni bir hareket tanıma yöntemi geliştirilmiştir. Geliştirilen yöntem günlük hareketlerin yer aldığı veri setinde test edilmiş ve %98.15 oranında bir başarı ile günlük hareketler tanınabilmiştir. Nowadays, the usage areas of depth cameras which equipped with sensors are increasing and growing up extensively, because of their economic prices. In this study, a new gesture recognition method is developed by combining Dynamic Time Warping and Symbolic Aggregation Approximation methods on data obtained from a Kinect device which is one of the most widely used among such devices. The developed method has been tested in the data set where the daily movements recorded in and they can be recognized with a success rate of 98.15%.

Exergames provide a challenging opportunity for home-based training and evaluation of postural control in the elderly population, but affordable sensor technology and algorithms for assessment of whole body movement patterns in the home environment are yet to be developed. The aim of the present study was to evaluate the use of Kinect, a commonly available video game sensor, for capturing and analyzing whole body movement patterns. Healthy adults (n=20) played a weight shifting exergame under five different conditions with varying amplitudes and speed of sway movement, while 3D positions of ten body segments were recorded in the frontal plane using Kinect and a Vicon 3D camera system. Principal Component Analysis (PCA) was used to extract and compare movement patterns and the variance in individual body segment positions explained by these patterns. Using the identified patterns, balance outcome measures based on spatiotemporal sway characteristics were computed. The results showed that both Vicon and Kinect capture >90% variance of all body segment movements within three PCs. Kinect-derived movement patterns were found to explain variance in trunk movements accurately, yet explained variance in hand and foot segments was underestimated and overestimated respectively by as much as 30%. Differences between both systems with respect to balance outcome measures range 0.3-64.3%. The results imply that Kinect provides the unique possibility of quantifying balance ability while performing complex tasks in an exergame environment.

The 3D acquisition of object structures has become a common technique in many fields of work, e.g., industrial quality management, cultural heritage or crime scene documentation. The requirements on the measuring devices are versatile, because spacious scenes have to be imaged with a high level of detail for selected objects. Thus, the used measuring systems are expensive and require an experienced operator. With the rise of low-cost 3D imaging systems, their integration into the digital documentation process is possible. However, common low-cost sensors have the limitation of a trade-off between range and accuracy, providing either a low resolution of single objects or a limited imaging field. Therefore, the use of multiple sensors is desirable. We show the combined use of two low-cost sensors, the Microsoft Kinect and the David laserscanning system, to achieve low-resolved scans of the whole scene and a high level of detail for selected objects, respectively. Afterwards, the high-resolved David objects are automatically assigned to their corresponding Kinect object by the use of surface feature histograms and SVM-classification. The corresponding objects are fitted using an ICP-implementation to produce a multi-resolution map. The applicability is shown for a fictional crime scene and the reconstruction of a ballistic trajectory.

Detailed 3D plant architectural data have numerous applications in plant science, but many existing approaches for 3D data collection are time-consuming and/or require costly equipment. Recently, there has been rapid growth in the availability of low-cost, 3D cameras and related open source software applications. 3D cameras may provide measurements of key components of plant architecture such as stem diameters and lengths, however, few tests of 3D cameras for the measurement of plant architecture have been conducted. Here, we measured Salix branch segments ranging from 2–13 mm in diameter with an Asus Xtion camera to quantify the limits and accuracy of branch diameter measurement with a 3D camera. By scanning at a variety of distances we also quantified the effect of scanning distance. In addition, we also test the sensitivity of the program KinFu for continuous 3D object scanning and modeling as well as other similar software to accurately record stem diameters and capture plant form (6 mm, Asus Xtion may provide a novel method for the collection of 3D data on the branching architecture of woody plants. Improvements in camera measurement accuracy and available software are likely to further improve the utility of 3D cameras for plant sciences in the future.

Non-contact human body measurement plays an important role in surveillance, physical healthcare, on-line business and virtual fitting. Current methods for measuring the human body without physical contact usually cannot handle humans wearing clothes, which limits their applicability in public environments. In this paper, we propose an effective solution that can measure accurate parameters of the human body with large-scale motion from a Kinect sensor, assuming that the people are wearing clothes. Because motion can drive clothes attached to the human body loosely or tightly, we adopt a space-time analysis to mine the information across the posture variations. Using this information, we recover the human body, regardless of the effect of clothes, and measure the human body parameters accurately. Experimental results show that our system can perform more accurate parameter estimation on the human body than state-of-the-art methods.

V magistrskem delu raziskujemo problem zaznavanja gest s pomočjo senzorja Kinect. Z razvojem naše aplikacije želimo odstraniti potrebo po standardnih vhodnih napravah. Razvili smo aplikacijo za prepoznavanje gest, ki zazna tako statične kot dinamične geste. Naša aplikacija temelji na obdelavi slikovnih podatkov, ki jih zajemamo s senzorjem Kinect. Iz zajetih podatkov z naprednimi postopki računalniškega vida izluščimo ključne značilnice, ki definirajo uporabnikovo gibanje. Časovno zaporedje teh značilnic uporabimo v naprednem algoritmu "Dynamic Time Warping" za klasifikacijo gest. Končno aplikacijo je testiralo 7 uporabnikov v 3 različnih eksperimentih. Na podlagi tega smo ocenili uspešnost naše rešitve. Aplikacija je kljub določenim pomanjkljivostim dosegla povprečno uspešnost okrog 88 odstotkov. Na podlagi rezultatov smo ugotovili, da aplikacija ustreza zastavljenim ciljem in potrjuje večino postavljenih hipotez. Izkazalo se je, da bi lahko naš algoritem uspešno uporabili kot napreden uporabniški vmesnik v komunikaciji človek – stroj praktično v kateri koli aplikaciji. This master's thesis deals with problems associated with gesture recognition by using the Kinect Sensor. Our goal is to develop an application that would remove the need for standard input devices. We have developed an application for gesture recognition, which is able to recognize static as well as dynamic gestures by processing imaging data gathered by the Kinect Sensor. High-level computer vision processes help us eliminate the key features, which define the user's movements. The time sequence of these elements is then used in the advanced algorithm "Dynamic Time Warping" for gesture classification. The application was tested by seven users in three different experiments, which helped us determine how successful our solution was. The application was 88% successful, on average, despite certain flaws. Based on the results we determined that the application meets the set goals and supports most of our hypotheses. We discovered that our algorithm could be used successfully as an advanced user interface in human–machine interaction in virtually any application.

Manual handling involves transporting of load by hand through lifting or lowering and operators on the manufacturing shop floor are daily faced with constant lifting and lowering operations which leads to Work-Related Musculoskeletal Disorders. The trend in data collection on the Shop floor for ergonomic evaluation during manual handling activities has revealed a gap in gesture detection as gesture triggered data collection could facilitate more accurate ergonomic data capture and analysis. This paper presents an application developed to detect gestures towards triggering real-time human motion data capture on the shop floor for ergonomic evaluations and risk assessment using the Microsoft Kinect. The machine learning technology known as the discrete indicator—precisely the AdaBoost Trigger indicator was employed to train the gestures. Our results show that the Kinect can be trained to detect gestures towards real-time ergonomic analysis and possibly offering intelligent automation assistance during human posture detrimental tasks.

Measurement of energy expenditure is an important tool in sport science and medicine, especially when trying to estimate the extent and intensity of physical activity. However, most approaches still rely on sensors or markers, placed directly on the body. In this paper, we present a novel approach using a fully contact-less, fully automatic method, that relies on computer vision algorithms and widely available and inexpensive imaging sensors. We rely on the estimation of the optical and scene flow to calculate Histograms of Oriented Optical Flow (HOOF) descriptors, which we subsequently augment with the Histograms of Absolute Flow Amplitude (HAFA). Descriptors are fed into regression model, which allows us to estimate energy consumption, and to a lesser extent, the heart rate. Our method has been tested both in lab environment and in realistic conditions of a sport match. Results confirm that these energy expenditures could be derived from purely contact-less observations. The proposed method can be used with different modalities, including near infrared imagery, which extends its future potential.

Diplomski rad opisuje izradu sustava za praćenje govornika. Sustav se sastoji od računala, pomične kamere visoke rezolucije i Microsoft Kinect senzora. Sustav koristi računalni vid kako bi locirao korisnike i analizirao njihove kretnje. Kinect senzor pronalazi govornika, i zatim prosljeđuje te informacije računalu. Računalna aplikacija pomoću spomenutih podataka računa kut za koji se pomična kamera mora pomaknuti kako bi se govornik nalazio u kadru. Sustav prati govornikovo gibanje u sve tri dimenzije, te koristi optičko povećanje u slučaju da se korisnik udalji. Sustav isto tako može tumačiti govornikove pokrete ruku, koje mu omogućuju upravljanje sustavom putem gesti. Uz diplomski rad, priložena je aplikacija koja sadrži opisane funkcionalnosti. This master's thesis describes how to build a speaker tracking system. The system consists of a computer, a high resolution pan-tilt-zoom camera and a Microsoft Kinect sensor. The system is using computer vision to locate users and analyze their movements. The Kinect sensor acquires the speaker and forwards the information to the computer. Using this data, the computer application calculates the angle required by the pan-tilt-zoom camera. PTZ camera then moves, keeping the speaker inside the video frame. The system tracks the speaker’s movement in all three dimensions and increases magnification in case the user moves backwards. The system can also interpret the speaker’s hand movements, which allows the speaker to utilize the system using gestures. An application containing the described functionalities is included in this master’s thesis.

This paper proposes a novel 3D direct interaction modelling system for manipulating voxel based objects using currently available 3D motion sensing input devices such as Microsoft Kinect. The guiding principle while developing the system was to imitate natural human modelling behaviour and provide a real life experience of 3D object manipulation, inspired by the techniques used in modelling clay. Properties of a functional prototype application are presented and software architecture of the created tool is analysed. Descriptions of newly developed algorithms are included, grouped into several categories. Visualization algorithms are used for defining properties and creating usable modelling mass from volumetric models. On the other hand, algorithms related to object recognition and human computer interaction include various techniques for depth segmentation, contour detection, finger recognition and virtual control with gestures.

Treball de fi de grau en Sistemes Audiovisuals Tutor: Pol Cirujeda En els últims anys el procés de captura de moviment ha estat basat en l'adquisició de dades amb sensors físics per animar models del món real. Degut a la complexitat logística que requereix aquesta tècnica, l’aparició d’una alternativa basada en càmeres de profunditat com Microsoft Kinect ha obert tot un ventall de possibilitats en l’industria no tan sols de l’animació sinó també mèdica i de l’esport. En aquesta línia de treball, diferents mètodes són necessaris per animar un model humà; l’obtenció de dades fent ús del sensor de profunditat i el seu postprocessat i filtratge per aconseguir un model natural i realista del cós humà; calcular i afitar una estructura esqueletal d’una manera eficaç d'acord amb el volum de dades depurades obtingut; i finalment la dificultat d’aconseguir animar l’avatar d’acord al moviment humà amb diferents mètodes de skinning. El propòsit d’aquest projecte per tant, consisteix en l’estudi geomètric, l’animació i la deformació de malles en 2D i 3D fent servir Microsoft Kinect. In the last years the motion capture process was based on data acquisition with physical sensors in order to animate models of the real world. Due to the logistical complexity this technique requires, the appearance of an alternative based on depth cameras such as Microsoft Kinect has opened a range of possibilities, not only in the entertainment industry, but also on medical and the sports areas. In this line of work, different methods are needed in order to animate a human model; obtaining data using the depth sensor and its post-processing and filtering in order to achieve a natural and realistic model of the human body; calculate and embed a skeletal structure to the volume data obtained in an efficient way; and finally the difficulty of animating an avatar according to human motion with different methods of blending. The purpose of this project therefore is to study the geometry, animation and deformation of meshes in 2D and 3D using Microsoft Kinect.

Accuracy in localizing the brain areas that generate neuromagnetic activity in magnetoencephalography (MEG) is dependent on properly co-registering MEG data to the participant’s structural magnetic resonance image (MRI). Effective MEG-MRI co-registration is, in turn, dependent on how accurately we can digitize anatomical landmarks on the surface of the head. In this study, we compared the performance of three devices —Polhemus electromagnetic system, NextEngine laser scanner and Microsoft Kinect for Windows — for source localization accuracy and MEG-MRI co-registration. A calibrated phantom was used for verifying the source localization accuracy. The Kinect improved source localization accuracy over the Polhemus and the laser scanner by 2.23 mm (137%) and 0.81 mm (50%), respectively. MEG-MRI co-registration accuracy was verified on data from five healthy human participants, who received the digitization process using all three devices. The Kinect device captured approximately 2000 times more surface points than the Polhemus in one third of the time (one minute compared to three minutes) and thrice as many points as the NextEngine laser scanner. Following automated surface matching, the calculated mean MEG-MRI co-registration error for the Kinect was improved by 2.85 mm with respect to the Polhemus device, and equivalent to the laser scanner. Importantly, the Kinect device automatically aligns 20-30 images per second in real-time, reducing the limitations on participant head movement during digitization that are implicit in NextEngine laser scan (~1 minute). We conclude that the Kinect scanner is an effective device for head digitization in MEG, providing the necessary accuracy in source localization and MEG-MRI co-registration, while reducing digitization time.

Service robots have to robustly follow and interact with humans. In this paper, we propose a very fast multi-people tracking algorithm designed to be applied on mobile service robots. Our approach exploits RGB-D data and can run in real-time at very high frame rate on a standard laptop without the need for a GPU implementation. It also features a novel depth-based sub-clustering method which allows to detect people within groups or even standing near walls. Moreover, for limiting drifts and track ID switches, an online learning appearance classifier is proposed featuring a three-term joint likelihood. We compared the performances of our system with a number of state-of-the-art tracking algorithms on two public datasets acquired with three static Kinects and a moving stereo pair, respectively. In order to validate the 3D accuracy of our system, we created a new dataset in which RGB-D data are acquired by a moving robot. We made publicly available this dataset which is not only annotated by hand, but the ground-truth position of people and robot are acquired with a motion capture system in order to evaluate tracking accuracy and precision in 3D coordinates. Results of experiments on these datasets are presented, showing that, even without the need for a GPU, our approach achieves state-of-the-art accuracy and superior speed.

Treball acadèmic realitzat a la Facultat de Matemàtiques (UB)

This paper presents a novel method to reproduce in real time the opening and closing gestures of a human hand, animating a three-dimensional model of it. In other works, this result can be achieved by mapping a set of significant points of a real hand on the corresponding points of the model to animate. We propose an alternative way to produce the same effect without mapping points, but using a level-based estimation of the degree of opening of the hand. The experiments have been executed using Microsoft KinectTM, but the method would work on any other Kinect-like devices (as defined herein). The results obtained are particularly encouraging and demonstrate real-time performance of the system.

The Microsoft Kinect sensor is largely used to detect and recognize body gestures and layout with enough reliability, accuracy and precision in a quite simple way. However, the pretty low resolution of the optical sensors does not allow the device to detect gestures of body parts, such as the fingers of a hand, with the same straightforwardness. Given the clear application of this technology to the field of the user interaction within immersive multimedia environments, there is the actual need to have a reliable and effective method to detect the pose of some body parts. In this paper we propose a method based on a neural network to detect in real time the hand pose, to recognize whether it is closed or not. The neural network is used to process information of color, depth and skeleton coming from the Kinect device. This information is preprocessed to extract some significant feature. The output of the neural network is then filtered with a time average, to reduce the noise due to the fluctuation of the input data. We analyze and discuss three possible implementations of the proposed method, obtaining an accuracy of 90% under good conditions of lighting and background, and even reaching the 95% in best cases, in real time.

The release of the Microsoft Kinect has attracted the attention of researchers in a variety of computer science domains. Even though this device is still relatively new, its recent applications have shown some promising results in terms of replacing current conventional methods like the stereo-camera for robotics navigation, multi-camera system for motion detection and laser scanner for 3D reconstruction. While most work around the Kinect is on how to take full advantage of its capabilities, so far only a few studies have been carried out on the limitations of this device and fewer that provide solutions to enhance the precision of its measurements. In this paper, we review and analyse current work in this area, and present and evaluate a temporal denoising algorithm to reduce the instability of the depth measurements provided by the Kinect over different distances.

In this paper, an authenticate live 3D point cloud video streaming system is presented, using a low cost 3D sensor camera, the Microsoft Kinect. The proposed system is implemented on a client-server network infrastructure. The live 3D video is captured from the Kinect RGB-D sensor, then a 3D point cloud is generated and processed. Filtering and compression are used to handle the spatial and temporal redundancies. A color histogram based conditional filter is designed to reduce the color information for each frame based on the mean and standard deviation. In addition to the designed filter, a statistical outlier removal filter is used. A certificate-based authentication is used where the client will verify the identity of the server during the handshake process. The processed 3D point cloud video is live streamed over a TCP/IP protocol to the client. The system is evaluated in terms of: compression ratio, total bytes per points, peak signal to noise ratio (PSNR), and Structural Similarity (SSIM) index. The experimental results demonstrate that the proposed video streaming system have a best case with SSIM 0.859, PSNR of 26.6 dB and with average compression ratio of 8.42 while the best average compression ratio case is about 15.43 with PSNR 18.5128 dB of and SSIM 0.7936.

This paper presents Kinect Monte Carlo Localization (KMCL), a new method for localization in three dimensional indoor environments using RGB-D cameras, such as the Microsoft Kinect. The approach makes use of a low fidelity a priori 3-D model of the area of operation composed of large planar segments, such as walls and ceilings, which are assumed to remain static. Using this map as input, the KMCL algorithm employs feature-based visual odometry as the particle propagation mechanism and utilizes the 3-D map and the underlying sensor image formation model to efficiently simulate RGB-D camera views at the location of particle poses, using a graphical processing unit (GPU). The generated 3D views of the scene are then used to evaluate the likelihood of the particle poses. This GPU implementation provides a factor of ten speedup over a pure distance-based method, yet provides comparable accuracy. Experimental results are presented for five different configurations, including: (1) a robotic wheelchair, (2) a sensor mounted on a person, (3) an Ascending Technologies quadrotor, (4) a Willow Garage PR2, and (5) an RWI B21 wheeled mobile robot platform. The results demonstrate that the system can perform robust localization with 3D information for motions as fast as 1.5 meters per second. The approach is designed to be applicable not just for robotics but other applications such as wearable computing.

This thesis will introduce the concept of collaboratively using freehand gestures to interact with visualizations. It could be problematic to work with data and visualizations together with others in the traditional desktop setting because of the limited screen size and a single user input device. Therefore this thesis suggests a solution by integrating computer vision and gestures with interactive visualizations. This integration resulted in a prototype where multiple users can interact with the same visualizations simultaneously. The prototype was evaluated and tested on ten potential users. The results from the tests show that using gestures have potential to support collaboration while working with interactive visualizations. It also shows what components are needed in order to enable gestural interaction with visualizations.