Search

Everyone aspires to live a healthy life, but many will inevitably experience some form of disease, illness, or accident that results in disability at some point. Rehabilitation plays a crucial role in helping individuals recover from these disabilities and return to their daily activities. Traditional rehabilitation methods are often expensive, are inefficient, and lead to slow progress for patients. However, in this era of technology, various sensor-based automatic rehabilitation is also possible. A Kinect sensor is a skeletal tracking device that captures human motions and gestures. It can provide feedback to the users, allowing them to better understand their progress and adjust their movements accordingly. In this study, stroke-based rehabilitation is presented along with the Toronto Rehab Stroke Pose Dataset (TRSP). Pre-processing of the raw dataset was performed using various features, and several state-of-the-art classifiers were applied to evaluate the data provided by the Kinect sensor. Among the various classifiers, eXtreme Gradient Boosing (XGB) attained the maximum accuracy of 92% for the TRSP dataset. Furthermore, hyperparameters of the XGB have been optimized using a metaheuristic gray wolf optimizer for better performance. [ABSTRACT FROM AUTHOR]

Body composition assessment using instruments such as dual X-ray densitometry (DXA) can be complex and their use is often limited to research. This cross-sectional study aimed to develop and validate a densitometric method for fat mass (FM) estimation using 3D cameras. Using two such cameras, stereographic images, and a mesh reconstruction algorithm, 3D models were obtained. The FM estimations were compared using DXA as a reference. In total, 28 adults, with a mean BMI of 24.5 (±3.7) kg/m2 and mean FM (by DXA) of 19.6 (±5.8) kg, were enrolled. The intraclass correlation coefficient (ICC) for body volume (BV) was 0.98–0.99 (95% CI, 0.97–0.99) for intra-observer and 0.98 (95% CI, 0.96–0.99) for inter-observer reliability. The coefficient of variation for kinetic BV was 0.20 and the mean difference (bias) for BV (liter) between Bod Pod and Kinect was 0.16 (95% CI, −1.2 to 1.6), while the limits of agreement (LoA) were 7.1 to −7.5 L. The mean bias for FM (kg) between DXA and Kinect was −0.29 (95% CI, −2.7 to 2.1), and the LoA was 12.1 to −12.7 kg. The adjusted R2 obtained using an FM regression model was 0.86. The measurements of this 3D camera-based system aligned with the reference measurements, showing the system's feasibility as a simpler, more economical screening tool than current systems. [ABSTRACT FROM AUTHOR]

Attention-Deficit/Hyperactivity Disorder (ADHD) is a neurodevelopmental disorder known for its significant heterogeneity and varied symptom presentation. Describing the different subtypes as predominantly inattentive (ADHD–I), combined (ADHD–C), and hyperactive–impulsive (ADHD–H) relies primarily on clinical observations, which can be subjective. To address the need for more objective diagnostic methods, this pilot study implemented a Microsoft Kinect-based Stroop Color–Word Test (KSWCT) with the objective of investigating the potential differences in executive function and motor control between different subtypes in a group of children and adolescents with ADHD. A series of linear mixture modeling were used to encompass the performance accuracy, reaction times, and extraneous movements during the tests. Our findings suggested that age plays a critical role, and older subjects showed improvements in KSWCT performance; however, no significant divergence in activity level between the subtypes (ADHD–I and ADHD–H/C) was established. Patients with ADHD–H/C showed tendencies toward deficits in motor planning and executive control, exhibited by shorter reaction times for incorrect responses and more difficulty suppressing erroneous responses. This study provides preliminary evidence of unique executive characteristics among ADHD subtypes, advances our understanding of the heterogeneity of the disorder, and lays the foundation for the development of refined and objective diagnostic tools for ADHD. [ABSTRACT FROM AUTHOR]

Innovative technologies can improve user usability and satisfaction in computer-based activities in the work environment, including surgeons working in the operating room (OR). A scrubbed surgeon must ask unscrubbed staff to retrieve medical images from a surgical PACS system on a monitor in a hybrid operating room. The study aimed to check users' usability and satisfaction with the designed and developed Ortho_Kinect_OR application, which enables contactless control of access to medical images during surgery. The application also facilitates access to telemedicine applications such as intraoperative telementoring during orthopedic procedures or tele-education. The application's usability was tested by assigning standardized tasks to surgeons for PACS and teleconferencing in the operating room. Hand gestures were recognized and translated to function like mouse buttons. Field tests conducted among orthopedic surgeons showed high usability and user satisfaction. PACS access in the operating room did not distract the orthopedic surgeon during orthopedic surgery procedures. All surgeons completed the tests and tasks without any problems. OR field test results showed high agreement among users and were very satisfactory. Junior surgeons and residents pointed out that people with average computer knowledge could use the application. It has been shown that the contactless system designed and built based on the Kinect sensor available on the shelves meets the requirements of the operating room environment and is easy to use. Touchless controller technology provides the opportunity to increase the use of intraoperative imaging previews and improve the safety of surgical patients by improving sterility and reducing unnecessary staff in the operating room. Using the Ortho_Kinect_OR application and the Kinect sensor, it is possible to provide contactless access to videoconference telementoring and PACS in the operating room without an orthopedic surgeon's unnecessary distraction in the operating room environment. [ABSTRACT FROM AUTHOR]

The severity of falls increases with age and reduced mobility. Falls are a frequent source of domestic accidents and accidental death on the part of fragile people. They produce anatomical injuries, reduce quality of life, cause dramatic psychological effects, and impose heavy financial burdens. A growing elderly population leads to a direct increase in health service costs, and indirectly to a deterioration of social life in the long term. Unsurprisingly, socioeconomic costs have triggered new scientific health research to detect falls in older people. One of the most appropriate solutions for monitoring the elderly and automatically detecting falls is computer vision. The Kinect camera plays a vital role in recognizing and detecting activities while ensuring seniors' comfort, safety, and privacy preferences in the fall detection system. This research surveys several Kinect-based works in the literature that cover the approaches used in fall detection. In addition, we discuss the public fall benchmark based on Kinect technology. In general, the main objective of this survey is to provide a complete description of the modules making up the fall detectors and thereby guide researchers in developing fall approaches based on Kinect. [ABSTRACT FROM AUTHOR]

Aiming at the problem of feature point tracking loss caused by large head rotation and facial occlusion in doctors, this paper designs a head-position-tracking system based on geometric triangles and unscented Kalman filtering. By interconnecting the three feature points of the left and right pupil centers and the tip of the nose, they form a coplanar triangle. When the posture of the doctor's head changes due to rotation, the shape of the corresponding geometric triangle will also deform. Using the inherent laws therein, the head posture can be estimated based on changes in the geometric model. Due to the inaccurate positioning of feature points caused by the deflection of the human head wearing a mask, traditional linear Kalman filtering algorithms are difficult to accurately track feature points. This paper combines geometric triangles with an unscented Kalman Filter (UKF) to obtain head posture, which has been fully tested in different environments, such as different faces, wearing/not wearing masks, and dark/bright light via public and measured datasets. The final experimental results show that compared to the linear Kalman filtering algorithm with a single feature point, the traceless Kalman filtering algorithm combined with geometric triangles in this paper not only improves the robustness of nonlinear angle of view tracking but also can provide more accurate estimates than traditional Kalman filters. [ABSTRACT FROM AUTHOR]

Mild Cognitive Impairment (MCI) in pensioners has become an important concern in the aging population, and there are an increasing number of products, especially virtual reality (VR)-based products, to assist in the identification, intervention, and treatment of MCI older adults. Multiple studies on the efficacy and usability of VR products are also receiving increased attention from designers. However, issues such as the efficacy testing of VR products still face serious challenges. In this study, we evaluated VR products looking at the interactive responsiveness of MCI older adults when using VR products, and analyzed the brain activation status and behavioral conditions of MCI older adults. Multisource data were generated by a functional near-infrared spectroscopy (fNIRS) device with high spatial resolution and a behavioral recording device reflecting motor abilities. Small-wave amplitudes were selected as indicators of brain activation to analyze six brain areas: LPFC, RPFC, LOL, ROL, LMC, and RMC. Eight aspects, such as overall upper-limb speed, upper-limb global acceleration, and median velocity, were selected as indicators for behavioral recording. The differences were observed by comparing the level of completion of interactive responses by MCI older adults between the two groups. The results showed that MCI older adults showed different levels of activation in brain regions when performing VR product-based tasks. The higher the level of cognition, the better the interactive response in the task and the stronger the activation of brain regions. Meanwhile, the level of interaction response had a significant correlation with the motor performance of MCI older adults, with stronger motor functions leading to a more effective interaction response to the product. This study proposes a new method to evaluate the feasibility of monitoring the interaction between the MCI elderly and VR-based products using fNIRS with Kinect, which provides a new way to evaluate the effectiveness of VR-based product-assisted treatment. [ABSTRACT FROM AUTHOR]

This research shows the development of a teleoperation system with an assistive robot (NAO) through a Kinect V2 sensor, a set of Meta Quest virtual reality glasses, and Nintendo Switch controllers (Joycons), with the use of the Robot Operating System (ROS) framework to implement the communication between devices. In this paper, two interchangeable operating models are proposed. An exclusive controller is used to control the robot's movement to perform assignments that require long-distance travel. Another teleoperation protocol uses the skeleton joints information readings by the Kinect sensor, the orientation of the Meta Quest, and the button press and thumbstick movements of the Joycons to control the arm joints and head of the assistive robot, and its movement in a limited area. They give image feedback to the operator in the VR glasses in a first-person perspective and retrieve the user's voice to be spoken by the assistive robot. Results are promising and can be used for educational and therapeutic purposes. [ABSTRACT FROM AUTHOR]

Automatic plant phenotype measurement technology based on the rapid and accurate reconstruction of maize structures at the seedling stage is essential for the early variety selection, cultivation, and scientific management of maize. Manual measurement is time-consuming, laborious, and error-prone. The lack of mobility of large equipment in the field make the high-throughput detection of maize plant phenotypes challenging. Therefore, a global 3D reconstruction algorithm was proposed for the high-throughput detection of maize phenotypic traits. First, a self-propelled mobile platform was used to automatically collect three-dimensional point clouds of maize seedling populations from multiple measurement points and perspectives. Second, the Harris corner detection algorithm and singular value decomposition (SVD) were used for the pre-calibration single measurement point multi-view alignment matrix. Finally, the multi-view registration algorithm and iterative nearest point algorithm (ICP) were used for the global 3D reconstruction of the maize seedling population. The results showed that the R2 of the plant height and maximum width measured by the global 3D reconstruction of the seedling maize population were 0.98 and 0.99 with RMSE of 1.39 cm and 1.45 cm and mean absolute percentage errors (MAPEs) of 1.92% and 2.29%, respectively. For the standard sphere, the percentage of the Hausdorff distance set of reconstruction point clouds less than 0.5 cm was 55.26%, and the percentage was 76.88% for those less than 0.8 cm. The method proposed in this study provides a reference for the global reconstruction and phenotypic measurement of crop populations at the seedling stage, which aids in the early management of maize with precision and intelligence. [ABSTRACT FROM AUTHOR]

Depth-based plethysmography (DPG) for the measurement of respiratory parameters is a mobile and cost-effective alternative to spirometry and body plethysmography. In addition, natural breathing can be measured without a mouthpiece, and breathing mechanics can be visualized. This paper aims at showing further improvements for DPG by analyzing recent developments regarding the individual components of a DPG measurement. Starting from the advantages and application scenarios, measurement scenarios and recording devices, selection algorithms and location of a region of interest (ROI) on the upper body, signal processing steps, models for error minimization with a reference measurement device, and final evaluation procedures are presented and discussed. It is shown that ROI selection has an impact on signal quality. Adaptive methods and dynamic referencing of body points to select the ROI can allow more accurate placement and thus lead to better signal quality. Multiple different ROIs can be used to assess breathing mechanics and distinguish patient groups. Signal acquisition can be performed quickly using arithmetic calculations and is not inferior to complex 3D reconstruction algorithms. It is shown that linear models provide a good approximation of the signal. However, further dependencies, such as personal characteristics, may lead to non-linear models in the future. Finally, it is pointed out to focus developments with respect to single-camera systems and to focus on independence from an individual calibration in the evaluation. [ABSTRACT FROM AUTHOR]

Hyperspectral imaging and distance data have previously been used in aerial, forestry, agricultural, and medical imaging applications. Extracting meaningful information from a combination of different imaging modalities is difficult, as the image sensor fusion requires knowing the optical properties of the sensors, selecting the right optics and finding the sensors' mutual reference frame through calibration. In this research we demonstrate a method for fusing data from Fabry–Perot interferometer hyperspectral camera and a Kinect V2 time-of-flight depth sensing camera. We created an experimental application to demonstrate utilizing the depth augmented hyperspectral data to measure emission angle dependent reflectance from a multi-view inferred point cloud. We determined the intrinsic and extrinsic camera parameters through calibration, used global and local registration algorithms to combine point clouds from different viewpoints, created a dense point cloud and determined the angle dependent reflectances from it. The method could successfully combine the 3D point cloud data and hyperspectral data from different viewpoints of a reference colorchecker board. The point cloud registrations gained 0.29 – 0.36 fitness for inlier point correspondences and RMSE was approx. 2, which refers a quite reliable registration result. The RMSE of the measured reflectances between the front view and side views of the targets varied between 0.01 and 0.05 on average and the spectral angle between 1.5 and 3.2 degrees. The results suggest that changing emission angle has very small effect on the surface reflectance intensity and spectrum shapes, which was expected with the used colorchecker. [ABSTRACT FROM AUTHOR]

Hereditary Amyloidosis associated with variant Transthyretin (ATTRv Amyloidosis) is a progressive and highly disabling neurological disorder that affects gait. Quantitative motion analysis is useful for assessing motor function, including gait, in diseases affecting movement. A single markerless RGB-D camera enables 3D full-body motion capture in a less expensive and intrusive, and more portable way than multi-camera marker-based systems. In this study, we examine whether a gait analysis system based on an RGB-D camera can be used to detect significant changes in the gait of ATTRv amyloidosis patients over time, when compared with a 12-camera system. We acquired 3D data provided by both systems from six ATTRv amyloidosis patients, while performing a simple gait task, once (T0) and 18 months later (T1). A direct comparison of systems has already been conducted. In this work, however, for each patient, we investigated if the RGB-D camera system detects statistically significant differences between the two different acquisitions in a similar way to the reference system, and whether it is reliable to use during patients' follow-up. The obtained results show that the differences detected between T0 and T1 for both systems follow the same tendency for 65% of the spatiotemporal gait parameters, and for 38% of the kinematic parameters (38%). The most reliable parameters were: stride duration/length, gait speed (and its variability), and arm/foot swing velocity, all with an almost perfect strength of agreement. [ABSTRACT FROM AUTHOR]

Stability during walking is considered a crucial aspect of assessing gait ability. The current study aimed to assess walking stability by applying principal component analysis (PCA) to decompose three-dimensional (3D) whole-body kinematic data of 104 healthy young adults (21.9 ± 3.5 years, 54 females) derived from a depth-sensing camera into a set of movement components/synergies called "principal movements" (PMs), forming together to achieve the task goal. The effect of sex as the focus area was tested on three PCA-based variables computed for each PM: the relative explained variance (rVAR) as a measure of the composition of movement structures; the largest Lyapunov exponent (LyE) as a measure of variability; and the number of zero-crossings (N) as a measure of the tightness of neuromuscular control. The results show that the sex effects appear in the specific PMs. Specifically, in PM1, resembling the swing-phase movement, females have greater LyE (p = 0.013) and N (p = 0.017) values than males. Moreover, in PM3, representing the mid-stance-phase movement, females have smaller rVAR (p = 0.020) but greater N (p = 0.008) values than males. These empirical findings suggest that the inherent sex differences in walking stability should be considered in assessing and training locomotion. [ABSTRACT FROM AUTHOR]

The multi-fingered dexterous robotic hand is increasingly used to achieve more complex and sophisticated human-like manipulation tasks on various occasions. This paper proposes a hybrid Surface Electromyography (SEMG) and Kinect-based human in-hand motion (HIM) capture system architecture for recognizing complex motions of the humans by observing the state information between an object and the human hand, then transferring the manipulation skills into bionic multi-fingered robotic hand realizing dexterous in-hand manipulation. First, an Adaptive Directed Acyclic Graph (ADAG) algorithm for recognizing HIMs is proposed and optimized based on the comparison of multi-class support vector machines; second, ten representative complex in-hand motions are demonstrated by ten subjects, and SEMG and Kinect signals are obtained based on a multi-modal data acquisition platform; then, combined with the proposed algorithm framework, a series of data preprocessing algorithms are realized. There is statistical symmetry in similar types of SEMG signals and images, and asymmetry in different types of SEMG signals and images. A detailed analysis and an in-depth discussion are given from the results of the ADAG recognizing HIMs, motion recognition rates of different perceptrons, motion recognition rates of different subjects, motion recognition rates of different multi-class SVM methods, and motion recognition rates of different machine learning methods. The results of this experiment confirm the feasibility of the proposed method, with a recognition rate of 95.10%. [ABSTRACT FROM AUTHOR]

This paper presents a modular approach to generic exergame design that combines custom physical exercises in a meaningful and motivating story. This aims to provide a tool that can be individually tailored and adapted to people with different needs, making it applicable to different diseases and states of disease. The game is based on motion capturing and integrates four example exercises that can be configured via our therapeutic web platform "Blexer-med". To prove the feasibility for a wide range of different users, evaluation tests were performed on 14 patients with various types and degrees of neuromuscular disorders, classified into three groups based on strength and autonomy. The users were free to choose their schedule and frequency. The game scores and three surveys (before, during, and after the intervention) showed similar experiences for all groups, with the most vulnerable having the most fun and satisfaction. The players were motivated by the story and by achieving high scores. The average usage time was 2.5 times per week, 20 min per session. The pure exercise time was about half of the game time. The concept has proven feasible and forms a reasonable basis for further developments. The full 3D exercise needs further fine-tuning to enhance the fun and motivation. [ABSTRACT FROM AUTHOR]

Exercise games (exergames) that combine both exercise and video gaming train people in a fun and competitive manner to lead a healthy lifestyle. Exergames promote more physical exertion and help users exercise more easily and independently in any place. Many studies have been conducted to evaluate the positive effects of exergames. However, in most studies, heart rate was mainly used to measure the effect of exercise. In this study, we evaluate the effects of exercise according to the exercise type (rest, walking, tennis, and running) and gameplay mode (single, competition, and cooperation) of exergaming via quantitative measurements using electrocardiogram (ECG) and Kinect. The multiple comparison results reveal that physical activity measured with Kinect was statistically significant even in exergames that did not show statistically significant differences according to ECG. Running was statistically significant compared to other exercise types, and there was a significant difference in competition compared to other gameplay modes. [ABSTRACT FROM AUTHOR]

In this paper, we designed a methodology to classify facial nerve function after head and neck surgery. It is important to be able to observe the rehabilitation process objectively after a specific brain surgery, when patients are often affected by face palsy. The dataset that is used for classification problems in this study only contains 236 measurements of 127 patients of complex observations using the most commonly used House–Brackmann (HB) scale, which is based on the subjective opinion of the physician. Although there are several traditional evaluation methods for measuring facial paralysis, they still suffer from ignoring facial movement information. This plays an important role in the analysis of facial paralysis and limits the selection of useful facial features for the evaluation of facial paralysis. In this paper, we present a triple-path convolutional neural network (TPCNN) to evaluate the problem of mimetic muscle rehabilitation, which is observed by a Kinect stereovision camera. A system consisting of three modules for facial landmark measure computation and facial paralysis classification based on a parallel convolutional neural network structure is used to quantitatively assess the classification of facial nerve paralysis by considering facial features based on the region and the temporal variation of facial landmark sequences. The proposed deep network analyzes both the global and local facial movement features of a patient's face. These extracted high-level representations are then fused for the final evaluation of facial paralysis. The experimental results have verified the better performance of TPCNN compared to state-of-the-art deep learning networks. [ABSTRACT FROM AUTHOR]

Obtaining accurate and objective assessments of an individual's personality is vital in many areas including education, medicine, sports and management. Currently, most personality assessments are conducted using scales and questionnaires. Unfortunately, it has been observed that both scales and questionnaires present various drawbacks. Their limitations include the lack of veracity in the answers, limitations in the number of times they can be administered, or cultural biases. To solve these problems, several articles have been published in recent years proposing the use of movements that participants make during their evaluation as personality predictors. In this work, a multiple linear regression model was developed to assess the examinee's personality based on their movements. Movements were captured with the low-cost Microsoft Kinect camera, which facilitates its acceptance and implementation. To evaluate the performance of the proposed system, a pilot study was conducted aimed at assessing the personality traits defined by the Big-Five Personality Model. It was observed that the traits that best fit the model are Extroversion and Conscientiousness. In addition, several patterns that characterize the five personality traits were identified. These results show that it is feasible to assess an individual's personality through his or her movements and open up pathways for several research. [ABSTRACT FROM AUTHOR]

Current literature lacks a comparative analysis of different motion capture systems for tracking upper limb (UL) movement as individuals perform standard tasks. To better understand the performance of various motion capture systems in quantifying UL movement in the prosthesis user population, this study compares joint angles derived from three systems that vary in cost and motion capture mechanisms: a marker-based system (Vicon), an inertial measurement unit system (Xsens), and a markerless system (Kinect). Ten healthy participants (5F/5M; 29.6 ± 7.1 years) were trained with a TouchBionic i-Limb Ultra myoelectric terminal device mounted on a bypass prosthetic device. Participants were simultaneously recorded with all systems as they performed standardized tasks. Root mean square error and bias values for degrees of freedom in the right elbow, shoulder, neck, and torso were calculated. The IMU system yielded more accurate kinematics for shoulder, neck, and torso angles while the markerless system performed better for the elbow angles. By evaluating the ability of each system to capture kinematic changes of simulated upper limb prosthesis users during a variety of standardized tasks, this study provides insight into the advantages and limitations of using different motion capture technologies for upper limb functional assessment. [ABSTRACT FROM AUTHOR]

Kinematic analysis of the upper limbs is a good way to assess and monitor recovery in individuals with stroke, but it remains little used in clinical routine due to its low feasibility. The aim of this study is to assess the validity and reliability of the Kinect v2 for the analysis of upper limb reaching kinematics. Twenty-six healthy participants performed seated hand-reaching tasks while holding a dumbbell to induce behaviour similar to that of stroke survivors. With the Kinect v2 and with the VICON, 3D upper limb and trunk motions were simultaneously recorded. The Kinect assesses trunk compensations, hand range of motion, movement time and mean velocity with a moderate to excellent reliability. In contrast, elbow and shoulder range of motion, time to peak velocity and path length ratio have a poor to moderate reliability. Finally, instantaneous hand and elbow tracking are not precise enough to reliably assess the number of velocity peaks and the peak hand velocity. Thanks to its ease of use and markerless properties, the Kinect can be used in clinical routine for semi-automated quantitative diagnostics guiding individualised rehabilitation of the upper limb. However, engineers and therapists must bear in mind the tracking limitations of the Kinect. [ABSTRACT FROM AUTHOR]

Human Action Recognition (HAR) is a rapidly evolving field impacting numerous domains, among which is Ambient Assisted Living (AAL). In such a context, the aim of HAR is meeting the needs of frail individuals, whether elderly and/or disabled and promoting autonomous, safe and secure living. To this goal, we propose a monitoring system detecting dangerous situations by classifying human postures through Artificial Intelligence (AI) solutions. The developed algorithm works on a set of features computed from the skeleton data provided by four Kinect One systems simultaneously recording the scene from different angles and identifying the posture of the subject in an ecological context within each recorded frame. Here, we compare the recognition abilities of Multi-Layer Perceptron (MLP) and Long-Short Term Memory (LSTM) Sequence networks. Starting from the set of previously selected features we performed a further feature selection based on an SVM algorithm for the optimization of the MLP network and used a genetic algorithm for selecting the features for the LSTM sequence model. We then optimized the architecture and hyperparameters of both models before comparing their performances. The best MLP model (3 hidden layers and a Softmax output layer) achieved 78.4%, while the best LSTM (2 bidirectional LSTM layers, 2 dropout and a fully connected layer) reached 85.7%. The analysis of the performances on individual classes highlights the better suitability of the LSTM approach. [ABSTRACT FROM AUTHOR]

The Azure Kinect, the successor of Kinect v1 and Kinect v2, is a depth sensor. In this paper we evaluate the skeleton tracking abilities of the new sensor, namely accuracy and precision (repeatability). Firstly, we state the technical features of all three sensors, since we want to put the new Azure Kinect in the context of its previous versions. Then, we present the experimental results of general accuracy and precision obtained by measuring a plate mounted to a robotic manipulator end effector which was moved along the depth axis of each sensor and compare them. In the second experiment, we mounted a human-sized figurine to the end effector and placed it in the same positions as the test plate. Positions were located 400 mm from each other. In each position, we measured relative accuracy and precision (repeatability) of the detected figurine body joints. We compared the results and concluded that the Azure Kinect surpasses its discontinued predecessors, both in accuracy and precision. It is a suitable sensor for human–robot interaction, body-motion analysis, and other gesture-based applications. Our analysis serves as a pilot study for future HMI (human–machine interaction) designs and applications using the new Kinect Azure and puts it in the context of its successful predecessors. [ABSTRACT FROM AUTHOR]

The Sit-to-Stand (STS) is a widely used test of physical function to screen older people at risk of falls and frailty and is also one of the most important components of standard screening for sarcopenia. There have been many recent studies in which instrumented versions of the STS (iSTS) have been developed to provide additional parameters that could improve the accuracy of the STS test. This systematic review aimed to identify whether an iSTS is a viable alternative to a standard STS to identify older people at risk of falling, frailty, and sarcopenia. A total of 856 articles were found using the search strategy developed, with 12 articles retained in the review after screening based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Six studies evaluated the iSTS in fallers, five studies in frailty and only one study in both fallers and frailty. The results showed that power and velocity parameters extracted from an iSTS have the potential to improve the accuracy of screening when compared to a standard STS. Future work should focus on standardizing the segmentation of the STS into phases to enable comparison between studies and to develop devices integrated into the chair used for the test to improve usability. [ABSTRACT FROM AUTHOR]

Studies have demonstrated the validity of Kinect-based systems to measure spatiotemporal parameters of gait. However, few studies have addressed test-retest, inter-rater and intra-rater reliability for spatiotemporal gait parameters. This study aims to assess test-retest, inter-rater and intra-rater reliability of SANE (eaSy gAit aNalysis system) as a measuring instrument for spatiotemporal gait parameters. SANE comprises a depth sensor and a software that automatically estimates spatiotemporal gait parameters using distances between ankles without the need to manually indicate where each gait cycle begins and ends. Gait analysis was conducted by 2 evaluators for 12 healthy subjects during 4 sessions. The reliability was evaluated using Intraclass Correlation Coefficients (ICC). In addition, the Standard Error of the Measurement (SEM), and Smallest Detectable Change (SDC) was calculated. SANE showed from an acceptable to an excellent test-retest, inter-rater and intra-rater reliability; test-retest reliability ranged from 0.62 to 0.81, inter-rater reliability ranged from 0.70 to 0.95 and intra-rater ranged from 0.74 to 0.92. The subject behavior had a greater effect on the reliability of SANE than the evaluator performance. The reliability values of SANE were comparable with other similar studies. SANE, as a feasible and markerless system, has large potential for assessing spatiotemporal gait parameters. [ABSTRACT FROM AUTHOR]

With the increase in life expectancy, one of the most important topic for scientific research, especially for the elderly, is good nutrition. In particular, with an advanced age and health issues because disorders such as Alzheimer and dementia, monitoring the subjects' dietary habits to avoid excessive or poor nutrition is a critical role. Starting from an application aiming to monitor the food intake actions of people during a meal, already shown in a previously published paper, the present work describes some improvements that are able to make the application work in real time. The considered solution exploits the Kinect v1 device that can be installed on the ceiling, in a top-down view in an effort to preserve privacy of the subjects. The food intake actions are estimated from the analysis of depth frames. The innovations introduced in this document are related to the automatic identification of the initial and final frame for the detection of food intake actions, and to the strong revision of the procedure to identify food intake actions with respect to the original work, in order to optimize the performance of the algorithm. Evaluation of the computational effort and system performance compared to the previous version of the application has demonstrated a possible real-time applicability of the solution presented in this document. [ABSTRACT FROM AUTHOR]

As indoor mobile navigation suffers from low positioning accuracy and accumulation error, we carried out research into an integrated location system for a robot based on Kinect and an Inertial Measurement Unit (IMU). In this paper, the close-range stereo images are used to calculate the attitude information and the translation amount of the adjacent positions of the robot by means of the absolute orientation algorithm, for improving the calculation accuracy of the robot’s movement. Relying on the Kinect visual measurement and the strap-down IMU devices, we also use Kalman filtering to obtain the errors of the position and attitude outputs, in order to seek the optimal estimation and correct the errors. Experimental results show that the proposed method is able to improve the positioning accuracy and stability of the indoor mobile robot. [ABSTRACT FROM AUTHOR]

Creating a navigation system for autonomous companion robots has always been a difficult process, which must contend with a dynamically changing environment, which is populated by a myriad of obstructions and an unspecific number of people, other than the intended person, to follow. This study documents the implementation of an indoor autonomous robot navigation model, based on multi-sensor fusion, using Microsoft Robotics Developer Studio 4 (MRDS). The model relies on a depth camera, a limited array of proximity sensors and an active IR marker tracking system. This allows the robot to lock onto the correct target for human-following, while approximating the best starting direction to begin maneuvering around obstacles for minimum required motion. The system is implemented according to a navigation algorithm that transforms the data from all three types of sensors into tendency arrays and fuses them to determine whether to take a leftward or rightward route around an encountered obstacle. The decision process considers visible short, medium and long-range obstructions and the current position of the target person. The system is implemented using MRDS and its functional test performance is presented over a series of Virtual Simulation Environment scenarios, greenlighting further extensive benchmark simulations. [ABSTRACT FROM AUTHOR]

During the last few years, there has been a growing interest in students getting engaged in digital game-making activities so as to enhance their thinking skills. The findings of studies that have examined the impact of such initiatives are quite positive, especially concerning the promotion of 21st century skills; however, many students seem to face difficulties in getting a deeper understanding of the game development life cycle. Additionally, students often have difficulties in meaningfully reusing and applying the concepts from various subjects, mainly mathematics and physics, into their game-making tasks or in understanding advanced programming commands while creating their games. The present study presents an innovative game-making teaching approach that suggests a series of steps for the systematic design and rapid development of motion-based touchless games, i.e., games that are based on natural user interaction technologies, like the Microsoft Kinect camera. Findings from evaluation studies in two (2) secondary schools indicate that this approach can increase student motivation, strengthen their computational thinking, enhance their understanding of geometric principles and improve their social skills. [ABSTRACT FROM AUTHOR]

The aim of this study was to compare the performance between young adults (n = 15), healthy old people (n = 10), and masters athletes (n = 15) using a depth sensor and automated digital assessment framework. Participants were asked to complete a clinically validated assessment of the sit-to-stand technique (five repetitions), which was recorded using a depth sensor. A feature encoding and evaluation framework to assess balance, core, and limb performance using time- and speed-related measurements was applied to markerless motion capture data. The associations between the measurements and participant groups were examined and used to evaluate the assessment framework suitability. The proposed framework could identify phases of sit-to-stand, stability, transition style, and performance between participant groups with a high degree of accuracy. In summary, we found that a depth sensor coupled with the proposed framework could identify performance subtleties between groups. [ABSTRACT FROM AUTHOR]

With the introduction of cost-effective depth sensors, a tremendous amount of research has been devoted to studying human action recognition using 3D motion data. However, most existing methods work in an offline fashion, i.e., they operate on a segmented sequence. There are a few methods specifically designed for online action recognition, which continually predicts action labels as a stream sequence proceeds. In view of this fact, we propose a question: can we draw inspirations and borrow techniques or descriptors from existing offline methods, and then apply these to online action recognition? Note that extending offline techniques or descriptors to online applications is not straightforward, since at least two problems—including real-time performance and sequence segmentation—are usually not considered in offline action recognition. In this paper, we give a positive answer to the question. To develop applicable online action recognition methods, we carefully explore feature extraction, sequence segmentation, computational costs, and classifier selection. The effectiveness of the developed methods is validated on the MSR 3D Online Action dataset and the MSR Daily Activity 3D dataset. [ABSTRACT FROM AUTHOR]

Traditional indoor laser scanning trolley/backpacks with multi-laser scanner, panorama cameras, and an inertial measurement unit (IMU) installed are a popular solution to the 3D indoor mapping problem. However, the cost of those mapping suits is quite expensive, and can hardly be replicated by consumer electronic components. The consumer RGB-Depth (RGB-D) camera (e.g., Kinect V2) is a low-cost option for gathering 3D point clouds. However, because of the narrow field of view (FOV), its collection efficiency and data coverages are lower than that of laser scanners. Additionally, the limited FOV leads to an increase of the scanning workload, data processing burden, and risk of visual odometry (VO)/simultaneous localization and mapping (SLAM) failure. To find an efficient and low-cost way to collect 3D point clouds data with auxiliary information (i.e., color) for indoor mapping, in this paper we present a prototype indoor mapping solution that is built upon the calibration of multiple RGB-D sensors to construct an array with large FOV. Three time-of-flight (ToF)-based Kinect V2 RGB-D cameras are mounted on a rig with different view directions in order to form a large field of view. The three RGB-D data streams are synchronized and gathered by the OpenKinect driver. The intrinsic calibration that involves the geometry and depth calibration of single RGB-D cameras are solved by homography-based method and ray correction followed by range biases correction based on pixel-wise spline line functions, respectively. The extrinsic calibration is achieved through a coarse-to-fine scheme that solves the initial exterior orientation parameters (EoPs) from sparse control markers and further refines the initial value by an iterative closest point (ICP) variant minimizing the distance between the RGB-D point clouds and the referenced laser point clouds. The effectiveness and accuracy of the proposed prototype and calibration method are evaluated by comparing the point clouds derived from the prototype with ground truth data collected by a terrestrial laser scanner (TLS). The overall analysis of the results shows that the proposed method achieves the seamless integration of multiple point clouds from three Kinect V2 cameras collected at 30 frames per second, resulting in low-cost, efficient, and high-coverage 3D color point cloud collection for indoor mapping applications. [ABSTRACT FROM AUTHOR]

In this paper, we demonstrate the feasibility of using the Microsoft Kinect and Google Tango frame-based depth sensors for individual tree stem measurements and reconstruction for the purpose of forest inventory. Conventionally field reference data in forest inventory are collected at tree and sample plot level by means of manual measurements (e.g., a caliper), which are both labor-intensive and time-consuming. In this study, color (i.e., red, green and blue channels, RGB) and range images acquired by a Kinect and Tango systems were processed and used to extract tree diameter measurements for the individual tree stems. For this, 121 reference stem diameter measurements were made with tape and caliper. Kinect-derived tree diameters agreed with tape measurements to a 1.90 cm root-mean-square error (RMSE). The stem curve from the ground to the diameter at breast height agreed with a bias of 0.7 cm and random error of 0.8 cm with respect to the reference trunk. For Tango measurements, the obtained stem diameters matched those from tape measurement with an RMSE of 0.73 cm, having an average bias of 0.3 cm. As highly portable and inexpensive systems, both Kinect and Tango provide an easy way to collect tree stem diameter and stem curve information vital to forest inventory. These inexpensive instruments may in future compete with both terrestrial and mobile laser scanning or conventional fieldwork using calipers or tape. Accuracy is adequate for practical applications in forestry. Measurements made using Kinect and Tango type systems could also be applied in crowdsourcing context. [ABSTRACT FROM AUTHOR]

To ensure the safety of a handle-type electric wheelchair (hereinafter, electric wheelchair), this paper describes the applicability of using a Kinect sensor. Ensuring the mobility of elderly people is a particularly important issue to be resolved. An electric wheelchair is useful as a means of transportation for elderly people. Considering that the users of electric wheelchairs are elderly people, it is important to ensure the safety of electric wheelchairs at night. To ensure the safety of an electric wheelchair at night, we constructed a hazardous object detection system using commercially available and inexpensive Kinect sensors and examined the applicability of the system. We examined warning timing with consideration to the cognition, judgment, and operation time of elderly people. Based on this, a hazardous object detection area was determined. Furthermore, the detection of static and dynamic hazardous objects was carried out at night and the results showed that the system was able to detect with high accuracy. We also conducted experiments related to dynamic hazardous object detection during daytime. From the above, it showed that the system could be applicable to ensuring the safety of the handle-type electric wheelchair. [ABSTRACT FROM AUTHOR]

This work presents an Indoor Positioning System to estimate the location of people navigating in complex indoor environments. The developed technique combines WiFi Positioning Systems and depth maps, delivering promising results in complex inhabited environments, consisting of various connected rooms, where people are freely moving. This is a non-intrusive system in which personal information about subjects is not needed and, although RGB-D cameras are installed in the sensing area, users are only required to carry their smart-phones. In this article, the methods developed to combine the above-mentioned technologies and the experiments performed to test the system are detailed. The obtained results show a significant improvement in terms of accuracy and performance with respect to previous WiFi-based solutions as well as an extension in the range of operation. [ABSTRACT FROM AUTHOR]

This paper presents a new approach for fall detection based on two features and their motion characteristics extracted from the human torso. The 3D positions of the hip center joint and the shoulder center joint in depth images are used to build a fall detection model named the human torso motion model (HTMM). Person's torso angle and centroid height are imported as key features in HTMM. Once a person comes into the scene, the positions of these two joints are fetched to calculate the person's torso angle. Whenever the angle is larger than a given threshold, the changing rates of the torso angle and the centroid height are recorded frame by frame in a given period of time. A fall can be identified when the above two changing rates reach the thresholds. By using the new feature, falls can be accurately and effectively distinguished from other fall-like activities, which are very difficult for other computer vision-based approaches to differentiate. Experiment results show that our approach achieved a detection accuracy of 97.5%, 98% true positive rate (TPR) and 97% true negative rate (TNR). Furthermore, the approach is time efficient and robust because of only calculating the changing rate of gravity and centroid height. [ABSTRACT FROM AUTHOR]

Contactless detection is one of the new frontiers of technological innovation in the field of healthcare, enabling unobtrusive measurements of biomedical parameters. Compared to conventional methods for Heart Rate (HR) detection that employ expensive and/or uncomfortable devices, such as the Electrocardiograph (ECG) or pulse oximeter, contactless HR detection offers fast and continuous monitoring of heart activities and provides support for clinical analysis without the need for the user to wear a device. This paper presents a validation study for a contactless HR estimation method exploiting RGB (Red, Green, Blue) data from a Microsoft Kinect v2 device. This method, based on Eulerian Video Magnification (EVM), Photoplethysmography (PPG) and Videoplethysmography (VPG), can achieve performance comparable to classical approaches exploiting wearable systems, under specific test conditions. The output given by a Holter, which represents the gold-standard device used in the test for ECG extraction, is considered as the ground-truth, while a comparison with a commercial smartwatch is also included. The validation process is conducted with two modalities that differ for the availability of a priori knowledge about the subjects' normal HR. The two test modalities provide different results. In particular, the HR estimation differs from the ground-truth by 2% when the knowledge about the subject's lifestyle and his/her HR is considered and by 3.4% if no information about the person is taken into account. [ABSTRACT FROM AUTHOR]

This work used a rotating disc electrode and quasi-steady state polarization curves to investigate the sodium borohydride electrooxidation of ordered intermetallic PtSn/C in alkaline solution. Under similar experimental conditions, PtSn/C proved to be a better electrocatalyst than Pt in an overall process that involved eight electrons. Assays performed in the presence of thiourea and S2- species evidenced that a chemical hydrolysis step took place, followed by electrochemical oxidation of the generated H2. The results presented herein suggest that PtSn/C constitutes a promising electrode material for application in alkaline borohydride fuel cell. [ABSTRACT FROM AUTHOR]

Camera calibration is a crucial problem in many applications, such as 3D reconstruction, structure from motion, object tracking and face alignment. Numerous methods have been proposed to solve the above problem with good performance in the last few decades. However, few methods are targeted at joint calibration of multi-sensors (more than four devices), which normally is a practical issue in the real-time systems. In this paper, we propose a novel method and a corresponding workflow framework to simultaneously calibrate relative poses of a Kinect and three external cameras. By optimizing the final cost function and adding corresponding weights to the external cameras in different locations, an effective joint calibration of multiple devices is constructed. Furthermore, the method is tested in a practical platform, and experiment results show that the proposed joint calibration method can achieve a satisfactory performance in a project real-time system and its accuracy is higher than the manufacturer’s calibration. [ABSTRACT FROM AUTHOR]

The feasibility of automated individual crop plant care in vegetable crop fields has increased, resulting in improved efficiency and economic benefits. A systems-based approach is a key feature in the engineering design of mechanization that incorporates precision sensing techniques. The objective of this study was to design new sensing capabilities to measure crop plant spacing under different test conditions (California, USA and Andalucía, Spain). For this study, three different types of optical sensors were used: an optical light-beam sensor (880 nm), a Light Detection and Ranging (LiDAR) sensor (905 nm), and an RGB camera. Field trials were conducted on newly transplanted tomato plants, using an encoder as a local reference system. Test results achieved a 98% accuracy in detection using light-beam sensors while a 96% accuracy on plant detections was achieved in the best of replications using LiDAR. These results can contribute to the decision-making regarding the use of these sensors by machinery manufacturers. This could lead to an advance in the physical or chemical weed control on row crops, allowing significant reductions or even elimination of hand-weeding tasks. [ABSTRACT FROM AUTHOR]

The realistic representation of deformations is still an active area of research, especially for deformable objects whose behavior cannot be simply described in terms of elasticity parameters. This paper proposes a data-driven neural-network-based approach for capturing implicitly and predicting the deformations of an object subject to external forces. Visual data, in the form of 3D point clouds gathered by a Kinect sensor, is collected over an object while forces are exerted by means of the probing tip of a force-torque sensor. A novel approach based on neural gas fitting is proposed to describe the particularities of a deformation over the selectively simplified 3D surface of the object, without requiring knowledge of the object material. An alignment procedure, a distance-based clustering, and inspiration from stratified sampling support this process. The resulting representation is denser in the region of the deformation (an average of 96.6% perceptual similarity with the collected data in the deformed area), while still preserving the object's overall shape (86% similarity over the entire surface) and only using on average of 40% of the number of vertices in the mesh. A series of feedforward neural networks is then trained to predict the mapping between the force parameters characterizing the interaction with the object and the change in the object shape, as captured by the fitted neural gas nodes. This series of networks allows for the prediction of the deformation of an object when subject to unknown interactions. [ABSTRACT FROM AUTHOR]

Depth information has been used in many fields because of its low cost and easy availability, since the Microsoft Kinect was released. However, the Kinect and Kinect-like RGB-D sensors show limited performance in certain applications and place high demands on accuracy and robustness of depth information. In this paper, we propose a depth sensing system that contains a laser projector similar to that used in the Kinect, and two infrared cameras located on both sides of the laser projector, to obtain higher spatial resolution depth information. We apply the block-matching algorithm to estimate the disparity. To improve the spatial resolution, we reduce the size of matching blocks, but smaller matching blocks generate lower matching precision. To address this problem, we combine two matching modes (binocular mode and monocular mode) in the disparity estimation process. Experimental results show that our method can obtain higher spatial resolution depth without loss of the quality of the range image, compared with the Kinect. Furthermore, our algorithm is implemented on a low-cost hardware platform, and the system can support the resolution of 1280 x 960, and up to a speed of 60 frames per second, for depth image sequences. [ABSTRACT FROM AUTHOR]

Previous studies on virtual soap bubbles mainly focused on methods for visualizing the physical and geometrical properties of soap bubbles and paid little attention to the possible ways to enhance the interaction between the simulation and the user. In this paper, a user interaction-based giant soap bubble simulation system is proposed in which the free-form shape, size, and position of giant soap bubbles are determined by the user's hand motions. Our method improves the controllability of soap bubble simulation by correcting the jerky hand trajectory and hand velocity to a smooth and gradual path. Our air flow transfer algorithm can produce detailed deformation and standing wave for soap film in real time. Our novel soap film bursting algorithm represents the process of the bursting phenomenon of soap-film and giant soap bubbles in a unified framework. The results of our experiment demonstrate that the system allows the user to experience the giant soap bubble blowing and bursting process in a virtual environment. [ABSTRACT FROM AUTHOR]

RGB-D (Red Green Blue and Depth) sensors are devices that can provide color and depth information from a scene at the same time. Recently, they have been widely used in many solutions due to their commercial growth from the entertainment market to many diverse areas (e.g., robotics, CAD, etc.). In the research community, these devices have had good uptake due to their acceptable level of accuracy for many applications and their low cost, but in some cases, they work at the limit of their sensitivity, near to the minimum feature size that can be perceived. For this reason, calibration processes are critical in order to increase their accuracy and enable them to meet the requirements of such kinds of applications. To the best of our knowledge, there is not a comparative study of calibration algorithms evaluating its results in multiple RGB-D sensors. Specifically, in this paper, a comparison of the three most used calibration methods have been applied to three different RGB-D sensors based on structured light and time-of-flight. The comparison of methods has been carried out by a set of experiments to evaluate the accuracy of depth measurements. Additionally, an object reconstruction application has been used as example of an application for which the sensor works at the limit of its sensitivity. The obtained results of reconstruction have been evaluated through visual inspection and quantitative measurements. [ABSTRACT FROM AUTHOR]

The work presented here suggests new ways to tackle exergames for physical rehabilitation and to improve the players' immersion and involvement. The primary (but not exclusive) purpose is to increase the motivation of children and adolescents with severe physical impairments, for doing their required exercises while playing. The proposed gaming environment is based on the Kinect sensor and the Blender Game Engine. A middleware has been implemented that efficiently transmits the data from the sensor to the game. Inside the game, different newly proposed mechanisms have been developed to distinguish pure exercise-gestures from other movements used to control the game (e.g., opening a menu). The main contribution is the amplification of weak movements, which allows the physically impaired to have similar gaming experiences as the average population. To test the feasibility of the proposed methods, four mini-games were implemented and tested by a group of 11 volunteers with different disabilities, most of them bound to a wheelchair. Their performance has also been compared to that of a healthy control group. Results are generally positive and motivating, although there is much to do to improve the functionalities. There is a major demand for applications that help to include disabled people in society and to improve their life conditions. This work will contribute towards providing them with more fun during exercise. [ABSTRACT FROM AUTHOR]

Assessment of body kinematics during performance of daily life activities at home plays a significant role in medical condition monitoring of elderly people and patients with neurological disorders. The affordable and non-wearable Microsoft Kinect ("Kinect") system has been recently used to estimate human subject kinematic features. However, the Kinect suffers from a limited range and angular coverage, distortion in skeleton joints' estimations, and erroneous multiplexing of different subjects' estimations to one. This study addresses these limitations by incorporating a set of features that create a unique "Kinect Signature". The Kinect Signature enables identification of different subjects in the scene, automatically assign the kinematics feature estimations only to the subject of interest, and provide information about the quality of the Kinect-based estimations. The methods were verified by a set of experiments, which utilize real-time scenarios commonly used to assess motor functions in elderly subjects and in subjects with neurological disorders. The experiment results indicate that the skeleton based Kinect Signature features can be used to identify different subjects in high accuracy. We demonstrate how these capabilities can be used to assign the Kinect estimations to the Subject of Interest, and exclude low quality tracking features. The results of this work can help in establishing reliable kinematic features, which can assist in future to obtain objective scores for medical analysis of patient condition at home while not restricted to perform daily life activities. [ABSTRACT FROM AUTHOR]

Human gait analysis plays an important role in musculoskeletal disorder diagnosis. Detecting anomalies in human walking, such as shuffling gait, stiff leg or unsteady gait, can be difficult if the prior knowledge of such a gait pattern is not available. We propose an approach for detecting abnormal human gait based on a normal gait model. Instead of employing the color image, silhouette, or spatio-temporal volume, our model is created based on human joint positions (skeleton) in time series. We decompose each sequence of normal gait images into gait cycles. Each human instant posture is represented by a feature vector which describes relationships between pairs of bone joints located in the lower body. Such vectors are then converted into codewords using a clustering technique. The normal human gait model is created based on multiple sequences of codewords corresponding to different gait cycles. In the detection stage, a gait cycle with normality likelihood below a threshold, which is determined automatically in the training step, is assumed as an anomaly. The experimental results on both marker-based mocap data and Kinect skeleton show that our method is very promising in distinguishing normal and abnormal gaits with an overall accuracy of 90.12%. [ABSTRACT FROM AUTHOR]