Search

Depth maps produced by consumer-grade sensors suffer from inaccurate measurements and missing data from either system or scene-specific sources. Data-driven denoising algorithms can mitigate such problems; however, they require vast amounts of ground truth depth data. Recent research has tackled this limitation using self-supervised learning techniques, but it requires multiple RGB-D sensors. Moreover, most existing approaches focus on denoising single isolated depth maps or specific subjects of interest highlighting a need for methods that can effectively denoise depth maps in real-time dynamic environments. This paper extends state-of-the-art approaches for depth-denoising commodity depth devices, proposing SelfReDepth, a self-supervised deep learning technique for depth restoration, via denoising and hole-filling by inpainting of full-depth maps captured with RGB-D sensors. The algorithm targets depth data in video streams, utilizing multiple sequential depth frames coupled with color data to achieve high-quality depth videos with temporal coherence. Finally, SelfReDepth is designed to be compatible with various RGB-D sensors and usable in real-time scenarios as a pre-processing step before applying other depth-dependent algorithms. Our results demonstrate our approach’s real-time performance on real-world datasets shows that it outperforms state-of-the-art methods in denoising and restoration performance at over 30 fps on Commercial Depth Cameras, with potential benefits for augmented and mixed-reality applications. [ABSTRACT FROM AUTHOR]

Learning about appearance embedding is of great importance for a variety of different computer-vision applications, which has prompted a surge in person re-identification (Re-ID) papers. The aim of these papers has been to identify an individual over a set of non-overlapping cameras. Despite recent advances in RGB–RGB Re-ID approaches with deep-learning architectures, the approach fails to consistently work well when there are low resolutions in dark conditions. The introduction of different sensors (i.e., RGB–D and infrared (IR)) enables the capture of appearances even in dark conditions. Recently, a lot of research has been dedicated to addressing the issue of finding appearance embedding in dark conditions using different advanced camera sensors. In this paper, we give a comprehensive overview of existing Re-ID approaches that utilize the additional information from different sensor-based methods to address the constraints faced by RGB camera-based person Re-ID systems. Although there are a number of survey papers that consider either the RGB–RGB or Visible-IR scenarios, there are none that consider both RGB–D and RGB–IR. In this paper, we present a detailed taxonomy of the existing approaches along with the existing RGB–D and RGB–IR person Re-ID datasets. Then, we summarize the performance of state-of-the-art methods on several representative RGB–D and RGB–IR datasets. Finally, future directions and current issues are considered for improving the different sensor-based person Re-ID systems. [ABSTRACT FROM AUTHOR]

Objective: Monitoring infants’ breathing activity is crucial in research and clinical applications but remains a challenge. This study aims to develop a contactless method to monitor breathing patterns and thoracoabdominal asynchronies in infants inside the incubator, using depth cameras. Methods: We proposed an algorithm to extract the 3D displacements of the ribcage and abdomen from the analysis of depth images. We evaluated the accuracy of the system in-vitro vs. a reference motion capture analyzer. We also conducted a feasibility study on 12 patients receiving non-invasive respiratory support to estimate the mean and the variability of the chest wall displacements in preterm infants and evaluate the suitability of the proposed system in the clinical setting. Results: In-vitro, the mean (95% CI) error in the measurement of amplitude, frequency and phase shift between compartmental displacements was −0.14 (−0.57, 0.28) mm, 0.02 (−0.99, 1.03) bpm, and −0.40 (−1.76, 0.95)°, respectively. In-vivo, the mean (95% CI) amplitude of the ribcage and abdomen displacements were 0.99 (0.34, 2.67) mm and 1.20 (0.40, 2.15) mm, respectively. Conclusions: The developed system proved accurate in-vitro and was suitable for the clinical environment. Clinical Impact: The proposed method has value for evaluating infants’ breathing patterns in research applications and, after further development, may represent a simple monitoring tool for infants’ respiratory activity inside the incubator.

The aim of this review was to present an overview of the state of the art in the use of the Microsoft Kinect camera to assess gait in post-stroke individuals through an analysis of the available literature. In recent years, several studies have explored the potentiality, accuracy, and effectiveness of this 3D optical sensor as an easy-to-use and non-invasive clinical measurement tool for the assessment of gait parameters in several pathologies. Focusing on stroke individuals, some of the available studies aimed to directly assess and characterize their gait patterns. In contrast, other studies focused on the validation of Kinect-based measurements with respect to a gold-standard reference (i.e., optoelectronic systems). However, the nonhomogeneous characteristics of the participants, of the measures, of the methodologies, and of the purposes of the studies make it difficult to adequately compare the results. This leads to uncertainties about the strengths and weaknesses of this technology in this pathological state. The final purpose of this narrative review was to describe and summarize the main features of the available works on gait in the post-stroke population, highlighting similarities and differences in the methodological approach and primary findings, thus facilitating comparisons of the studies as much as possible. [ABSTRACT FROM AUTHOR]

Analysing the dynamics in social interactions in indoor spaces entails evaluating spatial–temporal variables from the event, such as location and time. Additionally, social interactions include invisible spaces that we unconsciously acknowledge due to social constraints, e.g., space between people having a conversation with each other. Nevertheless, current sensor arrays focus on detecting the physically occupied spaces from social interactions, i.e., areas inhabited by physically measurable objects. Our goal is to detect the socially occupied spaces, i.e., spaces not physically occupied by subjects and objects but inhabited by the interaction they sustain. We evaluate the social representation of the space structure between two or more active participants, so-called F-Formation for small gatherings. We propose calculating body orientation and location from skeleton joint data sets by integrating depth cameras. The body orientation is derived by integrating the shoulders and spine joint data with head/face rotation data and spatial–temporal information from trajectories. From the physically occupied measurements, we can detect socially occupied spaces. In our user study implementing the system, we compared the capabilities and skeleton tracking datasets from three depth camera sensors, the Kinect v2, Azure Kinect, and Zed 2i. We collected 32 walking patterns for individual and dyad configurations and evaluated the system's accuracy regarding the intended and socially accepted orientations. Experimental results show accuracy above 90% for the Kinect v2, 96% for the Azure Kinect, and 89% for the Zed 2i for assessing socially relevant body orientation. Our algorithm contributes to the anonymous and automated assessment of socially occupied spaces. The depth sensor system is promising in detecting more complex social structures. These findings impact research areas that study group interactions within complex indoor settings. [ABSTRACT FROM AUTHOR]

Efficient management of on-shelf availability and inventory is a key issue to achieve customer satisfaction and reduce the risk of profit loss for both retailers and manufacturers. Conventional store audits based on physical inspection of shelves are labor-intensive and do not provide reliable assessment. This paper describes a novel framework for automated shelf monitoring, using a consumer-grade depth sensor. The aim is to develop a low-cost embedded system for early detection of out-of-stock situations with particular regard to perishable goods stored in countertop shelves, refrigerated counters, baskets or crates. The proposed solution exploits 3D point cloud reconstruction and modelling techniques, including surface fitting and occupancy grids, to estimate product availability, based on the comparison between a reference model of the shelf and its current status. No a priori knowledge about the product type is required, while the shelf reference model is automatically learnt based on an initial training stage. The output of the system can be used to generate alerts for store managers, as well as to continuously update product availability estimates for automated stock ordering and replenishment and for e-commerce apps. Experimental tests performed in a real retail environment show that the proposed system is able to estimate the on-shelf availability percentage of different fresh products with a maximum average discrepancy with respect to the actual one of about 5.0%.

Learning about appearance embedding is of great importance for a variety of different computer-vision applications, which has prompted a surge in person re-identification (Re-ID) papers. The aim of these papers has been to identify an individual over a set of non-overlapping cameras. Despite recent advances in RGB–RGB Re-ID approaches with deep-learning architectures, the approach fails to consistently work well when there are low resolutions in dark conditions. The introduction of different sensors (i.e., RGB–D and infrared (IR)) enables the capture of appearances even in dark conditions. Recently, a lot of research has been dedicated to addressing the issue of finding appearance embedding in dark conditions using different advanced camera sensors. In this paper, we give a comprehensive overview of existing Re-ID approaches that utilize the additional information from different sensor-based methods to address the constraints faced by RGB camera-based person Re-ID systems. Although there are a number of survey papers that consider either the RGB–RGB or Visible-IR scenarios, there are none that consider both RGB–D and RGB–IR. In this paper, we present a detailed taxonomy of the existing approaches along with the existing RGB–D and RGB–IR person Re-ID datasets. Then, we summarize the performance of state-of-the-art methods on several representative RGB–D and RGB–IR datasets. Finally, future directions and current issues are considered for improving the different sensor-based person Re-ID systems.

Person re-identification (Re-id) is the task of recognizing people across different non-overlapping sensors of a camera network. Despite the recent advances with deep learning (DL) models for multi-modal fusion, state-of-the-art Re-id approaches fail to leverage the depth guided contextual information to dynamically select the most discriminant convolutional filters for better feature embedding and inference. Thanks to low cost modern RGB-D sensors (e.g. Microsoft Kinect and Intel RealSense Depth camera) that avail us with different modalities such as illumination invariant high-quality depth images, RGB images and skeleton information can be obtained simultaneously. State-of-the-art Re-id approaches utilize multi-modal fusion in feature space where the chances of fused noisy features to dominate the final recognition process is high. In this paper, we address this issue by exploiting the advantage of using an effective fusion technique in dissimilarity space. Given a query RGB-D image of an individual, two CNNs are separately trained with 3-channel RGB and 4-channel RGB-D images to produce two different feature embeddings required for pair-wise matching with embeddings for reference images, where dissimilarity scores w.r.t reference images from both modalities are fused together for final ranking. Additionally, lack of a proper RGB-D Re-id dataset prompts us to contribute a new RGB-D Re-id dataset named SUCVL RGBD-ID, including RGB and depth images of 58 identities from three cameras where one camera was installed in poor illumination conditions and the remaining two cameras were installed in two different indoor locations with different indoor lighting environments. Extensive experimental analysis on our dataset and two publicly available datasets show the effectiveness of our proposed method. Moreover, our proposed method is general and can be applied to a multitude of different RGB-D based applications.

The aim of this review was to present an overview of the state of the art in the use of the Microsoft Kinect camera to assess gait in post-stroke individuals through an analysis of the available literature. In recent years, several studies have explored the potentiality, accuracy, and effectiveness of this 3D optical sensor as an easy-to-use and non-invasive clinical measurement tool for the assessment of gait parameters in several pathologies. Focusing on stroke individuals, some of the available studies aimed to directly assess and characterize their gait patterns. In contrast, other studies focused on the validation of Kinect-based measurements with respect to a gold-standard reference (i.e., optoelectronic systems). However, the nonhomogeneous characteristics of the participants, of the measures, of the methodologies, and of the purposes of the studies make it difficult to adequately compare the results. This leads to uncertainties about the strengths and weaknesses of this technology in this pathological state. The final purpose of this narrative review was to describe and summarize the main features of the available works on gait in the post-stroke population, highlighting similarities and differences in the methodological approach and primary findings, thus facilitating comparisons of the studies as much as possible.

Analysing the dynamics in social interactions in indoor spaces entails evaluating spatial–temporal variables from the event, such as location and time. Additionally, social interactions include invisible spaces that we unconsciously acknowledge due to social constraints, e.g., space between people having a conversation with each other. Nevertheless, current sensor arrays focus on detecting the physically occupied spaces from social interactions, i.e., areas inhabited by physically measurable objects. Our goal is to detect the socially occupied spaces, i.e., spaces not physically occupied by subjects and objects but inhabited by the interaction they sustain. We evaluate the social representation of the space structure between two or more active participants, so-called F-Formation for small gatherings. We propose calculating body orientation and location from skeleton joint data sets by integrating depth cameras. The body orientation is derived by integrating the shoulders and spine joint data with head/face rotation data and spatial–temporal information from trajectories. From the physically occupied measurements, we can detect socially occupied spaces. In our user study implementing the system, we compared the capabilities and skeleton tracking datasets from three depth camera sensors, the Kinect v2, Azure Kinect, and Zed 2i. We collected 32 walking patterns for individual and dyad configurations and evaluated the system’s accuracy regarding the intended and socially accepted orientations. Experimental results show accuracy above 90% for the Kinect v2, 96% for the Azure Kinect, and 89% for the Zed 2i for assessing socially relevant body orientation. Our algorithm contributes to the anonymous and automated assessment of socially occupied spaces. The depth sensor system is promising in detecting more complex social structures. These findings impact research areas that study group interactions within complex indoor settings.

The accurate and reliable assessment of gait parameters is assuming an important role, especially in the perspective of designing new therapeutic and rehabilitation strategies for the remote follow-up of people affected by disabling neurological diseases, including Parkinson’s disease and post-stroke injuries, in particular considering how gait represents a fundamental motor activity for the autonomy, domestic or otherwise, and the health of neurological patients. To this end, the study presents an easy-to-use and non-invasive solution, based on a single RGB-D sensor, to estimate specific features of gait patterns on a reduced walking path compatible with the available spaces in domestic settings. Traditional spatio-temporal parameters and features linked to dynamic instability during walking are estimated on a cohort of ten parkinsonian and eleven post-stroke subjects using a custom-written software that works on the result of a body-tracking algorithm. Then, they are compared with the “gold standard” 3D instrumented gait analysis system. The statistical analysis confirms no statistical difference between the two systems. Data also indicate that the RGB-D system is able to estimate features of gait patterns in pathological individuals and differences between them in line with other studies. Although they are preliminary, the results suggest that this solution could be clinically helpful in evolutionary disease monitoring, especially in domestic and unsupervised environments where traditional gait analysis is not usable.

Measuring Range of Motion (ROM) is the first step of physical therapy. A new method to measure ROM by Kinect V2 whose camera type is time of flight is proposed. Colored markers are attached to related joints and then their camera centered three-dimensional world coordinates are located by Kinect. Using these coordinates, joint angle, and ROM can be accurately calculated. To analyze reliability and validity of the method, ROM measurements of right and left elbow from ten participants are taken by standard goniometer and Kinect separately. For inter-observer reliability, measurements were taken in two sessions by three physiotherapists. The reliability tests Intra-class Correlation Coefficient (ICC), Standard Error of Measure (SEM), and Minimal Detectable Change (MDC) belonging to the measurements have been obtained. To compute absolute accuracy of the method, a goniometer marked with colors has been recorded at four different angles (45, 90, 135, and 180° ) by Kinect in six sessions having 50-frame periods each. Mean, Standard Deviation (SD), Root Mean Square Error (RMSE), and Limits of Agreement (LOA) values are given for each angle and session. The measurements taken for absolute accuracy clearly shows that Kinect has 1- to 3-degree error rate and below 1-degree standard deviation. Analyzing the collected data, the ICC values of Kinect measurements that are 0.94 for right arm and 0.93 for left arm in contrast with the ICC values of goniometric measurements taken by observers are 0.78 for the right arm and 0.81 for the left arm. This study indicates the proposed method has a high level of accuracy and reliability, and it can be efficiently used to measure ROM accurately.

Hareket aralığı ölçümü fizik tedavinin ilk aşamasını oluşturmaktadır. Bu çalışmada, derinlik bilgisi veren bir kamera türü olan Kinect V2 kullanılarak yeni bir hareket aralığı ölçüm yöntemi önerilmiştir. İlgili uzva renkli işaretçiler yapıştırılıp bu işaretçilerin her birinin ağırlık merkezine ait kamera merkezli üç boyutlu dünya koordinatları bulunmuştur. Bu koordinatlar kullanılarak eklem açıları ve hareket aralığı ölçülmüştür. Yöntemin geçerlilik ve güvenirliğini test etmek amacıyla 10 katılımcının sağ ve sol dirsek açıları standart gönyometre ve Kinect ile ayrı ayrı ölçülmüştür. Gözlemci içi güvenirliklerin test edilmesi için ölçümler üç oturumda her biri en az 10 yıl tecrübeli üç fizik tedavi uzmanı ve Kinect ile alınmıştır. Güvenirlik analizlerinde ölçümlere ait sınıf içi korelasyon katsayısı (ICC), Ölçüm standart hatası (SEM) ve tespit edilebilir minimal değişim (MDC) hesaplanmıştır. Cihaz ile yapılan ölçümlerin mutlak doğruluğunu gözlemlemek için gönyometre üzerine işaretçiler yapıştırılıp dört farklı açıya (45, 90, 135 ve 180°) ayarlanarak altışar oturumda ölçüm alınmıştır. Her bir açı ve oturum için ölçümlere ait ortalama, standart sapma, ortalama karesel hata (RMSE) ve karar sınırları (LOA) bulunmuştur. Mutlak doğruluk için yapılan ölçümlerde kullanılan yöntemin 1-3° hata payı ve 1° altında standart sapması olduğu görülmüştür. Fizik tedavi uzmanlarının yaptığı ölçümlerde sağ ve sol kol için sınıf içi korelasyon katsayıları sırasıyla 0.78 ve 0.81 olarak bulunurken bu değerler Kinect için 0.94 ve 0.93 olarak elde edilmiştir. Bu çalışmada önerilen yöntemin yapılan analizler sonrası geçerli ve güvenilir olduğu anlaşılıp klinik uygulamalarda kullanılabileceği görülmüştür.

Recent technological advances in computer vision and the advent of commercial RGB-D sensors have certainly boosted 3D modeling applications. This work presents an integrated system that enables the digitization of big objects, like vehicles, with low-cost RGB-D sensors. The implemented system can be used for visualization and monitoring of vehicles in large fleets that currently require a time-consuming manual inspection process. The main objective is to achieve an efficient consolidation of multiple views of a vehicle inside a moving frame, to acquire color and depth data and generate its 3D representation. The proposed integrated system denoises the acquired depth maps, aligns the produced point clouds captured in different time instances, and builds the 3D-reconstructed mesh. Finally, we apply a texture mapping algorithm to acquire realistic texture details and remove any visible seams. We evaluate all modules of the implemented system by performing several experiments with scanned vehicles. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel Grassmann manifold based framework for dynamic hand gesture recognition from depth data. Automated dynamic hand gesture recognition is important for improving man-machine communication and understanding human behavior. It finds various applications such as human computer interaction, ambient assisted living, automated driver assisted systems. We use depth data or skeleton information to detect the fingertip and store the fingertip points to create the trajectory. In fingertip detection using depth data first we detect the hand using the depth data and used hand shape properties such as finger thickness, finger length, finger width and finger orientation angle to find the shape of the hand. If skeleton data is available we use skeleton information to detect the fingertip in each frame. Then geometrical features are extracted and a unique gesture subspaces created using SVD for each feature vector matrix of each gesture set. These gesture subspaces lie on a Grassmann manifold and capture the intra-class variations and increase the inter-class discriminatory power. We apply Grassmann manifold based discriminant analysis for recognizing each test gesture. We perform experiments on standard datasets and the results show that we have achieved recognition accuracy comparable to the state-of-the-art. [ABSTRACT FROM AUTHOR]

Person re-identification (reID) is a crucial aspect of intelligent surveillance systems, enabling the recognition of individuals across non-overlapping camera views. As compared to the RGB modality, RGB-D based reID has the potential to achieve robust and high performance by leveraging rich complementary features of both modalities, making it applicable in various occluded scenarios. However, current multimodal reID approaches often rely on late fusion or feature-level fusion techniques to combine multiple modalities, which limits their ability to proficiently exploit complementary visual and depth-related semantic information and capture complex interactions between unimodal features. To address these limitations, this paper introduces a cross-modality online transformer (CMOT) for RGB-D based person reID with online learning capabilities, which effectively utilizes both RGB and depth modalities for the extraction of spatio-temporal features and fuses across modalities. Our CMOT is composed of three main components: (1) a hypothesis generation module based on a person detector and tracker, (2) dual-stream feature extractors via convolutional neural networks (CNNs), (3) and a fusion transformer based on a self-attention-driven self-attentive modality refinement module (SAMR) and a cross-attention-driven cross-attentive modality interaction module (CAMI) to refine and fuse RGB-D complementary features extracted from the dual-stream, RGB and depth stream, CNNs. Additionally, we introduce a bottleneck enhancement feed-forward block to enhance the model's representation capability within SAMR and CAMI, significantly reducing parameters and computations compared to the traditional feed-forward network. Moreover, we design the triplet loss function with distance measuring ability for incorporating online learning and finally CMOT works as a few-shot network for reID. Experimental results on three RGB-D person re-identification datasets, namely BIWI RGBD-ID, RobotPKU RGBD-ID, and TVPR2, demonstrate the effectiveness and robustness of CMOT. • Proposes a dual-stream pipeline for RGB-depth modality, for person re-identification. • Proposes a Transformer for the effective fusion using self- and Cross-attention. • Online learning ability of person reidentification Transformer via distance learning. [ABSTRACT FROM AUTHOR]

Stroke is one of the most significant causes of permanent functional impairment and severe motor disability. Hemiplegia or hemiparesis are common consequences of the acute event, which negatively impacts daily life and requires continuous rehabilitation treatments to favor partial or complete recovery and, consequently, to regain autonomy, independence, and safety in daily activities. Gait impairments are frequent in stroke survivors. The accurate assessment of gait anomalies is therefore crucial and a major focus of neurorehabilitation programs to prevent falls or injuries. This study aims to estimate, using a single RGB-D sensor, gait patterns and parameters on a short walkway. This solution may be suitable for monitoring the improvement or worsening of gait disorders, including in domestic and unsupervised scenarios. For this purpose, some of the most relevant spatiotemporal parameters, estimated by the proposed solution on a cohort of post-stroke individuals, were compared with those estimated by a gold standard system for a simultaneous instrumented 3D gait analysis. Preliminary results indicate good agreement, accuracy, and correlation between the gait parameters estimated by the two systems. This suggests that the proposed solution may be employed as an intermediate tool for gait analysis in environments where gold standard systems are impractical, such as home and ecological settings in real-life contexts.

Human interaction recognition has played a major role in building intelligent video surveillance systems. Recently, depth data captured by the emerging RGB‐D sensors began to show its importability in human interaction recognition. This study proposes a novel framework for human interaction recognition using depth information including an algorithm to reconstruct depth sequence with as few key frames as possible. The proposed framework includes two essential modules. First, key frames extraction by sparse constraint, then the fusion multi‐feature, is constructed by using two types of available features and Max‐pooling, respectively. Finally, multiple features are directly sent to the SVM for the recognition of the human activity. This study explores the static and dynamic feature fusion method to improve the recognition performance with contextual relevance of continuous frames. A weight is used to fuse shape and optical flow features, which not only enhance the description capability of human behavioural characteristics in the spatiotemporal domain, but also effectively reduces the adverse impact of certain distortion point of interest for target recognition. Experimental results show that the proposed approach yields considerable performance improvement over the state‐of‐the‐art approaches with respect to accuracy on a public action dataset.

Future home and service robots will require advanced navigation and interaction capabilities. In particular, domestic environments present open challenges that are hard to identify by conducting controlled tests in home-like settings: there is the need to test and evaluate navigation in the actual homes of users. This paper presents the experiences of operating a mobile robot with manipulation capabilities and an open set of tasks in extensive trials with real users, in their own homes. The main difficulties encountered are the requirement to move safely in cluttered 3D environments, the problems related to navigation in narrow spaces, and the need for an adaptive rather than fixed way to approach the users. We describe our solutions based on RGB-D perception and evaluate the integrated system for navigation in real home environments, pointing out remaining challenges towards more advanced commercial solutions. [ABSTRACT FROM AUTHOR]

Abstract Mobile robots need to know their position in the environment to perform complex tasks. Self-localization methods for indoor environments usually reduce sensory information to two dimensions in order to accelerate calculations, instead of taking advantage of the three-dimensional (3D) features. This paper proposes a computationally affordable probabilistic self-localization method that takes full advantage of the capacities of RGB-D sensors, particularly, using 3D dimensions and color. We use an a priori known map, previously built, that keeps the information of the environment's structure and color. Storing this data allows to reduce symmetries caused by 2D and colorless simplification. Our approach is based on Monte Carlo algorithm, where hypothesis population varies dynamically to adapt itself to the existing conditions at any given time, which makes it more efficient in 3D information management. [ABSTRACT FROM AUTHOR]

Computer vision for control is a growing domain of research and it is widespread in industry and the autonomous vehicle field. A further step is the employment of low-cost cameras to perform these applications. To apply such an approach, the development of proper algorithms to interpret vision data is mandatory. Here, we firstly propose the development of an algorithm to measure the displacement of a mechanical system in contactless mode. Afterwards, we show two procedures that use a 3D camera as a feedback in control strategies. The first one aims to track a moving object. In the second one, the information gained from vision data acquisition allows the mechanical system control to ensure the equilibrium of a ball placed on a moving slide.

In physical rehabilitation, motion capture solutions are well-known but not as widespread as they could be. The main limit to their diffusion is not related to cost or usability but to the fact that the data generated when tracking a person must be elaborated according to the specific context and aim. This paper proposes a solution including customized motion capture and data elaboration with the aim of supporting medical personnel in the assessment of spinal cord-injured (SCI) patients using a wheelchair. The configuration of the full-body motion capturing system is based on an asymmetric 3 Microsoft Kinect v2 sensor layout that provides a path of up to 6 m, which is required to properly track the wheelchair. Data elaboration is focused on the automatic recognition of the pushing cycles and on plotting any kinematic parameter that may be interesting in the assessment. Five movements have been considered to evaluate the wheelchair propulsion: the humeral elevation, the horizontal abduction of the humerus, the humeral rotation, the elbow flexion and the trunk extension along the sagittal plane. More than 60 volunteers with a spinal cord injury were enrolled for testing the solution. To evaluate the reliability of the data computed with SCI APPlication (APP) for the pushing cycle analysis, the patients were subdivided in four groups according to the level of the spinal cord injury (i.e., high paraplegia, low paraplegia, C7 tetraplegia and C6 tetraplegia). For each group, the average value and the standard deviation were computed and a comparison with similar acquisitions performed with a high-end solution is shown. The measurements computed by the SCI-APP show a good reliability for analyzing the movements of SCI patients’ propulsion wheelchair.

This research aims to improve dietetic-nutritional treatment using state-of-the-art RGB-D sensors and virtual reality (VR) technology. Recent studies show that adherence to treatment can be improved using multimedia technologies. However, there are few studies using 3D data and VR technologies for this purpose. On the other hand, obtaining 3D measurements of the human body and analyzing them over time (4D) in patients undergoing dietary treatment is a challenging field. The main contribution of the work is to provide a framework to study the effect of 4D body model visualization on adherence to obesity treatment. The system can obtain a complete 3D model of a body using low-cost technology, allowing future straightforward transference with sufficient accuracy and realistic visualization, enabling the analysis of the evolution (4D) of the shape during the treatment of obesity. The 3D body models will be used for studying the effect of visualization on adherence to obesity treatment using 2D and VR devices. Moreover, we will use the acquired 3D models to obtain measurements of the body. An analysis of the accuracy of the proposed methods for obtaining measurements with both synthetic and real objects has been carried out.

State-of-the-art intelligent versatile applications provoke the usage of full 3D, depth-based streams, especially in the scenarios of intelligent remote control and communications, where virtual and augmented reality will soon become outdated and are forecasted to be replaced by point cloud streams providing explorable 3D environments of communication and industrial data. One of the most novel approaches employed in modern object reconstruction methods is to use a priori knowledge of the objects that are being reconstructed. Our approach is different as we strive to reconstruct a 3D object within much more difficult scenarios of limited data availability. Data stream is often limited by insufficient depth camera coverage and, as a result, the objects are occluded and data is lost. Our proposed hybrid artificial neural network modifications have improved the reconstruction results by 8.53% which allows us for much more precise filling of occluded object sides and reduction of noise during the process. Furthermore, the addition of object segmentation masks and the individual object instance classification is a leap forward towards a general-purpose scene reconstruction as opposed to a single object reconstruction task due to the ability to mask out overlapping object instances and using only masked object area in the reconstruction process.

To generate indoor as-built building information models (AB BIMs) automatically and economically is a great technological challenge. Many approaches have been developed to address this problem in recent years, but it is far from being settled, particularly for the point cloud segmentation and the extraction of the relationship among different elements due to the complicated indoor environment. This is even more difficult for the low-quality point cloud generated by low-cost scanning equipment. This paper proposes an automatic as-built BIMs generation framework that transforms the noisy 3D point cloud produced by a low-cost RGB-D sensor (about 708 USD for data collection equipment, 379 USD for the Structure sensor and 329 USD for iPad) to the as-built BIMs, without any manual intervention. The experiment results show that the proposed method has competitive robustness and accuracy, compared to the high-quality Terrestrial Lidar System (TLS), with the element extraction accuracy of 100%, mean dimension reconstruction accuracy of 98.6% and mean area reconstruction accuracy of 93.6%. Also, the proposed framework makes the BIM generation workflows more efficient in both data collection and data processing. In the experiments, the time consumption of data collection for a typical room, with an area of 45−67 m 2 , is reduced to 4−6 min with an RGB-D sensor from 50−60 min with TLS. The processing time to generate BIM models is about half minutes automatically, from around 10 min with a conventional semi-manual method.

Abstract Active research has been carried out for human action recognition using 3D human skeleton joints with the release of cost-efficient RGB-D sensors. However, extracting discriminative features from noisy skeleton sequences to effectively distinguish various human action or interaction categories still remains challenging. This paper proposes a structured multi-feature representation for human action and interaction recognition. Specifically, a novel kernel enhanced bag of semantic words (BSW) is designed to represent the dynamic property of skeleton trajectories. By aggregating BSW with the geometric feature, a GBSW representation is constructed for human action recognition. For human interaction recognition where the cooperation of each subject matters, a GBSWC representation is proposed via combining the GBSW feature with a correlation feature which addresses the intrinsic relationship between interactive persons. Experimental results on several human action and interaction datasets demonstrate the superior performances of the proposed features over the state-of-the-art methods. [ABSTRACT FROM AUTHOR]