Search

Highlights • A socially-aware assistant, consisting of both perceptive and feedback components. • Our approach is based on a wearable platform. • Our system has been tested with visually impaired people. Abstract Visual impairment affects the normal course of activities in everyday life including mobility, education, employment, and social interaction. Most of the existing technical solutions devoted to empowering the visually impaired people are in the areas of navigation (obstacle avoidance), access to printed information and object recognition. Less effort has been dedicated so far in developing solutions to support social interactions. In this paper, we introduce a Social-Aware Assistant (SAA) that provides visually impaired people with cues to enhance their face-to-face conversations. The system consists of a perceptive component (represented by smartglasses with an embedded video camera) and a feedback component (represented by a haptic belt). When the vision system detects a head nodding, the belt vibrates, thus suggesting the user to replicate (mirror) the gesture. In our experiments, sighted persons interacted with blind people wearing the SAA. We instructed the former to mirror the noddings according to the vibratory signal, while the latter interacted naturally. After the face-to-face conversation, the participants had an interview to express their experience regarding the use of this new technological assistant. With the data collected during the experiment, we have assessed quantitatively and qualitatively the device usefulness and user satisfaction. [ABSTRACT FROM AUTHOR]

This paper introduces a tool for calibrating multiple Kinect V2 sensors. To achieve the calibration, at least three acquisitions are needed from each camera. The method uses the Kinect's coordinate mapping capabilities between the sensors to register data between camera, depth, and color spaces. The method uses a novel approach where it obtains multiple 3D points matches between adjacent sensors, and use them to estimating the camera parameters. Once the cameras are calibrated, the tool can perform point cloud fusion transforming all the 3D points to a single reference. We tested the system with a network of four Kinect V2 sensors and present calibration results. The tool is implemented in MATLAB using the Kinect for Windows SDK 2.0. [ABSTRACT FROM AUTHOR]

This paper introduces Kin2 , a Kinect 2 toolbox for MATLAB. This toolbox encapsulates most of the Kinect for Windows SDK 2.0 functionality in a single class with high-level methods. The toolbox is written mostly in C++ with MATLAB Mex functions providing access to color, depth, infrared, and body index frames; coordinate mapping capabilities; real-time six-body tracking with 25 joints and hands states; face and high-definition face processing; and real-time 3D reconstruction. We showed that the performance of a Kin2 application decreases 30% on average with respect to a native C++ application, however, Kin reduces in one order of magnitude the average code length yielding a significant reduction in development time for prototyping and research. [ABSTRACT FROM AUTHOR]

During face-to-face human interaction, nonverbal communication plays a fundamental role. A relevant aspect that takes part during social interactions is represented by mirroring, in which a person tends to mimic the non-verbal behavior (head and body gestures, vocal prosody, etc.) of the counterpart. In this paper, we introduce a computer vision-based system to detect mirroring in dyadic social interactions with the use of a wearable platform. In our context, mirroring is inferred as simultaneous head noddings displayed by the interlocutors. Our approach consists of the following steps: (1) facial features extraction; (2) facial features stabilization; (3) head nodding recognition; and (4) mirroring detection. Our system achieves a mirroring detection accuracy of 72% on a custom mirroring dataset. [ABSTRACT FROM AUTHOR]

Computing advances and increased smartphone use gives technology system designers greater flexibility in exploiting computer vision to support visually impaired users. Understanding these users' needs will certainly provide insight for the development of improved usability of computing devices. [ABSTRACT FROM PUBLISHER]

• Azure Kinect library for Python and Matlab. • Easy-to-use high-level functions to access Azure Kinect functionality. • Additional 32 hand keypoints to the existing 32 body keytpoints for tracking. This paper introduces KinZ , an Azure Kinect toolkit for Python and Matlab. This toolkit encapsulates the Azure Kinect's main functionality in an easy-to-use class with high-level methods. This software is written mostly in C/C++ with a pybind interface for Python and Mex functions for Matlab. It provides access to color, depth, infrared cameras, a mapping between cameras, colored point clouds, IMU sensor data, body tracking with 32 body joints plus 32 hand keypoints. We showed that KinZ applications could run in real-time at 1080p resolution, with the benefit of a high-level language reducing the learning curve and allowing a significant decrease in development time for prototyping and research. [ABSTRACT FROM AUTHOR]

Owing to their large field of view, a catadioptric imaging system is widely used in several research areas, including robot navigation, surveillance, and three‐dimensional (3D) reconstruction. The modelling and calibration of these systems allow the accurate measurement of objects. This paper presents a calibration method for a panoramic 3D reconstruction system based on pattern projection composed of two modules. The first module, which contains a charged coupled device camera and a parabolic mirror allows the acquisition of catadioptric images. The second module is a catadioptric projection system made of a light projector and a parabolic mirror that generates a projected pattern over the target for reconstruction purposes. Both the modules are calibrated independently. The method achieves a reprojection error of 0.6 pixels in the camera module and 1.2 pixels in the catadioptric projection system. To evaluate the performance of the proposed method, the results show 3D reconstructions of multiple objects with tolerances around 1.3 mm. [ABSTRACT FROM AUTHOR]