Search

Most of the humanoid social robots currently diffused are designed only for verbal and animated interactions with users, and despite being equipped with two upper arms for interactive animation, they lack object manipulation capabilities. In this paper, we propose the MONOCULAR (eMbeddable autONomous ObjeCt manipULAtion Routines) framework, which implements a set of routines to add manipulation functionalities to social robots by exploiting the functional data fusion of two RGB cameras and a 3D depth sensor placed in the head frame. The framework is designed to: (i) localize specific objects to be manipulated via RGB cameras; (ii) define the characteristics of the shelf on which they are placed; and (iii) autonomously adapt approach and manipulation routines to avoid collisions and maximize grabbing accuracy. To localize the item on the shelf, MONOCULAR exploits an embeddable version of the You Only Look Once (YOLO) object detector. The RGB camera outcomes are also used to estimate the height of the shelf using an edge-detecting algorithm. Based on the item's position and the estimated shelf height, MONOCULAR is designed to select between two possible routines that dynamically optimize the approach and object manipulation parameters according to the real-time analysis of RGB and 3D sensor frames. These two routines are optimized for a central or lateral approach to objects on a shelf. The MONOCULAR procedures are designed to be fully automatic, intrinsically protecting sensitive users' data and stored home or hospital maps. MONOCULAR was optimized for Pepper by SoftBank Robotics. To characterize the proposed system, a case study in which Pepper is used as a drug delivery operator is proposed. The case study is divided into: (i) pharmaceutical package search; (ii) object approach and manipulation; and (iii) delivery operations. Experimental data showed that object manipulation routines for laterally placed objects achieves a best grabbing success rate of 96%, while the routine for centrally placed objects can reach 97% for a wide range of different shelf heights. Finally, a proof of concept is proposed here to demonstrate the applicability of the MONOCULAR framework in a real-life scenario.

Levodopa administration is currently the most common treatment to alleviate Parkinson's Disease (PD) symptoms. Nevertheless, prolonged use of Levodopa leads to a wearing-off (WO) phenomenon, causing symptoms to reappear. To build a personalized treatment plan aiming to manage PD and its symptoms effectively, there is a need for a technological system able to continuously and objectively assess the WO phenomenon during daily life. In this context, this paper proposes a WO tracker able to exploit neuromuscular data acquired by a dedicated wireless sensor network to discriminate between a Levodopa benefit phase and the reappearance of symptoms. The proposed architecture has been implemented on a heterogeneous computing platform, that statistically analyzes neural and muscular features to identify the best set of features to train the classifier model. Eight models among shallow and deep learning approaches are analyzed in terms of performance, timing and complexity metrics to identify the best inference engine. Experimental results on five subjects experiencing WO, showed that, in the best case, the proposed WO tracker can achieve an accuracy of ~84%, providing the inference in less than 41 ms. It is possible by employing a simple fully-connected neural network with 1 hidden layer and 32 units.