Search

This paper showcases a developed end-user framework for a human-robot collaborative system for common tasks, such as pick and place. The system is designed for semi-automated pick and place tasks as well as manual operation making it flexible for multiple use-case scenarios. The goal of the system is to make the robot multi-functional, easy to use with a graphical user interface and to perform common tasks with the help of a human teammate. Integration with object recognition neural network (YoloV3) and an RGB-Depth camera helps automate pick and place tasks with a wide variety of objects.

This paper presents a framework for developing Intelligent, Interactive, and Immersive Storytelling systems for vocational training by improving task performance. We present a systematic framework that can be used to personalize training for a worker in a factory environment. We also present a system implementation that builds upon the vIIIS framework and also describes the design decisions made throughout the system. In this paper, we focus on improving a user’s episodic and working memory by employing picture sequence and object sorting tasks taken from the NIH toolbox and presenting an intelligent Augmented Reality system. A major advantage of using an intelligent and interactive system using storytelling is that the training can be curated towards each specific user, thereby enhancing the learning outcome based on individual needs.

Cognitive assessments are a crucial part of rehabilitation in persons with a neurological disorder and vocational rehabilitation, where people need to be trained to improve their cognitive abilities. While human action involves using several cognitive skills and physical skills, most assessment systems focus on detecting or assessing either the cognitive ability or just physical ability. There is a need for a system that bridges the gap between real-world activity, which involves physical activity and cognition, and clinical tests that are tailored for a specific use. To address this need, we propose a novel interactive 9-Hole Pegboard called the 9-Peg Move (9PM) capable of performing both cognitive and physical assessments in the same system. The system incorporates wearable sensors to collect data for objective evaluation. Preliminary machine learning results indicate that the data collected using our system can reliably recognize cognitive factors like perceived mental effort, perceived task difficulty, and perceived interest in a task. These results are the first step toward building an automated immersive assessment system.

Working memory and episodic memory are mainly responsible for the storage and recollection of information, whereas Storytelling is the most ancient and effective way of relaying this information to the user. We have designed an interactive and immersive storytelling system using Augmented Reality to improve episodic and working memory. This paper presents an overview of the tasks we used to improve these 2 sections of memory and also presents a study design on how we plan to evaluate our system to prove its effectiveness in comparison to desktop 2D-based training.

Nurses have to carry out a myriad of tasks, which is one of the main reasons for burnout. In this paper, a robotic Multitasking Intelligent Nurse Aid (MINA) is proposed to assist nurses with everyday tasks and tackle nurse burnout. MINA uses Simultaneous Localization And Mapping (SLAM) to map shelves in a storage room and pointcloud using an RGB-Depth camera. The barcode of the item is first detected using zbar library, and then the pointcloud is generated in ROS and saved accordingly. Preliminary experiments with respect to the creation of a pointcloud map with barcode detection are presented. The preliminary results reveal that the system shows a 100% barcode detection rate, and with each barcode detection a corresponding pointcloud map is generated and saved automatically without any intervention.

Manifolk is a tool to visualize the output of dimensionality reduction algorithms like t-SNE, PCA etc. One of this tool’s main uses is that it de-clutters graphs by plotting data points pertaining to a subset of labels. The subset of labels to be plotted can be selected using the provided checkboxes. A case study on data from a publicly available action recognition dataset like UCF101 shows how this tool can help find outliers. With the rise in self-supervised methods for training deep neural networks, this tool helps researchers better visualize the embeddings learned by the model.