Search

Human–robot walking with prosthetic legs and exoskeletons, especially over complex terrains, such as stairs, remains a significant challenge. Egocentric vision has the unique potential to detect the walking environment prior to physical interactions, which can improve transitions to and from stairs. This motivated us to develop the StairNet initiative to support the development of new deep learning models for visual perception of real-world stair environments. In this study, we present a comprehensive overview of the StairNet initiative and key research to date. First, we summarize the development of our large-scale data set with over 515,000 manually labeled images. We then provide a summary and detailed comparison of the performances achieved with different algorithms (i.e., 2D and 3D CNN, hybrid CNN and LSTM, and ViT networks), training methods (i.e., supervised learning with and without temporal data, and semi-supervised learning with unlabeled images), and deployment methods (i.e., mobile and embedded computing), using the StairNet data set. Finally, we discuss the challenges and future directions. To date, our StairNet models have consistently achieved high classification accuracy (i.e., up to 98.8%) with different designs, offering trade-offs between model accuracy and size. When deployed on mobile devices with GPU and NPU accelerators, our deep learning models achieved inference speeds up to 2.8 ms. In comparison, when deployed on our custom-designed CPU-powered smart glasses, our models yielded slower inference speeds of 1.5 s, presenting a trade-off between human-centered design and performance. Overall, the results of numerous experiments presented herein provide consistent evidence that StairNet can be an effective platform to develop and study new deep learning models for visual perception of human–robot walking environments, with an emphasis on stair recognition. This research aims to support the development of next-generation vision-based control syst

Human-robot walking with prosthetic legs and exoskeletons, especially over complex terrains such as stairs, remains a significant challenge. Egocentric vision has the unique potential to detect the walking environment prior to physical interactions, which can improve transitions to and from stairs. This motivated us to create the StairNet initiative to support the development of new deep learning models for visual sensing and recognition of stairs, with an emphasis on lightweight and efficient neural networks for onboard real-time inference. In this study, we present an overview of the development of our large-scale dataset with over 515,000 manually labeled images, as well as our development of different deep learning models (e.g., 2D and 3D CNN, hybrid CNN and LSTM, and ViT networks) and training methods (e.g., supervised learning with temporal data and semi-supervised learning with unlabeled images) using our new dataset. We consistently achieved high classification accuracy (i.e., up to 98.8%) with different designs, offering trade-offs between model accuracy and size. When deployed on mobile devices with GPU and NPU accelerators, our deep learning models achieved inference speeds up to 2.8 ms. We also deployed our models on custom-designed CPU-powered smart glasses. However, limitations in the embedded hardware yielded slower inference speeds of 1.5 seconds, presenting a trade-off between human-centered design and performance. Overall, we showed that StairNet can be an effective platform to develop and study new visual perception systems for human-robot locomotion with applications in exoskeleton and prosthetic leg control.

Experiments prompting people to engage in more prosocial behavior (e.g., acts of kindness) or simple social interactions (e.g., acting extraverted) have both shown promise in boosting well-being. However, little is known about how much the impact on well-being depends on the type of interpersonal interaction (i.e., social versus prosocial) or on other proximal features (e.g., whether the interaction takes place online versus in-person, the closeness of the relationship, or in-the-moment social connection). We randomly assigned a sample of full-time employees recruited via mTurk (N = 754) to perform weekly acts of kindness online vs. in person, to engage in weekly social interactions online vs. in person, or to list their daily activities (control) over the course of 4 weeks. First, on average, all conditions reported improvements in well-being (i.e., increases in positive affect and life satisfaction, decreases in negative affect) across the 4-week intervention period. Second, relative to controls, the four experimental groups reported increases in general social connectedness over time. Finally, according to auxiliary analyses collapsed across experimental condition, closer relationship with target and non-digital medium of delivery predicted in-the-moment (or episode-level) social connection, which, in turn, was associated with general social connectedness and positive affect. We conclude that the “who” and the “how” of a behavior (i.e., its target, its delivery method, and the feelings of social connection generated) may be as important for well-being as the “what” (i.e., whether the behavior is social or prosocial).