Search

Image-based deformation estimation is an important tool used in a variety of engineering problems, including crack propagation, fracture, and fatigue failure. These tools have been important in biomechanics research where measuring in vitro and in vivo tissue deformations are important for evaluating tissue health and disease progression. However, accurately measuring tissue deformation in vivo is particularly challenging due to limited image signal-to-noise ratio. Therefore, we created a novel deep-learning approach for measuring deformation from a sequence of images collected in vivo called StrainNet. Utilizing a training dataset that incorporates image artifacts, StrainNet was designed to maximize performance in challenging, in vivo settings. Artificially generated image sequences of human flexor tendons undergoing known deformations were used to compare benchmark StrainNet against two conventional image-based strain measurement techniques. StrainNet outperformed the traditional techniques by nearly 90%. High-frequency ultrasound imaging was then used to acquire images of the flexor tendons engaged during contraction. Only StrainNet was able to track tissue deformations under the in vivo test conditions. Findings revealed strong correlations between tendon deformation and applied forces, highlighting the potential for StrainNet to be a valuable tool for assessing rehabilitation strategies or disease progression. Additionally, by using real-world data to train our model, StrainNet was able to generalize and reveal important relationships between the effort exerted by the participant and tendon mechanics. Overall, StrainNet demonstrated the effectiveness of using deep learning for image-based strain analysis in vivo.

Non-technical skills (NTS) challenges experienced by surgeons may degrade performance, ultimately impacting the safety and quality of care delivered to patients. The objectives of this work were to develop a framework for NTS coaching for surgeons and implement a coaching program utilizing the developed NTS coaching framework. Leveraging adult learning and self-determination theories, a specialty-agnostic NTS coaching framework was developed for individual coaching sessions with robotic surgeons. The framework was used to deliver NTS coaching sessions to robotic surgeons. Surgeon's robotic procedures were recorded, and expert raters assessed their NTS using the Non-Technical Skills for Surgeons tool. Measures of surgeon satisfaction, learning outcomes, and performance improvement were determined. Cohen's d statistic was used to estimate the effect size of the coaching intervention. To pilot the program, ten robotic-assisted surgeries (five pre-coaching and five post-coaching) were observed from five practicing robotic surgeons who were recruited from a large academic healthcare system. Expert raters' assessment of surgeons' NTS revealed several exemplar and non-exemplar behaviors. Surgeons were satisfied with the coaching, rating its quality very highly on all NTS dimensions. On a Likert scale of 1 (strongly disagree) to 5 (strongly agree), surgeons had a stronger agreement that the coaching could improve their situation awareness (4.0±.5) and leadership (4.8±.2) skills compared to their decision making and communication (3.8±.7). From the post-coaching observations, coaching had medium-to-large effect on situation awareness (d =.65) and leadership (d =.41), small effect on communication and teamwork (d =.14), and no effect on decision-making. Overall, the coaching intervention had a medium effect on total NTS (d =.33). We presented a quality improvement initiative to enhance the NTS of surgeons by implementing a coaching program that leverages our developed NTS framework. Recognizing the importance of NTS in surgeries, our initiative shows a commitment to continuous improvement of patient safety and quality of care. [ABSTRACT FROM AUTHOR]

Objective: This study developed a fixation-related electroencephalography band power (FRBP) approach for situation awareness (SA) assessment in automated driving. Background: Maintaining good SA in Level 3 automated vehicles is crucial to drivers' takeover performance when the automated system fails. A multimodal fusion approach that enables the analysis of the visual behavioral and cognitive processes of SA can facilitate real-time assessment of SA in future driver state monitoring systems. Method: Thirty participants performed three simulated automated driving tasks. After each task, the Situation Awareness Global Assessment Technique (SAGAT) was deployed to capture their SA about key elements that could affect their takeover task performance. Participants eye movements and brain activities were recorded. Data on their brain activity after each eye fixation on the key elements were extracted and labeled according to the correctness of the SAGAT. Mixed-effects models were used to identify brain regions that were indicative of SA, and machine learning models for SA assessment were developed based on the identified brain regions. Results: Participants' alpha and theta oscillation at frontal and temporal areas are indicative of SA. In addition, the FRBP technique can be used to predict drivers' SA with an accuracy of 88% using a neural network model. Conclusion: The FRBP technique, which incorporates eye movements and brain activities, can provide more comprehensive evaluation of SA. Findings highlight the potential of utilizing FRBP to monitor drivers' SA in real-time. Application: The proposed framework can be expanded and applied to driver state monitoring systems to measure human SA in real-world driving. [ABSTRACT FROM AUTHOR]

Objective: This study developed and evaluated a mental workload-based adaptive automation (MWL-AA) that monitors surgeon cognitive load and assist during cognitively demanding tasks and assists surgeons in robotic-assisted surgery (RAS). Background: The introduction of RAS makes operators overwhelmed. The need for precise, continuous assessment of human mental workload (MWL) states is important to identify when the interventions should be delivered to moderate operators' MWL. Method: The MWL-AA presented in this study was a semi-autonomous suction tool. The first experiment recruited ten participants to perform surgical tasks under different MWL levels. The physiological responses were captured and used to develop a real-time multi-sensing model for MWL detection. The second experiment evaluated the effectiveness of the MWL-AA, where nine brand-new surgical trainees performed the surgical task with and without the MWL-AA. Mixed effect models were used to compare task performance, objective- and subjective-measured MWL. Results: The proposed system predicted high MWL hemorrhage conditions with an accuracy of 77.9%. For the MWL-AA evaluation, the surgeons' gaze behaviors and brain activities suggested lower perceived MWL with MWL-AA than without. This was further supported by lower self-reported MWL and better task performance in the task condition with MWL-AA. Conclusion: A MWL-AA systems can reduce surgeons' workload and improve performance in a high-stress hemorrhaging scenario. Findings highlight the potential of utilizing MWL-AA to enhance the collaboration between the autonomous system and surgeons. Developing a robust and personalized MWL-AA is the first step that can be used do develop additional use cases in future studies. Application: The proposed framework can be expanded and applied to more complex environments to improve human-robot collaboration. [ABSTRACT FROM AUTHOR]

Objective: The purpose of this study was to identify objective measures that predict surgeon nontechnical skills (NTS) during surgery. Background: NTS are cognitive and social skills that impact operative performance and patient outcomes. Current methods for NTS assessment in surgery rely on observation-based tools to rate intraoperative behavior. These tools are resource intensive (e.g., time for observation or manual labeling) to perform; therefore, more efficient approaches are needed. Method: Thirty-four robotic-assisted surgeries were observed. Proximity sensors were placed on the surgical team and voice recorders were placed on the surgeon. Surgeon NTS was assessed by trained observers using the NonTechnical Skills for Surgeons (NOTSS) tool. NTS behavior metrics from the sensors included communication, speech, and proximity features. The metrics were used to develop mixed effect models to predict NOTSS score and in machine learning classifiers to distinguish between exemplar NTS scores (highest NOTSS score) and non-exemplar scores. Results: NTS metrics were collected from 16 nurses, 12 assistants, 11 anesthesiologists, and four surgeons. Nineteen behavior features and overall NOTSS score were significantly correlated (12 communication features, two speech features, five proximity features). The random forest classifier achieved the highest accuracy of 70% (80% F1 score) to predict exemplar NTS score. Conclusion: Sensor-based measures of communication, speech, and proximity can potentially predict NOTSS scores of surgeons during robotic-assisted surgery. These sensing-based approaches can be utilized for further reducing resource costs of NTS and team performance assessment in surgical environments. Application: Sensor-based assessment of operative teams' behaviors can lead to objective, real-time NTS measurement. [ABSTRACT FROM AUTHOR]

Objective: (1) To assess mental workloads of intensive care unit (ICU) nurses in 12-hour working shifts (days and nights) using eye movement data; (2) to explore the impact of stress on the ocular metrics of nurses performing patient care in the ICU. Background: Prior studies have employed workload scoring systems or accelerometer data to assess ICU nurses' workload. This is the first naturalistic attempt to explore nurses' mental workload using eye movement data. Methods: Tobii Pro Glasses 2 eye-tracking and Empatica E4 devices were used to collect eye movement and physiological data from 15 nurses during 12-hour shifts (252 observation hours). We used mixed-effect models and an ordinal regression model with a random effect to analyze the changes in eye movement metrics during high stress episodes. Results: While the cadence and characteristics of nurse workload can vary between day shift and night shift, no significant difference in eye movement values was detected. However, eye movement metrics showed that the initial handoff period of nursing shifts has a higher mental workload compared with other times. Analysis of ocular metrics showed that stress is positively associated with an increase in number of eye fixations and gaze entropy, but negatively correlated with the duration of saccades and pupil diameter. Conclusion: Eye-tracking technology can be used to assess the temporal variation of stress and associated changes with mental workload in the ICU environment. A real-time system could be developed for monitoring stress and workload for intervention development. [ABSTRACT FROM AUTHOR]

Objective: The goal of this systematic literature review is to investigate the relationship between indirect physiological measurements and direct measures of situation awareness (SA). Background: Across different environments and tasks, assessments of SA are often performed using techniques designed specifically to directly measure SA, such as SAGAT, SPAM, and/or SART. However, research suggests that indirect physiological sensing methods may also be capable of predicting SA. Currently, it is unclear which particular physiological approaches are sensitive to changes in SA. Method: Seven databases were searched using the PRISMA reporting guidelines. Eligibility criteria included human-subject experiments that used at least one direct SA assessment technique, as well as at least one physiological measurement. Information extracted from each article was the physiological metric(s), the direct SA measurement(s), the correlation between these two metrics, and the experimental task(s). All studies underwent a quality assessment. Results: Twenty-five articles were included in this review. Eye tracking techniques were the most commonly used physiological measures, and correlations between conscious aspects of eye movement measures and direct SA scores were observed. Evidence for cardiovascular predictors of SA were mixed. EEG studies were too few to form strong conclusions, but were consistently positive. Conclusion: Further investigation is needed to methodically collect more relevant data and comprehensively model the relationships between a wider range of physiological measurements and direct assessments of SA. Application: This review will guide researchers and practitioners in methods to indirectly assess SA with sensors and highlight opportunities for future research on wearables and SA. [ABSTRACT FROM AUTHOR]

Previous studies in robotic-assisted surgery (RAS) have studied cognitive workload by modulating surgical task difficulty, and many of these studies have relied on self-reported workload measurements. However, contributors to and their effects on cognitive workload are complex and may not be sufficiently summarized by changes in task difficulty alone. This study aims to understand how multi-task requirement contributes to the prediction of cognitive load in RAS under different task difficulties. Multimodal physiological signals (EEG, eye-tracking, HRV) were collected as university students performed simulated RAS tasks consisting of two types of surgical task difficulty under three different multi-task requirement levels. EEG spectral analysis was sensitive enough to distinguish the degree of cognitive workload under both surgical conditions (surgical task difficulty/multi-task requirement). In addition, eye-tracking measurements showed differences under both conditions, but significant differences of HRV were observed in only multi-task requirement conditions. Multimodal-based neural network models have achieved up to 79% accuracy for both surgical conditions. [ABSTRACT FROM AUTHOR]

Objective: The purpose of this study was to identify, synthesize, and discuss objective behavioral or physiological metrics of surgeons' nontechnical skills (NTS) in the literature.Background: NTS, or interpersonal or cognitive skills, have been identified to contribute to safe and efficient surgical performance; however, current assessments are subjective, checklist-based tools. Intraoperative skill evaluation, such as technical skills, has been previously utilized as an objective measure to address such limitations.Methods: Five databases in engineering, behavioral science, and medicine were searched following PRISMA reporting guidelines. Eligibility criteria included studies with NTS objective measurements, surgeons, and took place within simulated or live operations.Results: Twenty-three articles were included in this review. Objective metrics included communication metrics and measures from physiological responses such as changes in brain activation and motion of the eye. Frequencies of content-coded communication in surgery were utilized in 16 studies and were associated with not only the communication construct but also cognitive constructs of situation awareness and decision making. This indicates the underlying importance of communication in evaluating the NTS constructs. To synthesize the scoped literature, a framework based on the one-way communication model was used to map the objective measures to NTS constructs.Conclusion: Objective NTS measurement of surgeons is still preliminary, and future work on leveraging objective metrics in parallel with current assessment tools is needed.Application: Findings from this work identify objective NTS metrics for measurement applications in a surgical environment. [ABSTRACT FROM AUTHOR]

Objective: The aim of this study is to assess the relationship between eye-tracking measures and perceived workload in robotic surgical tasks.Background: Robotic techniques provide improved dexterity, stereoscopic vision, and ergonomic control system over laparoscopic surgery, but the complexity of the interfaces and operations may pose new challenges to surgeons and compromise patient safety. Limited studies have objectively quantified workload and its impact on performance in robotic surgery. Although not yet implemented in robotic surgery, minimally intrusive and continuous eye-tracking metrics have been shown to be sensitive to changes in workload in other domains.Methods: Eight surgical trainees participated in 15 robotic skills simulation sessions. In each session, participants performed up to 12 simulated exercises. Correlation and mixed-effects analyses were conducted to explore the relationships between eye-tracking metrics and perceived workload. Machine learning classifiers were used to determine the sensitivity of differentiating between low and high workload with eye-tracking features.Results: Gaze entropy increased as perceived workload increased, with a correlation of .51. Pupil diameter and gaze entropy distinguished differences in workload between task difficulty levels, and both metrics increased as task level difficulty increased. The classification model using eye-tracking features achieved an accuracy of 84.7% in predicting workload levels.Conclusion: Eye-tracking measures can detect perceived workload during robotic tasks. They can potentially be used to identify task contributors to high workload and provide measures for robotic surgery training.Application: Workload assessment can be used for real-time monitoring of workload in robotic surgical training and provide assessments for performance and learning. [ABSTRACT FROM AUTHOR]

Objective: The objective of this study was to identify potential needs and barriers related to using exoskeletons to decrease musculoskeletal (MS) symptoms for workers in the operating room (OR).Background: MS symptoms and injuries adversely impact worker health and performance in surgical environments. Half of the surgical team members (e.g., surgeons, nurses, trainees) report MS symptoms during and after surgery. Although the ergonomic risks in surgery are well recognized, little has been done to develop and sustain effective interventions.Method: Surgical team members (n = 14) participated in focus groups, performed a 10-min simulated surgical task with a commercial upper-body exoskeleton, and then completed a usability questionnaire. Content analysis was conducted to determine relevant themes.Results: Four themes were identified: (1) characteristics of individuals, (2) perceived benefits, (3) environmental/societal factors, and (4) intervention characteristics. Participants noted that exoskeletons would benefit workers who stand in prolonged, static postures (e.g., holding instruments for visualization) and indicated that they could foresee a long-term decrease in MS symptoms with the intervention. Specifically, raising awareness of exoskeletons for early-career workers and obtaining buy-in from team members may increase future adoption of this technology. Mean participant responses from the System Usability Scale was 81.3 out of 100 (SD = 8.1), which was in the acceptable range of usability.Conclusion: Adoption factors were identified to implement exoskeletons in the OR, such as the indicated need for exoskeletons and usability. Exoskeletons may be beneficial in the OR, but barriers such as maintenance and safety to adoption will need to be addressed.Application: Findings from this work identify facilitators and barriers for sustained implementation of exoskeletons by surgical teams. [ABSTRACT FROM AUTHOR]