Search

Observer, manual single-frame video, and automated computer vision measures of the Hand Activity Level (HAL) were compared. HAL can be measured three ways: (1) observer rating (HAL

A robust computer vision-based approach is developed to estimate the load asymmetry angle defined in the revised NIOSH lifting equation. The angle of asymmetry enables the computation of a recommended weight limit for repetitive lifting operations in a workplace to prevent lower back injuries. An open-source package OpenPose is applied to estimate the two-dimensional (2-D) locations of skeletal joints of the worker from two synchronous videos. Combining these joint location estimates, a computer vision correspondence and depth estimation method is developed to estimate the 3-D coordinates of skeletal joints during lifting. The angle of asymmetry is then deduced from a subset of these 3-D positions. Error analysis reveals unreliable angle estimates due to occlusions of upper limbs. A robust angle estimation method that mitigates this challenge is developed. We propose a method to flag unreliable angle estimates based on the average confidence level of 2-D joint estimates provided by OpenPose. An optimal threshold is derived that balances the percentage variance reduction of the estimation error and the percentage of angle estimates flagged. Tested with 360 lifting instances in a NIOSH-provided dataset, the standard deviation of angle estimation error is reduced from 10.13° to 4.99°. To realize this error variance reduction, 34% of estimated angles are flagged and require further validation.

Trunk kinematics (i.e. speed and acceleration), a risk factor associated with low back pain (LBP), is difficult to measure in the field and is not incorporated in popular lifting analysis tools. In this study, computationally efficient computer vision-based methods were used to estimate trunk kinematics from video recordings collected in a previous prospective study. We explored the relationships between trunk kinematics and health outcomes, and if incorporating trunk kinematics into the revised NIOSH Lifting Equation (RNLE) may improve its predictability of LBP risk. Significant correlations between the percentage of LBP and the average trunk speed and acceleration were found. A multivariate logistic regression model was constructed using the kinematics variables and the RNLE lifting index (LI) as independent variables to predict the probability that a task was associated with high LBP risk. When trunk kinematics was incorporated to the LI model, the predictability of the model improved significantly ( p=0.003).

Manufacturing systems involve machines and people. Both productivity and ergonomic performance are of significant importance in manufacturing. However, there is no integrated model to analyze them simultaneously. To bridge this gap, a unified model is introduced to evaluate the productivity and ergonomic performance of both manual operations and collaborative robot assembly systems. Specifically, flow time is used to represent productivity measurement and strain index characterizes ergonomic performance. Models and solutions of both performance measures in manual operations and collaborative assembly processes are introduced. Then a unified model to integrate both metrics, throughput rate per unit of work effort time, is proposed. Finally, a cylinder head assembly example is introduced to illustrate the applicability of the model.

Objective: This study creates linear and generalized additive models (GAMs) of video-recorded two-dimensional hand motion (synonymously referred to as hand movements or hand kinematics) to predict expert-rated performance along a series of surgical motion scales. Background : Surgical performance assessments are costly and time consuming. Automatically quantifying hand motion may offload some burden of surgical coaching and intervention by automatically collecting features of psychomotor performance. Methods : Five experts rated anonymized video clips of benchtop suturing and tying tasks ( n = 219) along four visual-analog (0–10) performance scales: fluidity of motion, motion economy, tissue handling, and hand coordination. Custom software tracked both participant hands across successive video frames and populated a robust feature set to train a series of predictive models to reproduce the expert ratings. Results : A GAM (which accounts for nonlinear effects) predicted fluidity of motion ratings with slope = 0.71, intercept = 1.98, and ${{\boldsymbol{R}}^2}$ = 0.77 for clinicians of different experience levels. Fluidity of motion and motion economy models outperformed those created to predict hand coordination and tissue handling ratings. Conclusions : Hand motion tracking may not address all contextual features of surgical tasks. Future work will explore how well simulation-based models extrapolate to more dynamic settings of the operating room.

Objective A computer vision method was developed for estimating the trunk flexion angle, angular speed, and angular acceleration by extracting simple features from the moving image during lifting. Background Trunk kinematics is an important risk factor for lower back pain, but is often difficult to measure by practitioners for lifting risk assessments. Methods Mannequins representing a wide range of hand locations for different lifting postures were systematically generated using the University of Michigan 3DSSPP software. A bounding box was drawn tightly around each mannequin and regression models estimated trunk angles. The estimates were validated against human posture data for 216 lifts collected using a laboratory-grade motion capture system and synchronized video recordings. Trunk kinematics, based on bounding box dimensions drawn around the subjects in the video recordings of the lifts, were modeled for consecutive video frames. Results The mean absolute difference between predicted and motion capture measured trunk angles was 14.7°, and there was a significant linear relationship between predicted and measured trunk angles ( R2 = .80, p < .001). The training error for the kinematics model was 2.3°. Conclusion Using simple computer vision-extracted features, the bounding box method indirectly estimated trunk angle and associated kinematics, albeit with limited precision. Application This computer vision method may be implemented on handheld devices such as smartphones to facilitate automatic lifting risk assessments in the workplace.

Objective Surgeon tremor was measured during vitreoretinal microscopic surgeries under different hand support conditions. Background While the ophthalmic surgeon’s forearm is supported using a standard symmetric wrist rest when operating on the patient’s same side as the dominant hand (SSD), the surgeon’s hand is placed directly on the patient’s forehead when operating on the contralateral side of the dominant hand (CSD). It was hypothesized that more tremor is associated with CSD surgeries than SSD surgeries and that, using an experimental asymmetric wrist rest where the contralateral wrist bar gradually rises and curves toward the patient’s operative eye, there is no difference in tremor associated with CSD and SSD surgeries. Methods Seventy-six microscope videos, recorded from three surgeons performing macular membrane peeling operations, were analyzed using marker-less motion tracking, and movement data (instrument path length and acceleration) were recorded. Tremor acceleration frequency and magnitude were measured using spectral analysis. Following 47 surgeries using a conventional symmetric wrist support, surgeons incorporated the experimental asymmetric wrist rest into their surgical routine. Results There was 0.11 mm/s2 (22%) greater ( p = .05) average tremor acceleration magnitude for CSD surgeries (0.62 mm/s2, SD = 0.08) than SSD surgeries (0.51 mm/s2, SD = 0.09) for the symmetric wrist rest, while no significant ( p > .05) differences were observed (0.57 mm, SD = 0.13 for SSD and 0.58 mm, SD = 0.11 for CSD surgeries) for the experimental asymmetric wrist rest. Conclusion The asymmetric wrist support reduced the difference in tremor acceleration between CSD and SSD surgeries.

Temporal frame sub-sampling (TFS) for reducing video object tracking (VOT) computing time is investigated. With a sampling ratio N, the TFS VOT algorithm will process a shorter video by sampling 1 out of N frames of the given video. The object trajectory of the remaining frames will be interpolated linearly based on those of sampled frames. Thus, TFS can result in a significant reduction of processing time at a cost of losing tracking accuracy. More importantly, it can be applied to accelerate any VOT algorithms. However, it is observed that when the object trajectory is smooth, the tracking accuracy of a TFS VOT algorithm may be improved compared to non-TFS results. In this work, we provide an empirical analysis of this unexpected outcome of the TFS scheme. We also suggest rules of thumb to leverage this property to use TFS to enhance efficiency and accuracy of VOT.