Search

Objective: This study creates linear and generalized additive models (GAMs) of video-recorded two-dimensional hand motion (synonymously referred to as hand movements or hand kinematics) to predict expert-rated performance along a series of surgical motion scales. Background : Surgical performance assessments are costly and time consuming. Automatically quantifying hand motion may offload some burden of surgical coaching and intervention by automatically collecting features of psychomotor performance. Methods : Five experts rated anonymized video clips of benchtop suturing and tying tasks ( n = 219) along four visual-analog (0–10) performance scales: fluidity of motion, motion economy, tissue handling, and hand coordination. Custom software tracked both participant hands across successive video frames and populated a robust feature set to train a series of predictive models to reproduce the expert ratings. Results : A GAM (which accounts for nonlinear effects) predicted fluidity of motion ratings with slope = 0.71, intercept = 1.98, and ${{\boldsymbol{R}}^2}$ = 0.77 for clinicians of different experience levels. Fluidity of motion and motion economy models outperformed those created to predict hand coordination and tissue handling ratings. Conclusions : Hand motion tracking may not address all contextual features of surgical tasks. Future work will explore how well simulation-based models extrapolate to more dynamic settings of the operating room.

Previous efforts used digital video to develop computer-generated assessments of surgical hand motion economy and fluidity of motion. This study tests how well previously trained assessment models match expert ratings of suturing and tying video clips recorded in a new operating room (OR) setting.Enabled through computer vision of the hands, this study tests the applicability of assessments born out of benchtop simulations to in vivo suturing and tying tasks recorded in the OR.Compared with expert ratings, computer-generated assessments for fluidity of motion (slope = 0.83, intercept = 1.77, R2 = 0.55) performed better than motion economy (slope = 0.73, intercept = 2.04, R2 = 0.49), although 85% of ratings for both models were within ±2 of the expert response. Neither assessment performed as well in the OR as they did on the training data. Assessments were sensitive to changing hand postures, dropped ligatures, and poor tissue contact-features typically missing from training data. Computer-generated assessment of OR tasks was contingent on a clear, consistent view of both surgeon's hands.Computer-generated assessment may help provide formative feedback during deliberate practice, albeit with greater variability in the OR compared with benchtop simulations. Future work will benefit from expanded available bimanual video records.

OBJECTIVE: Computer vision was used to predict expert performance ratings from surgeon hand motions for tying and suturing tasks. SUMMARY BACKGROUND DATA: Existing methods, including the objective structured assessment of technical skills (OSATS) have proven reliable, but do not readily discriminate at the task level. Computer vision may be used for evaluating distinct task performance throughout an operation. METHODS: Open surgeries were videoed and surgeon hands were tracked without using sensors or markers. An expert panel of three attending surgeons rated tying and suturing video clips on continuous scales from 0 to 10 along three task measures adapted from the broader OSATS: motion economy, fluidity of motion, and tissue handling. Empirical models were developed to predict the expert consensus ratings based on the hand kinematic data records. RESULTS: The predicted v. panel ratings for suturing had slopes from 0.73 to 1, and intercepts from 0.36 to 1.54 (Average R(2) = 0.81). Predicted v. panel ratings for tying had slopes from 0.39 to 0.88, and intercepts from 0.79 to 4.36 (Average R(2) = 0.57). The mean square error among predicted and expert ratings were consistently less than the mean squared difference among individual expert ratings and the eventual consensus ratings. CONCLUSIONS: The computer algorithm consistently predicted the panel ratings of individual tasks, and were more objective and reliable than individual assessment by surgical experts.

Historically low, the proportion of female urology residents now exceeds 25% in recent years. Self-assessment is a widely used tool to track progress in medical education. However, the validity of its results and gender differences may influence interpretation. Simulation of surgical skills is increasingly common in modern residency training and standardizes certain objective tasks and skills. The objective of this study was to identify gender differences in self-assessment of surgeons and trainees when using simulation of surgical skills.Medical students, residents, and attending and retired surgeons completed simple interrupted suturing. Assessment was self-rated using previously tested visual analog motion scales. Tasks were video recorded and rated by blinded expert surgeons using identical motion scales. Computer vision motion tracking software was used to objectively analyze the kinematics of surgical tasks.Proportion of female (n = 17) and male (n = 20) participants did not differ significantly by the level of training, P = 0.76. Five expert surgeons evaluated 84 video segments of simple interrupted suturing tasks (mean 3.0 segments per task per participant). Self-assessment correlated well overall with expert rating for motion economy (Pearson correlation coefficient 0.61, P 0.001) and motion fluidity (0.55, P = 0.002). Women underrated their performance in accordance with mean individual difference of self-assessment and expert assessment scores (Δ SAS-EAS) for both economy of motion (mean ± SEM -1.1 ± 0.38, P = 0.01) and fluidity of motion (-1.3 ± 0.39, P 0.01). On the same measures, men tended to rate themselves in accordance with experts (-0.16 ± 0.36, P = 0.63; -0.09 ± 0.41, P = 0.82, respectively). Δ SAS-EAS did not differ significantly on any rating scale across levels of training. Expert ratings did not differ significantly by gender for any domain.Female surgeons and trainees underrate some technical skills on self-assessment when compared with expert ratings, whereas male surgeon and trainee self-ratings and expert ratings were similar. Further work is needed to determine if these differences are accentuated across increasingly difficult tasks.

This study evaluates if hand movements, tracked using digital video, can quantify in-context surgical performance. Participants of varied experience completed simple interrupted suturing and running subcuticular suturing tasks. Marker-less motion tracking software traced the two-dimensional position of a region of the hand for every video frame. Four expert observers rated 219 short video clips of participants performing the task from 0 to 10 along the following visual analog scales: fluidity of motion, motion economy, tissue handling, and coordination. Expert ratings of attending surgeon hand motions (mean=7.5, sd=1.3) were significantly greater ( p

Background Often in simulated settings, quantitative analysis of technical skill relies largely on specially tagged instruments or tracers on surgeons’ hands. We investigated a novel, marker-less technique for evaluating technical skill during open operations and for differentiating tasks and surgeon experience level. Methods We recorded the operative field via in-light camera for open operations. Sixteen cases yielded 138 video clips of suturing and tying tasks ≥5 seconds in duration. Video clips were categorized based on surgeon role (attending, resident) and task subtype (suturing tasks: body wall, bowel anastomosis, complex anastomosis; tying tasks: body wall, superficial tying, deep tying). We tracked a region of interest on the hand to generate kinematic data. Nested, multilevel modeling addressed the nonindependence of clips obtained from the same surgeon. Results Interaction effects for suturing tasks were seen between role and task categories for average speed ( P = .04), standard deviation of speed ( P = .05), and average acceleration ( P = .03). There were significant differences across task categories for standard deviation of acceleration ( P = .02). Significant differences for tying tasks across task categories were observed for maximum speed ( P = .02); standard deviation of speed ( P = .04); and average ( P = .02), maximum ( P P = .03) of acceleration. Conclusion We demonstrated the ability to detect kinematic differences in performance using marker-less tracking during open operative cases. Suturing task evaluation was most sensitive to differences in surgeon role and task category and may represent a scalable approach for providing quantitative feedback to surgeons about technical skill.

Objective: This study explores how common machine learning techniques can predict surgical maneuvers from a continuous video record of surgical benchtop simulations. Background: Automatic computer vision recognition of surgical maneuvers (suturing, tying, and transition) could expedite video review and objective assessment of surgeries. Method: We recorded hand movements of 37 clinicians performing simple and running subcuticular suturing benchtop simulations, and applied three machine learning techniques (decision trees, random forests, and hidden Markov models) to classify surgical maneuvers every 2 s (60 frames) of video. Results: Random forest predictions of surgical video correctly classified 74% of all video segments into suturing, tying, and transition states for a randomly selected test set. Hidden Markov model adjustments improved the random forest predictions to 79% for simple interrupted suturing on a subset of randomly selected participants. Conclusion: Random forest predictions aided by hidden Markov modeling provided the best prediction of surgical maneuvers. Training of models across all users improved prediction accuracy by 10% compared with a random selection of participants. Application: Marker-less video hand tracking can predict surgical maneuvers from a continuous video record with similar accuracy as robot-assisted surgical platforms, and may enable more efficient video review of surgical procedures for training and coaching.

INTRODUCTION AND OBJECTIVES:Historically low, the proportion of female urology residents now exceeds 25% in recent years. Simulation of surgical skills is increasingly common in modern residency tr...

Objective This paper provides a literature review and detailed discussion of surgical skill terminology. Culminating in a novel model that proposes a set of unique definitions, this review is designed to facilitate shared understanding to study and develop metrics quantifying surgical skill. Design Objective surgical skill analysis depends on consistent definitions and shared understanding of terms like performance, expertise, experience, aptitude, ability, competency, and proficiency. Structure Each term is discussed in turn, drawing from existing literature and colloquial uses. Implications A new model of definitions is proposed to cement a common and consistent lexicon for future skills analysis, and to quantitatively describe a surgeon's performance throughout their career.

This paper compares clinician hand motion for common suturing tasks across a range of experience levels and tissue types. Medical students (32), residents (41), attending surgeons (10), and retirees (2) were recorded on digital video while suturing on one of: foam, pig feet, or porcine bowel. Depending on time in position, each medical student, resident, and attending participant was classified as junior or senior, yielding six experience categories. This work focuses on trends associated with increasing tenure observed from those medical students (10), residents (15), and attendings (10) who sutured on foam, and draws comparison across tissue types where pertinent. Utilizing custom software, the two-dimensional location of each of the participant's hands were automatically recorded in every video frame, producing a rich spatiotemporal feature set. While suturing on foam, increasing clinician experience was associated with conserved path length per cycle of the non-dominant hand, significantly reducing from junior medical students (mean = 73.63 cm, sd = 33.21 cm) to senior residents (mean = 46.16 cm, sd = 14.03 cm, p = 0.015), and again between senior residents and senior attendings (mean = 30.84 cm, sd = 14.51 cm, p = 0.045). Despite similar maneuver rates, attendings also accelerated less with their non-dominant hand (mean = 16.27 cm/s2, sd = 81.12 cm/s2, p = 0.002) than senior residents (mean = 24.84 cm/s2, sd = 68.29 cm/s2, p = 0.002). While tying, medical students moved their dominant hands slower (mean = 4.39 cm/s, sd = 1.73 cm/s, p = 0.033) than senior residents (mean = 6.53 cm/s, sd = 2.52 cm/s). These results suggest that increased psychomotor performance during early training manifest through faster dominant hand function, while later increases are characterized by conserving energy and efficiently distributing work between hands. Incorporating this scalable video-based motion analysis into regular formative assessment routines may enable greater quality and consistency of feedback throughout a surgical career.

A new equation for predicting the hand activity level (HAL) used in the American Conference for Government Industrial Hygienists threshold limit value®(TLV®) was based on exertion frequency (F) and percentage duty cycle (D). The TLV® includes a table for estimating HAL from F and D originating from data in Latko et al. (Latko WA, Armstrong TJ, Foulke JA, Herrin GD, Rabourn RA, Ulin SS, Development and evaluation of an observational method for assessing repetition in hand tasks. American Industrial Hygiene Association Journal, 58(4):278-285, 1997) and post hoc adjustments that include extrapolations outside of the data range. Multimedia video task analysis determined D for two additional jobs from Latko's study not in the original data-set, and a new nonlinear regression equation was developed to better fit the data and create a more accurate table. The equation, HAL = 6:56 ln D[F(1:31) /1+3:18 F(1:31), generally matches the TLV® HAL lookup table, and is a substantial improvement over the linear model, particularly for F1.25 Hz and D60% jobs. The equation more closely fits the data and applies the TLV® using a continuous function.

We are developing video processing algorithms for automatically measuring the ACGIH TLV® hand activity level (HAL) using marker-less tracking of hand movements. An equation for computing HAL ratings directly from tracked kinematics, rather than using a frequency-duty cycle (DC) look-up table, more readily lends itself to automated processing. Videos from the 33 Latko et al. (1997) jobs were digitized and analyzed using marker-less tracking, and hand root mean square (RMS) speed (S) was measured. A linear regression model was developed for predicting the average observer rated HAL based on the measured hand RMS speed and DC. Since the videos did not contain distance calibration, speed was quantified in pixels/s and normalized by the distance of each worker’s hand breadth, measured in pixels. A Monte Carlo simulation was performed using the US Army (1991) hand anthropometry data to determine how variation is introduced in the equation as hand breadth varies. The resulting equation was HAL= −1.06 + 0.0047 S + 0.053 DC and it predicted HAL ratings within ±1. The development of an accurate equation for estimating HAL ratings should enable use of automated and objective measurement in practice. While expert observer HAL ratings offer speed and efficiency, use of objective measurements based on worker hand kinematics should provide greater reliability, as well as offering specific engineering aspects of the job that may be addressed for reducing exposures and the risk of musculoskeletal disorders. Furthermore, automated videos analysis may help improve the speed and efficiency of making objective measurements in practice.

Patterns of physical stress exposure are often difficult to measure, and the metrics of variation and techniques for identifying them is underdeveloped in the practice of occupational ergonomics. Computer vision has previously been used for evaluating repetitive motion tasks for hand activity level (HAL) utilizing conventional 2D videos. The approach was made practical by relaxing the need for high precision, and by adopting a semi-automatic approach for measuring spatiotemporal characteristics of the repetitive task. In this paper, a new method for visualizing task factors, using this computer vision approach, is demonstrated. After videos are made, the analyst selects a region of interest on the hand to track and the hand location and its associated kinematics are measured for every frame. The visualization method spatially deconstructs and displays the frequency, speed and duty cycle components of tasks that are part of the threshold limit value for hand activity for the purpose of identifying patterns of exposure associated with the specific job factors, as well as for suggesting task improvements. The localized variables are plotted as a heat map superimposed over the video, and displayed in the context of the task being performed. Based on the intensity of the specific variables used to calculate HAL, we can determine which task factors most contribute to HAL, and readily identify those work elements in the task that contribute more to increased risk for an injury. Work simulations and actual industrial examples are described. This method should help practitioners more readily measure and interpret temporal exposure patterns and identify potential task improvements.

Marker-less 2D video tracking was studied as a practical means to measure upper limb kinematics for ergonomics evaluations. Hand activity level (HAL) can be estimated from speed and duty cycle. Accuracy was measured using a cross-correlation template-matching algorithm for tracking a region of interest on the upper extremities. Ten participants performed a paced load transfer task while varying HAL (2, 4, and 5) and load (2.2 N, 8.9 N and 17.8 N). Speed and acceleration measured from 2D video were compared against ground truth measurements using 3D infrared motion capture. The median absolute difference between 2D video and 3D motion capture was 86.5 mm/s for speed, and 591 mm/s(2) for acceleration, and less than 93 mm/s for speed and 656 mm/s(2) for acceleration when camera pan and tilt were within ± 30 degrees. Single-camera 2D video had sufficient accuracy (100 mm/s) for evaluating HAL. Practitioner Summary: This study demonstrated that 2D video tracking had sufficient accuracy to measure HAL for ascertaining the American Conference of Government Industrial Hygienists Threshold Limit Value(®) for repetitive motion when the camera is located within ± 30 degrees off the plane of motion when compared against 3D motion capture for a simulated repetitive motion task.

Objective: This study investigates using marker-less video tracking to evaluate hands-on clinical skills during simulated clinical breast examinations (CBEs). Background: There are currently no standardized and widely accepted CBE screening techniques. Methods: Experienced physicians attending a national conference conducted simulated CBEs presenting different pathologies with distinct tumorous lesions. Single hand exam motion was recorded and analyzed using marker-less video tracking. Four kinematic measures were developed to describe temporal (time pressing and time searching) and spatial (area covered and distance explored) patterns. Results: Mean differences between time pressing, area covered, and distance explored varied across the simulated lesions. Exams were objectively categorized as either sporadic, localized, thorough, or efficient for both temporal and spatial categories based on spatiotemporal characteristics. The majority of trials were temporally or spatially thorough (78% and 91%), exhibiting proportionally greater time pressing and time searching (temporally thorough) and greater area probed with greater distance explored (spatially thorough). More efficient exams exhibited proportionally more time pressing with less time searching (temporally efficient) and greater area probed with less distance explored (spatially efficient). Just two (5.9 %) of the trials exhibited both high temporal and spatial efficiency. Conclusions: Marker-less video tracking was used to discriminate different examination techniques and measure when an exam changes from general searching to specific probing. The majority of participants exhibited more thorough than efficient patterns. Application: Marker-less video kinematic tracking may be useful for quantifying clinical skills for training and assessment.

An equation was developed for estimating hand activity level (HAL) directly from tracked root mean square (RMS) hand speed (S) and duty cycle (D). Table lookup, equation or marker-less video tracking can estimate HAL from motion/exertion frequency (F) and D. Since automatically estimating F is sometimes complex, HAL may be more readily assessed using S. Hands from 33 videos originally used for the HAL rating were tracked to estimate S, scaled relative to hand breadth (HB), and single-frame analysis was used to measure D. Since HBs were unknown, a Monte Carlo method was employed for iteratively estimating the regression coefficients from US Army anthropometry survey data. The equation: HAL = 10[e(-15:87+0:02D+2:25 ln S)/(1+e(-15:87+0:02D+2:25 ln S)], R(2) = 0.97, had a residual range ± 0.5 HAL. The S equation superiorly fits the Latko et al. ( 1997 ) data and predicted independently observed HAL values (Harris 2011) better (MSE = 0.16) than the F equation (MSE = 1.28).

This study investigates the potential of using marker-less video tracking for evaluating hands-on clinical skills. Experienced family practitioners attending a national conference were recruited and asked to conduct a breast examination on a simulator that presents different clinical pathologies. Videos were taken of the clinician’s hands during the exam. Video processing software for tracking and quantifying hand motion kinematics was used. Videos were divided into two segments: a general search segment and a mass exploration segment. The general exploration segments exhibited motion patterns which included 72% faster movement and 73% higher acceleration across clinical pathologies. The most complex pathology exhibited 14% greater displacement for pressing/rubbing than for general exploration. Marker-less video kinematic tracking shows promise in discriminating between different examination procedures, clinicians, and pathologies.