Search

Technical skills assessment can provide objective measures of surgical performance. This study assessed technical performance in cleft palate repair using a newly developed assessment tool and high fidelity cleft palate simulator through a longitudinal simulation training exercise. Three residents performed five and one resident performed nine consecutive endoscopically recorded cleft palate repairs using a cleft palate simulator. Two fellows in pediatric plastic surgery and two expert cleft surgeons also performed recorded simulated repairs. A cleft palate objective structured assessment of technical skill (CLOSATS) scoring scale and end-product score were developed to assess performance. Two blinded cleft surgeons assessed the recordings and the final repairs using the CLOSATS, end-product scores and a previously developed global rating scale. The average procedure specific (CLOSATS), global and end-product scores increased logarithmically after each successive simulation session for the residents. Reliability of the CLOSATS (average item ICC=0.85±0.093) and global ratings (average item ICC=0.91±0.02) amongst the raters was high. Reliability of the end-product assessments was lower (average item ICC=0.66±0.15). Standard setting linear regression using an overall cutoff score of 7 out of 10 corresponded to a pass score for the CLOSATS, and global rating score of 44 (maximum 60) and 23 (maximum 30) respectfully. Using logarithmic best fit curves, 6.3 simulation sessions are required to reach the minimum standard. A high fidelity cleft palate simulator has been developed that improves technical performance in cleft palate repair. The simulator and technical assessment scores can be used to determine performance before operating on patients.

Background The clinical course of ulcerative colitis (UC) and the effects of treatment are assessed through patient-reported signs and symptoms (S&S), and endoscopic evidence of inflammation. The Ulcerative Colitis Patient-Reported Outcomes Signs and Symptoms (UC-PRO/SS) measure was developed to standardize the quantification of gastrointestinal S&S of UC in clinical trials through direct report from patient ratings. Design The UC-PRO/SS was developed by collecting data from concept elicitation (focus groups, and individual interviews), then refined through a process of cognitive interviews of 57 UC patients. Measurement properties, including item-level statistics, scaling structure, reliability, and validity, were evaluated in an observational, four-week study of adults with mild to severe UC (N = 200). Results Findings from qualitative focus groups and interviews identified nine symptom items covering bowel and abdominal symptoms. The final UC-PRO/SS daily diary includes two scales: Bowel S&S (six items) and Abdominal Symptoms (three items), each scored separately. Each scale showed evidence of adequate reliability (α = 80 and 0.66, respectively); reproducibility (intraclass correlation coefficient = 0.81, 0.71) and validity, including moderate-to-high correlations with the Partial Mayo Score (0.79; 0.45) and Inflammatory Bowel Disease Questionnaire (IBDQ) total score (− 0.70; − 0.61). Scores discriminated by level of disease severity, as defined by the Partial Mayo Score, Patient Global Rating, and Clinician Global Rating (p

Objectives To systematically review and synthesise the validity evidence supporting intraoperative and simulation-based assessments of technical skill in urological robot-assisted surgery (RAS), and make evidence-based recommendations for the implementation of these assessments in urological training. Materials and methods A literature search of the Medline, PsycINFO and Embase databases was performed. Articles using technical skill and simulation-based assessments in RAS were abstracted. Only studies involving urology trainees or faculty were included in the final analysis. Results Multiple tools for the assessment of technical robotic skill have been published, with mixed sources of validity evidence to support their use. These evaluations have been used in both the ex vivo and in vivo settings. Performance evaluations range from global rating scales to psychometrics, and assessments are carried out through automation, expert analysts, and crowdsourcing. Conclusion There have been rapid expansions in approaches to RAS technical skills assessment, both in simulated and clinical settings. Alternative approaches to assessment in RAS, such as crowdsourcing and psychometrics, remain under investigation. Evidence to support the use of these metrics in high-stakes decisions is likely insufficient at present.

Background: Global rating of change (GROC) scores provide a means of measuring patients’ self-perceived change in health status over time. Objectives: The purposes of the study were to examine agre...

There continues to be a need in the field of psychology for measures that quantify qualitative data in dynamically rich and clinically meaningful ways. The Social Cognition and Object Relations Scale-Global Rating Method (SCORS-G; Westen, 1995) is a clinician-rated measure with a strong theoretical foundation that assesses sophisticated underlying constructs, specifically 8 dimensions of object relations (Stein & Slavin-Mulford, 2018 ). The SCORS-G has steadily increased in popularity and this special section of the Journal of Personality Assessment reflects this growing interest. In this introduction, we provide readers with a brief overview of the 5 articles in this special section and note ways in which they reflect and build on the existing SCORS-G empirical literature. Contextualizing these articles within the broader literature allows us to highlight how the measure is employed in clinical and research settings. Each of these 5 articles also highlights areas and opportunities for future research that would address gaps in the SCORS-G literature and strengthen the utility of the measure. We close by noting how each of studies can be seen as a model for future lines of research.

Objectives: The Tinnitus Functional Index (TFI) has been optimised as a diagnostic tool for quantifying the functional impact of tinnitus in US veteran and civilian groups. However, the TFI has not been evaluated for use in other English-speaking clinical populations despite its increasingly popular uptake. Here, a prospective multi-site longitudinal validation study was conducted to evaluate psychometric properties relevant to the UK clinical population. Guided by quality criteria for the measurement properties of health-related questionnaires, we specifically evaluated three diagnostic properties relating to the degree to which the TFI (i) covers the eight dimensions proposed to be important for diagnosis, (ii) reliably distinguishes individual differences in severity of tinnitus, and (iii) reliably measures the functional impact of tinnitus. We also examine whether clinically meaningful interpretations of the scores can be produced for the UK population. Methods: Twelve National Health Service audiology clinics across the UK recruited 255 tinnitus patients to complete questionnaires at four time-intervals, from initial clinical assessment and then over a nine-month period. Patients completed the TFI, the Tinnitus Handicap Inventory (THI), tinnitus case history questions, a Global rating of Perceived Problem with tinnitus and a Clinical Global Impression of perceived change in tinnitus. Baseline TFI data were used to examine the factor structure, construct validity and interpretability of the TFI. Follow-up TFI data were used to examine reliability. Results: Confirmatory factor analysis suggested that of the eight subscales (factors) initially established for the TFI, the ‘Auditory’ subscale did not contribute to the overall construct ‘functional impact of tinnitus’, and a modified seven-factor model (TFI-22) better fit the variance in the patient scores. Both the global 25-item TFI and the global TFI-22 scores showed exceptionally high internal consistency (α ≥ 0.95), high construct validity with the THI (r = 0.80) and high test-retest reliability (ICC = 0.87). Test-retest agreement however was only deemed to be borderline acceptable (89%). Receiver Operator Characteristic analysis indicated the 25-item TFI and TFI-22 has excellent ability to distinguish between different levels of impact (Area under the curve > 0.7). Conclusion: The TFI was confirmed to cover multiple symptom domains, measuring a multi-domain construct of tinnitus, and satisfies a range of psychometric requirements for a good clinical measure, including having excellent reliability, stability over time and sensitivity to individual differences in tinnitus severity. However, a modified seven-factor structure without the Auditory subscale (TFI-22) is recommended for calculating a global composite score for UK patients. Using patients’ experience and Receiver Operator Characteristic analysis, a grading system was presented which identifies the distinct grades of tinnitus impact in the UK clinical population that is broadly comparable to the US-based system.

Competency-based assessment tools are used in regional anaesthesia to measure the performance of study participants, trainees, and consultants. This narrative review was performed to appraise currently published assessment tools for regional anaesthesia. A literature search found 397 citations of which 28 peer-reviewed studies met the inclusion criteria of primary psychometric evaluation of assessment tools for regional anaesthesia. The included studies were diverse in the type of assessment and the skill set being assessed. The types of assessments included multiple-choice questions, hand-motion analysis, cumulative sum, visuospatial and psychomotor screening, checklists, and global rating scales. The skill sets that were assessed included holistic regional anaesthesia technical and non-technical performance observed at the bedside, to isolated part-tasks, such as needle tip visualisation under ultrasound. To evaluate validity and reliability, we compared the studies against published medical education consensus statements on ideal assessment tools. We discuss the relative merits of different tools when used to assess regional anaesthesia, the importance of psychometrically robust assessment tools in competency-based anaesthesia education, and directions for future education research in regional anaesthesia.

SummaryProgram fidelity instruments are a key ingredient for clinical supervision and implementation as well as effectiveness studies. This study examines the factor structure of the Functional Family Parole services Global Rating Measure (FFP-GRM); the program fidelity instrument of Functional Family Parole services for case management in youth parole, child protection and child welfare services. Between October 2012 and February 2015, program fidelity was measured with the FFP-GRM by Functional Family Parole supervisors. Confirmatory factor analysis was performed on 380 cases and internal consistency reliability coefficients were calculated.FindingsConfirmatory factor analyses showed that the 33-item and four-factor model of the FFP-GRM achieved a good fit to the data. Internal validity testing results showed that subscale Cronbach’s α ranged between .82 and .90.ApplicationsFindings affirm a good fit to the data and a good-to-excellent internal consistency of the FFP-GRM, which is considered sufficient to justify its use. The results are discussed with regard to the use of fidelity instruments for both clinical and research purposes.

Simulation has been established as an integral part of microsurgical training. The aim of this study was to assess and categorize the various simulation models in relation to the complexity of the microsurgical skill being taught and analyze the assessment methods commonly employed in microsurgical simulation training. Numerous courses have been established using simulation models. These models can be categorized, according to the level of complexity of the skill being taught, into basic, intermediate, and advanced. Microsurgical simulation training should be assessed using validated assessment methods. Assessment methods vary significantly from subjective expert opinions to self-assessment questionnaires and validated global rating scales. The appropriate assessment method should carefully be chosen based on the simulation modality. Simulation models should be validated, and a model with appropriate fidelity should be chosen according to the microsurgical skill being taught. Assessment should move from traditional simple subjective evaluations of trainee performance to validated tools. Future studies should assess the transferability of skills gained during simulation training to the real-life setting.

Purpose To examine the measurement properties of the Nepali version of the Quick-Disabilities of the Arm, Shoulder and Hand (QuickDASH-NP). Materials and methods The Nepali DASH and Shoulder Pain and Disability Index (SPADI) were completed at baseline assessment, and again at follow-up with the Nepali Global Rating of Change (GROC-NP) score. The 11 items of the QuickDASH-NP were extracted from the DASH and tested for confirmatory factory analysis (CFA), exploratory factor analysis (EFA), internal consistency (α), item-total correlation (ITC), test-retest reliability (ICC), measurement errors, hypothesis testing (correlation with DASH and SPADI) and responsiveness (effect size-ES, standardised response mean-SRM). Results A total of 156 participants completed questionnaires at baseline and 121 at follow-up with all questionnaires valid (no participant leaving more than one question blank). CFA suggested a poor fit for the single-factor model. The EFA demonstrated two factors with acceptable internal consistency (α = 0.79 and 0.75) for each factor. The test-retest reliability was excellent (ICC = 0.94; 95%CI:0.92–0.98), correlation was positive and very strong with the DASH-NP (r = 0.96) and strong with the SPADI-NP (r = 0.81). The Standard Error of Measurement was 2.83 and Smallest Detectable Change 7.84/100. The ES and SRM were moderate to high. Conclusions The QuickDASH-NP is reliable, valid, and able to detect change in shoulder symptoms among Nepali participants. It offers a short, easy to complete self-reporting tool for clinical use and research.

Sustainable campuses have a substantial role to play in achieving the Sustainable Development Goals (SDGs). Worldwide, there are many rating schemes to assess universities’ sustainability, UI GreenMetric is one of the most common global rating schemes aimed at encouraging green campuses and university sustainability worldwide. However, none of the existing rating schemes quantitatively measure the contributions of the implementation of its assessment indicators to achieve SDGs. There is a shortage of information on how sustainable campuses contribute to achieving SDGs. Thus, this research aimed to develop a novel index to assess and manage the contributions of sustainable campuses certified by UI GreenMetric to achieve SDGs. This article proposes novel indices, the GreenMetric Indicator Contribution Index (GMICI) and the GreenMetric Sustainable Campus Index (GMSCI), to evaluate the contributions of implementing UI GreenMetric indicators for achieving the SDGs. By implementing questionnaire surveys, we collected the relevant data. Structured questionnaire surveys yielded 35 responses from experts. The contributions of UI GreenMetric indicators to achieving SDGs were evaluated using the Relative Importance Index (RII). The results indicated an important relationship between the GreenMetric indicator and SDGs 3, 4, 6, 7, 8, 9,11, 12, 13 and 15. This research concludes that the proposed GMICI and GMSCI are a rigorous means for evaluating the contribution of UI GreenMetric indicators to UN-SDGs’ achievement.

Aim: This study aimed to evaluate Australian nursing students’ views of placements at seven tertiary education institutions with the use of the Placement Evaluation Tool (PET). Background: Clinical placements are a core element of healthcare education programs around the world (Chuan and Barnett, 2012) with undergraduate nursing students required to complete a prescribed number of hours as part of their degree. The quality of nursing clinical placements varies with a range of positive and negative learning experiences. Design: A survey design was used with a contemporary survey tool– the Placement Evaluation Tool (PET). Using Qualtrics software (Qualtrics, 2005) the on-line survey was distributed to approximately 6265 undergraduate nursing students at six Australian universities and one Technical and Further Education (TAFE) college where Bachelor of Nursing degree students were enrolled. Three Australian States were covered. Sites were selected where a project team member was employed. Methods: A total of 1263 nursing students completed the Placement Evaluation Tool (PET) − 19 items (rated 1–5), one global rating (rated 1–10) − following placement in three Australian States (July 2019−February 2020). Most - 618 (48.9%) completed a placement in acute care with placements positively rated overall. Results: The total PET mean score was 78.3% with 29.8% being ‘extremely satisfied’ (10 out of 10 – Item 20). However, 11.0% were dissatisfied with global ratings of four or less, whilst ratings between States differed significantly (p =

SUMMARYObjective: To determine the level of satisfactionof users of dentistry of an institutionof contributory regime at the city of Cali.Methods: This is a cross-sectional study.The calculated sample size was 242 patientsobtained by simple random probabilitysampling. The dates were analyzed in thestatistical package Epi-info v. 5.4Results: Female gender predominated with62,8% (CI 95%: 56,4 – 68,9); the moreprevalent type of membership was contributorwith 60,7% (CI 95%: 54,3 – 66,9).Despite of limitations of Health MandatoryPlan in dentistry area, the global rating ofsatisfaction was 4.1 in a scale from 1 to 5.Conclusions: The level of satisfaction ofthe users of dentistry service at the surveyedinstitution was 82%. Today the knowledgeof level of satisfaction of users of healthservices is an important tool for the administration,researching, and marketing ofthese services.Keywords: Satisfaction of users, quality,process of attention, live well

The Patient and Observer Scar Assessment Scale (POSAS) is one of the most robust instruments to assess scar quality, but there is no Italian version, and no other competing instruments are available in Italian. The aim of this study was to translate and validate an Italian version of POSAS (POSAS-I). POSASv2.0 was culturally adapted in accordance with international standards. The psychometric assessment included acceptability/feasibility, internal consistency, reproducibility, construct validity and sensitivity to change. Cultural equivalence of POSAS-I with the English version was confirmed. The validation study included 102 subjects with surgical scars. Both subscales demonstrated acceptable internal consistency (Cronbach's α = 0·72–0·80). Reproducibility of the OSAS-I (ICCs = 0·93–0·94; SEM = 1·8 points; MDC95 = 5·1 points) was superior to that of PSAS-I (ICC = 0·65; SEM = 5·7 points; MDC95 = 15·7 points). OSAS-I showed moderate to good correlations with the Vancouver Scar Scale (VSS), Global Rating of Change Scale (GRCS) and PSAS-I. Sensitivity to change was large for PSAS-I (effect size = 1·08; standardised response mean = 0·96) and moderate to large for OSAS-I (ES = 0·69; SRM = 0·92). This study confirmed that POSAS-I can be used to assess patients with surgical scars in the Italian population. OSAS-I is useful for clinical and research purposes, while PSAS-I should be better used to capture patients' own opinions and symptoms in clinical settings.

Background The Care-Related Quality of Life survey for Chronic Heart Failure (CaReQoL CHF) is a newly developed patient-reported outcome measure (PROM) that measures care-related quality of life in patients diagnosed with chronic heart failure. This study describes the psychometric properties of the questionnaire and its relationship with disease severity and global rating of quality of care. Method Insurance companies selected patients with a recorded diagnosis of chronic heart failure and for whom the hospital submitted a billing statement in the last year. Exploratory factor analysis, Cronbach’s alpha and item-rest correlation were used to construct the CaReQoL CHF. Construct validity was assessed by examining the mean values of the CaReQoL CHF scales for the categories of the widely-used New York Heart Association (NYHA) functional classification and by correlating the global rating of quality of care with the CaReQoL CHF scales. Results One thousand eighteen patients with chronic heart failure filled out the CaReQoL CHF (RR: 35.7%). The CaReQoL CHF consists of 20 items and three scales: social and emotional problems, physical limitations, and being in safe hands. The mean scores of the three scales differed significantly for the NYHA categories, particularly for the social-emotional problems and physical limitation scales. The ‘being in safe hands’ scale showed a moderate positive correlation with the global rating of quality of care. Conclusions The CaReQoL CHF is a concise and valid PROM that matches patients’ priorities in healthcare. It adds a new element to existing quality of life questionnaires for patients with chronic heart failure, that is ‘being in safe hands’ scale. This scale is relevant for patients because they experience anxiety and tension about their condition. Future research should determine whether the CaReQoL CHF can help healthcare providers in daily practice to focus treatment on outcomes of care that are relevant to individual patients.

Rationale and Objectives Workplace-based assessments gauge the highest tier of clinical competence. Chart-stimulated recall (CSR) is a workplace-based assessment method that complements chart audit with an interview based on the residents' notes. It allows evaluation of the residents' knowledge and heuristics while providing opportunities for feedback and self-reflection. We evaluated the utility of CSR for improving the radiology residents' reporting skills. Materials and Methods Residents in each year of training were randomly assigned to an intervention group (n = 12) or a control group (n = 13). Five pre-intervention and five post-intervention reports of each resident were independently evaluated by three blinded reviewers using a modified Bristol Radiology Report Assessment Tool. The study intervention comprised a CSR interview tailored to each individual resident's learning needs based on the pre-intervention assessment. The CSR process focused on the clinical relevance of the radiology reports. Student's t test (P

As urology training programs move to a competency based medical education model, iterative assessments with objective standards will be required. To develop a valid set of technical skills standards we initiated a national skills assessment study focusing initially on laparoscopic skills.Between February 2014 and March 2016 the basic laparoscopic skill of Canadian urology trainees and attending urologists was assessed using 4 standardized tasks from the AUA (American Urological Association) BLUS (Basic Laparoscopic Urological Surgery) curriculum, including peg transfer, pattern cutting, suturing and knot tying, and vascular clip applying. All performances were video recorded and assessed using 3 methods, including time and error based scoring, expert global rating scores and C-SATS (Crowd-Sourced Assessments of Technical Skill Global Rating Scale), a novel, crowd sourced assessment platform. Different methods of standard setting were used to develop pass-fail cut points.Six attending urologists and 99 trainees completed testing. Reported laparoscopic experience and training level correlated with performance (p 0.01). Attending urologists were significantly better than trainees (p0.05), demonstrating construct validity evidence for the 4 AUA BLUS tasks. The C-SATS method of assessment correlated well with the traditional methods of time and error based scoring, and the global rating scale. We were able to use relative and absolute standard setting methods to define pass-fail cut points for all 4 AUA BLUS tasks.The 4 AUA BLUS tasks demonstrated good construct validity evidence for use in assessing basic laparoscopic skill. Performance scores using the novel C-SATS platform correlated well with traditional time-consuming methods of assessment. Various standard setting methods were used to develop pass-fail cut points for educators to use when making formative and summative assessments of basic laparoscopic skill.

Risk matrices have been widely used as a risk evaluation tool in many fields due to their simplicity and intuitive nature. Designing a rating scheme, i.e., determining the number of ratings used in a risk matrix and assigning different ratings to different cells, is an essential part of risk matrix construction. However, most of the related literature has focused on applying a risk matrix to various fields, instead of researching how to design risk matrices. Based on the analysis of several current rules, we propose a new approach, namely, the sequential updating approach (SUA), to design the rating scheme of a risk matrix in a reliable way. In this article, we propose three principles and a rating algorithm based on these principles. The three principles, namely, adjusted weak consistency, consistent internality, and continuous screening, characterize a good rating scheme. The resulting rating scheme has been proven to be unique. A global rating algorithm is then proposed to create the design that satisfies the three principles. We then explore the performance of the SUA. An illustrative application is first given to explain the feasibility of our approach. The sensitivity analysis shows that our method captures a resolution-reliability tradeoff for decisionmakers in choosing an appropriate rating scheme for a risk matrix. Finally, we compare the designs based on the SUA and Cox's axioms, highlighting the advantages of the SUA.

Media use among US children is high and access to new media outlets continues to grow, which has important implications for how children spend their time. This study builds upon research on other lifestyle behaviors to explore the impact of a global rating of parent–child relationship quality on recreational media use (RMU). Data from a nationally representative sample of US children and adolescents aged 8–18 years (n = 1727) were analyzed. Multiple linear regression tested the association between parent–child relationship quality and RMU as well as if the association between parent–child relationship quality and RMU varied with age. Overall, higher quality parent–child relationships were associated with lower RMU, but this association was only significant for children aged 11–13 years. This study points to possible intervention strategies to limit excessive media use in children such as improving parent–child relationship quality, especially during the critical period of the pre-teen years.

The aim of this study was to evaluate the results from 129 projects involved in different socio-educational actions designed to meet the reception and integration needs of immigrants and contribute to the development of an intercultural society. To this end, through evaluative research, structured interviews were carried out with project coordinators. The results indicate a good global rating of the projects as well as a high degree of attainment of the project objectives and take-up by the intended users. We found no statistically significant difference with regards to the type of project and the lines of action taken.

In this paper, we examine the effects of the global rating recalibration conducted by Moody's and Fitch credit rating agencies in early 2010. We test the hypothesis forwarded by the rating agencies that the recalibration was “yield neutral.” Using time series methods, we find that the rating recalibration brought a structural change to the municipal bond market and increased the spread of municipal bonds in the Aaa, Aa, and A rating categories over their risk-free comparison group by approximately 15 basis points. The impact of the rating recalibration on the spread of Baa-rated bonds was not statistically significant.

Competency-based surgical training requires assessment tools, and these should be both valid and reliable. Global rating scales are one such method, with several developed and investigated for use in arthroscopic skills assessment. These global rating scales have shown acceptable reliability and construct validity, most commonly demonstrated using simulated arthroscopic tasks. However, questions remain regarding their readiness for use in the operating room and for the assessment of complex tasks.

Study Design Clinical measurement study. Background Ankle sprain is a common condition. Therefore, translated, culturally adapted, and clinimetrically tested instruments for measuring the ankle are needed. Objectives To translate and cross-culturally adapt the Sports Athlete Foot and Ankle Score (SAFAS) and the global rating of change (GROC) into Brazilian Portuguese. This study also aimed to test the measurement properties of the SAFAS, the GROC, and the existing Brazilian-Portuguese versions of the numeric pain-rating scale (NPRS) and the ankle-hindfoot scale of the American Orthopaedic Foot and Ankle Society (AOFAS) in a group with ankle sprain. Methods The SAFAS and GROC instruments were cross-culturally adapted into Brazilian Portuguese and pretested in this population (n = 18). The measurement properties of the SAFAS, GROC, NPRS, and AOFAS (internal consistency, ceiling and floor effects, construct validity, reproducibility, and responsiveness) were tested in a test-retest design in patients with ankle sprain at baseline (n = 63) and at follow-up periods of 48 to 72 hours (n = 53) and 4 weeks (n = 43) after baseline. Results The SAFAS, GROC, NPRS, and AOFAS yielded considerable internal consistency (Cronbach alpha ranging from .76 to .97), moderate reliability (intraclass correlation coefficient model 2,1 ranging from 0.59 to 0.80), and dubious agreement (percentage of the standard error of measurement ranging from 13% to 22.43%). The highest correlations on construct validity were observed among the 4 SAFAS subscales (Pearson r = 0.75 to 0.86, P.01), but statistically significant values appeared in the intercorrelation of instruments. There were no observed floor or ceiling effects in any of the instruments. All of their scores of change showed moderate correlation with the GROC (r = -0.48 to 0.30, P.05) and could be classified as responsive. Conclusion The Brazilian-Portuguese versions of the SAFAS, GROC, NPRS, and AOFAS had acceptable measurement properties in high-performing athletes. J Orthop Sports Phys Ther 2016;46(12):1042-1050. Epub 8 Nov 2016. doi:10.2519/jospt.2016.6218.

Purpose Some educational programs have adopted the premise that the same assessment can serve both formative and summative goals; however, how observers understand and integrate the intended uses of assessment may affect the way they execute the assessment task. The objective of this study was to explore the effect of foregrounding a different intended use (formative vs summative learner assessment) on observer contributions (ratings and comments). Method In this randomized, experimental, between-groups, mixed-methods study (May-September 2017), participants observed 3 prerecorded clinical performances under formative or summative assessment conditions. Participants rated performances using a global rating tool and provided comments. Participants were then asked to reconsider their ratings from the alternative perspective (from which they were originally blinded). They received the opportunity to alter their ratings and comments and to provide rationales for their decision to change or preserve their original ratings and comments. Outcomes included participant-observers' comments, ratings, changes to each, and stated rationales for changing or preserving their contributions. Results Foregrounding different intended uses of assessment data for participant-observers did not result in differences in ratings, number or type of comments (both emphasized evaluative over constructive statements), or the ability to differentiate among performances. After adopting the alternative perspective, participant-observers made only small changes in ratings or comments. Participant-observers reported that they engage in the process in an evaluative manner despite different intended uses. Conclusions Foregrounding different intended uses for assessments did not result in significant systematic differences in the assessment data generated. Observers provided more evaluative than constructive statements overall, regardless of the intended use of the assessment. Future research is needed to explore whether these results hold in social/workplace-based contexts and how they might affect learners.

Background and aims The Core Outcome Measures Index (COMI) is a short, self-reported questionnaire for assessing important outcomes in patients with low back pain (LBP). The present study was conducted to explore the responsiveness and longitudinal validity of the Persian COMI (COMI-P) in patients with non-specific chronic LBP. Methods In this prospective cohort study of patients with non-specific chronic LBP receiving physiotherapy, patients completed a booklet containing the COMI-P, Persian Functional Rating Index (FRI-P), and a visual analogue scale (VAS) for pain before and after the end of ten-sessions of physiotherapy. Patients also completed a global rating of change scale (GRCS) at the end of the physiotherapy. Responsiveness was examined by means of internal responsiveness methods [t-test, standard effect size (SES); standardized response mean (SRM), and Guyatt responsiveness index (GRI)] and external responsiveness methods [correlation with external criteria and receiver operating characteristics (ROC) curve]. Results Fifty patients with a mean age of 50.62 ± 13.8 years participated. The paired t-test showed significant changes in COMI-P scores (p p p p = 0.02). The COMI-P change scores showed excellent correlation with the dichotomized smallest detectable change (SDC) criterion (r = 0.83, p Conclusions The COMI-P appears to have responsiveness and longitudinal validity in detecting changes after physiotherapy for non-specific chronic LBP. An improvement of 2.15 points in COMI-P total score is required to be interpreted as minimally clinically important change in individual patients.

ObjectiveThe purpose of this systematic review was to critically appraise and synthesise the psychometric properties of Global Rating of Change (GROC) scales for assessment of patients with neck pain.DesignSystematic review.Data sourcesA search was performed in four databases (MEDLINE, EMBASE, CINAHL, SCOPUS) until February 2019.Data extraction and synthesisEligible articles were appraised using Consensus-based Standards for the selection of health Measurement Instruments checklist and the Quality Appraisal for Clinical Measurement Research Reports Evaluation Form.ResultsThe search obtained 16 eligible studies and included in total 1533 patients with neck pain. Test–retest reliability of global perceived effect (GPE) was very high (intraclass correlation coefficient=0.80 to 0.92) for patients with whiplash. Pooled data of Pearson’s r indicated that GROC scores were moderately correlated with neck disability change scores (0.53, 95% CI: 0.47 to 0.59). Pooled data of Spearman’s correlations indicated that GROC scores were moderately correlated with neck disability change scores (0.56, 95% CI: 0.41 to 0.68).ConclusionsThis study found excellent quality evidence of very good-to-excellent test–retest reliability of GPE for patients with whiplash-associated disorders. Evidence from very good-to-excellent quality studies found that GROC scores are moderately correlated to an external criterion patient-reported outcome measure evaluated pre-post treatment in patients with neck pain. No studies were found that addressed the optimal form of GROC scales for patients with neck disorders or compared the GROC to other options for single-item global assessment.PROSPERO registration numberCRD42018117874.

Background: low back pain (LBP) is the main cause of years lived with disability worldwide. Psychosocial factors have been shown to be good predictors of persistent LBP. Within these, unhelpful beliefs about the back seem to be important in the development and chronicity of the symptoms. The Back Pain Attitudes Questionnaire (Back-PAQ) is an instrument that explores beliefs about the back that has been validated for people with and without back pain and healthcare professionals. However, until now, it has not been translated and validated for the Argentine population. Objective: translate into Spanish, cross-cultural adapt and validate the Back-PAQ for the Argentine population with and without back pain. Study design: study of diagnostic accuracy/assessment scale. Methods: the study was carried out in three consecutive phases: translation, cross-cultural adaptation and validation. We included Argentinians aged 18 years or more. We used the Back-PAQ, modified Fear Avoidance Beliefs Questionnaire (mFABQ) and the Global Rating of Change (GROC) scale to assess the psychometric properties. Results: three hundred and seventy-two participants were included for the analysis. The time taken to answer and score the questionnaire was 5.6 and 1.6 min, respectively. Neither a ceiling nor a floor effect was observed. Internal consistency was 0.76. One hundred and eighty-six participants were considered stable. Test-retest reliability was 0.90. A weak correlation (0.33) was found between the Back-PAQ and the mFABQ. Conclusion: the Argentine version of the Back-PAQ is a viable, reliable and valid tool for the assessment of the back beliefs of the Argentine population.

Microsurgical skill acquisition is an integral component of training in plastic surgery. Current microsurgical training is based on the subjective Halstedian model. An ideal microsurgery assessment tool should be able to deconstruct all the subskills of microsurgery and assess them objectively and reliably. For our study, to analyze the feasibility, reliability, and validity of microsurgery skill assessment, a video-based objective structured assessment of technical skill tool was chosen. Two blinded experts evaluated 40 videos of six residents performing microsurgical anastomosis for arteriovenous fistula surgery. The generic Reznick's global rating score (GRS) and University of Western Ontario microsurgical skills acquisition/assessment (UWOMSA) instrument were used as checklists. Correlation coefficients of 0.75 to 0.80 (UWOMSA) and 0.71 to 0.77 (GRS) for interrater and intrarater reliability showed that the assessment tools were reliable. Convergent validity of the UWOMSA tool with the prevalidated GRS tool showed good agreement. The mean improvement of scores with years of residency was measured with analysis of variance. Both UWOMSA (p-value: 0.034) and GRS (p-value: 0.037) demonstrated significant improvement in scores from postgraduate year 1 (PGY1) to PGY2 and a less marked improvement from PGY2 to PGY3. We conclude that objective assessment of microsurgical skills in an actual clinical setting is feasible. Tools like UWOMSA are valid and reliable for microsurgery assessment and provide feedback to chart progression of learning. Acceptance and validation of such objective assessments will help to improve training and bring uniformity to microsurgery education.

Simulation-based training has been considered as the most promising curriculum for neurosurgical education to finally improve surgical skills with the greatest efficiency and safety. However, most of the simulators including physical models and virtual reality systems are relatively expensive, which limits their promotion. In this study, the authors tried to develop a realistic, low-cost, and reusable simulator for endoscopic third ventriculostomy (ETV) and evaluate its validity. A 3D-printed rigid skull with the ventricular system originated from a de-identified patient with obstructive hydrocephalus was constructed. The third ventricular floor was designed as a replaceable module. Thirty-nine neurosurgeons tested the simulator and a rating system was established to assess their performance. All participants filled out questionnaires to evaluate the simulator after training. Five neurosurgical students were recruited to finish the whole training for ten times in order to explore the learning curve of ETV. We found that (1) the more experienced surgeons performed obviously better than the rather inexperienced surgeons which verified that our model could reflect the ability of the trainees; (2) as the training progressed, the scores of the post-graduates increased and the fifth training average score was obviously higher than their first training average score. The feedback questionnaires showed the average scores for value of the simulator as a training tool and global rating were 3.15 and 3.54 (on a 4-point scale). Our model was practical for ETV training. The results of our program showed that our model could precisely reflect the operators’ ability to perform ETV and could make it more efficient to master basic skills.

Patient-reported outcomes (PRO’s) are important in clinical decision making. Common PRO’s for COPD may provide limited insight into specific aspects of dyspnea or can be time-consuming to administer. We evaluated the use of 4 Dyspnea Numeric Rating Scales (NRS), measuring intensity and unpleasantness of dyspnea during both rest and activity as a PRO in pulmonary rehabilitation (PR) of COPD patients. Patients (n=379, FEV1%=55±17) completed Dyspnea NRS, COPD Assessment Test (CAT), Saint-George Respiratory Questionnaire (SGRQ), 2 weeks before (T0), at the start (T1) and end (T2) of PR. Spirometry and 6 min walking test (6MWT) was performed at T1 and T2. Global Rating of Change (GRC) was provided at T2. We assessed factor structure, reliability and validity of the NRS, assessed sensitivity to change, and estimated MCID. Single- and 2-factor solutions (rest and activity), explained 69% vs. 93% of variance, and had good internal consistency (αtotal=.85, αrest=.92, αactivity=.93). NRStotal had good test-retest reliability (ICC=.74). Correlations with SGRQ were high (rtotal=.75, rrest=.61, ractivity=.67); comparable to CAT-SGRQ correlations (r=.78). Change in NRS was associated with change in CAT (r=.52) and SGRQ (r=.51). None of the PRO’s correlated with changes in 6MWT or FEV1. NRS, CAT and SGRQ showed similar sensitivity to change (d=.74, .81, .76). Criterion- and reliability-based methods led to similar MCID estimates for NRS (GRC: 3-5, SEM: 4). Dyspnea NRS are easy to use and may provide a more comprehensive assessment of dyspnea compared to other PRO’s and. A 4-point change on the Dyspnea NRS implies reliable and clinically important change during PR in COPD.

Background Accurate tools for measuring physical activity are important for monitoring patients with chronic pain. However, these tools have not been properly validated in this population. Objective To determine the suitability of two physical activity measures for use in chronic pain populations. Design Longitudinal observational study. Setting Pain Management and Research Centre. Participants Sixty-four patients presenting to the Pain Management and Research Centre with chronic pain. Methods Participants underwent a 3-week pain management program that included cognitive behavioral strategies, education, and multiple exercises sessions per day. Physical activity was measured by the Active Australia Survey and the Actigraph GT3X at the start and end of the program. Feasibility of the physical activity measures was assessed. Criterion validity and responsiveness between the measures was compared. Correlation of physical activity with self-rated global rating of change (GROC) scales in health were calculated. Main outcome measurements Minutes per day spent in low, moderate, and vigorous physical activity; global rating of change. Results Fifty-two percent (n = 33) and 78% (n = 50) of participants had valid Actigraph and Active Australia data, respectively, for both time points. For criterion validity and responsiveness correlations varied (rho = -.12-.52). All correlations between physical activity measures and GROC were small or negligible (rho ≤ .18). Conclusion Feasibility of the Actigraph was poor compared to the Active Australia Survey. The criterion validity and responsiveness of the Active Australia Survey to the Actigraph was inconsistent and the relationship of both these measures to clinical outcomes was poor. However, limitations due to missing data mean that further research is required. Level of evidence III.

Background The Shoulder Pain and Disability Index (SPADI) is a 13-item shoulder-specific patient-reported outcome measure (PROM). The English version is easy to use and has demonstrated excellent measurement properties for both clinical and research settings. The availability of the SPADI in Nepali would facilitate shoulder research and enhance management of patients with shoulder pain in Nepal. Therefore, the purpose of this study was to translate and cross-culturally adapt the SPADI into Nepali (SPADI-NP) and evaluate its measurement properties. Methods The translation and adaptation process followed international guidelines. Participants completed SPADI-NP on two assessments (N = 150 at initial and 119 at follow-up assessment). A Nepali version of the Global Rating of Change score was completed at follow-up. Assessment of measurement properties included analysis of internal consistency (Cronbach’s α), minimal detectable change (MDC) with standard error of measurement (SEM), test-retest reliability (intraclass correlation coefficient; ICC), validity (factor structure, construct using Pearson’s correlation with the Disability of Arm and Hand [DASH]) and responsiveness (area under the curve; AUC) with minimal important change (MIC). Results Minor changes were integrated in the adaptation process to improve cultural relevance such as dress items. Items were largely loaded under two factors (pain and disability), internal consistencies were good for the pain construct (α = 0.82) and disability (α = 0.88) and test-retest reliability was excellent (pain = 0.89, disability = 0.96). MDC was 5.7 (out of 100) with SEM = 2.1. Strong associations with the DASH (r = 0.63 pain, r = 0.81 disability) demonstrated its construct validity. The AUC was 0.68 and MIC was 12.3 (out of 100). Conclusion The Nepali version of the SPADI demonstrated excellent reliability and validity. It can be used for the assessment of shoulder pain and disability in patients with shoulder pain in Nepal in both clinical practice and research.

We present the British Society of Gastroenterology (BSG) position statement on patient experience of GI endoscopy, recently published on the BSG website—www.bsg.org.uk/resource/patient-experience-of-gi-endoscopy-2019.html. The three dimensions of healthcare quality are patient safety, clinical effectiveness and patient experience, with much of healthcare practice focusing on the first two dimensions. Greater emphasis is now being given to the patient experience dimension in light of reports from Francis1 and Darzi2 highlighting the interaction between patient experience and quality of care. Clinical standards and safety in endoscopy are well reported and reviewed via the Joint Advisory Group on GI Endoscopy (JAG) accreditation programme. The Global Rating Scale3 used in JAG assessments includes a patient experience domain, but gives limited guidance available on how that should be measured or what standards should be …

Almost all depression measures have been developed without discussing how to best conceptualize and assess the severity of depression. The most valid rating format of depression severity scales is unsettled and has been little studied. In the present study from the Rhode Island Methods to Improve Diagnostic Assessment and Services (MIDAS) project, we compared the validity of alternative approaches towards rating the severity of depressive symptoms. Data was collected using TurkPrime. One hundred eighty-five participants currently in treatment for a mental health problem completed a self-report measure of depression and rated the symptoms on two 4-point ordinal scales assessing symptom frequency and symptom intensity. The respondents also rated their global level of depression severity and completed a measure of psychosocial functioning and quality of life. The symptom ratings based on intensity and frequency were highly correlated with each other, and equally highly correlated with subjects' global rating of overall severity of depression, as well as ratings of psychosocial functioning and quality of life. A composite index of severity based on the sum of frequency and severity ratings was no more highly correlated with the external validators. The results of the present study suggest that ratings of depressive symptoms based on either symptom intensity or symptom frequency are equally valid.

Developing high levels of competence in the execution of surgical procedures through training is a key factor for obtaining good clinical results in healthcare. To improve the effectiveness of the training, it is advisable to provide feedback to each student tailored to how the student has performed the procedure on each occasion. Current state-of-the-art feedback is based on Checklists and Global Rating Scales, which indicate whether all process steps have been carried out and the quality of each execution step. However, there is a process perspective that is not captured successfully by these instruments, e.g., steps performed, but in an undesired order, group of activities that are repeated an unnecessary number of times, or an excessive transition time between two consecutive steps. In this research, we propose a novel use of process mining techniques to effectively identify desired and undesired process patterns regarding rework, the order in which activities are performed, and time performance, in order to complement the tailored feedback for surgical procedures using a process perspective. The proposed approach was applied to analyze a real case of ultrasound-guided Central Venous Catheter placement training. It was quantitatively and qualitatively validated that the students who participated in the training program perceived the process-oriented feedback they received as favorable for their learning.

ObjectiveTo compare the effects of a programme of load management education plus exercise, corticosteroid injection use, and no treatment on pain and global improvement in individuals with gluteal tendinopathy.DesignProspective, three arm, single blinded, randomised clinical trial.SettingBrisbane and Melbourne, Australia.ParticipantsIndividuals aged 35–70 years, with lateral hip pain for more than 3 months, at least 4/10 on the pain numerical rating scale, and gluteal tendinopathy confirmed by clinical diagnosis and MRI; and with no corticosteroid injection use in previous 12 months, current physiotherapy, total hip replacement, or neurological conditions.InterventionsA physiotherapy led education and exercise programme of 14 sessions over 8 weeks (EDX; n=69), one corticosteroid injection (CSI; n=66), and a wait and see approach (WS; n=69).Main outcomesPrimary outcomes were patient reported global rating of change in hip condition (on an 11 point scale, dichotomised to success and non-success) and pain intensity in the past week (0=no pain, 10=worst pain) at 8 weeks, with longer term follow-up at 52 weeks.ResultsOf 204 randomised participants (including 167 women; mean age 54.8 years (SD 8.8)), 189 (92.6%) completed 52 week follow-up. Success on the global rating of change was reported at 8 weeks by 51/66 EDX, 38/65 CSI, and 20/68 WS participants. EDX and CSI had better global improvement scores than WS (risk difference 49.1% (95% CI 34.6% to 63.5%), number needed to treat 2.0 (95% CI 1.6 to 2.9); 29.2% (13.2% to 45.2%), 3.4 (2.2 to 7.6); respectively). EDX had better global improvement scores than CSI (19.9% (4.7% to 35.0%); 5.0 (2.9 to 21.1)). At 8 weeks, reported pain on the numerical rating scale was mean score 1.5 (SD 1.5) for EDX, 2.7 (2.4) for CSI, and 3.8 (2.0) for WS. EDX and CSI participants reported less pain than WS (mean difference −2.2 (95% CI −2.89 to −1.54); −1.2 (−1.85 to −0.50); respectively), and EDX participants reported less pain than CSI (−1.04 (−1.72 to −0.37)). Success on the global rating of change was reported at 52 weeks by 51/65 EDX, 36/63 CSI, and 31/60 WS participants; EDX was better than CSI (20.4% (4.9% to 35.9%); 4.9 (2.8 to 20.6)) and WS (26.8% (11.3% to 42.3%); 3.7 (2.4 to 8.8)). Reported pain at 52 weeks was 2.1 (2.2) for EDX, 2.3 (1.9) for CSI, and 3.2 (2.6) for WS; EDX did not differ from CSI (−0.26 (−1.06 to 0.55)), but both treatments did better than WS (1.13 (−1.93 to −0.33); 0.87 (−1.68 to −0.07); respectively).ConclusionsFor gluteal tendinopathy, education plus exercise and corticosteroid injection use resulted in higher rates of patient reported global improvement and lower pain intensity than no treatment at eight weeks. Education plus exercise performed better than corticosteroid injection use. At 52 week follow-up, education plus exercise led to better global improvement than corticosteroid injection use, but no difference in pain intensity. These results support EDX as an effective management approach for gluteal tendinopathy.Trial registration numberProspectively registered at the Australian New Zealand Clinical Trials Registry (ACTRN12612001126808).

to investigate the benefit of adding stretching exercises to cervical joint mobilization and active rotation exercises for patients with non-specific mechanical neck pain.Thirty-eight subjects with non-specific mechanical neck pain were randomly assigned to a standard procedure group (passive cervical mobilization and active cervical rotation range of motion exercise) or a combined procedure (passive cervical mobilization, active cervical rotation range of motion exercises, and stretching procedures). Mixed factorial analysis of variance was used to compare changes between groups over time in active cervical range of motion, Numeric Pain Rating Scale, Neck Disability Index, Global Rating of Change, and Pressure Pain Threshold.There was a significant change in mean active range of motion in all directions, Pressure Pain Threshold, perceived pain, disability levels, and global rating of change over time (p 0.001). There was a significant group by time interaction in mean active range of motion during extension (p = 0.01), right rotation (p = 0.004), right and left lateral flexion (p = 0.05, and p = 0.02 respectively). However, there was no significant group by time interaction in mean active range of motion during flexion, left rotation, pain intensity (p = 0.09), right and left pressure pain threshold (p = 0.30, 0.47, respectively), and disability (p = 0.07).Both study groups improved significantly in all subjective and objective outcome measures. However, data from this study suggest that adding stretching to the standard procedures may be more effective than the standard procedure alone at improving cervical extension, right rotation, and lateral flexion active range of motion, but not pain and disability.

Single-item measures of global job stressfulness are increasingly used in occupational health research, yet their construct validity remains unexplored. This study used a qualitative approach to identify frames of reference that underlie self-ratings on such a measure. Data were collected from a convenience sample of 55 adults in full-time employment who completed a single-item measure inviting a rating of the extent to which their job is generally stressful. A cognitive interview schedule was used to explore the factors taken into account when providing a global rating, with thematic analysis applied to identify themes in the interview transcripts. The most common frames of reference were the presence of problematic psychosocial working conditions, particularly job demands. Health characteristics, predominantly poor psychological wellbeing, emerged as a further less dominant secondary theme. Almost half the sample cited four or more referents. In terms of the timeframe under consideration, most participants referred to a long timeframe such as their work in general, with some specifying their current job and, a few, recent weeks. These findings shed light on the frames of reference used to inform judgements on global job stressfulness elicited by a single-item measure and in doing so contribute to the evidence base to support the application of such measures in occupational health research and organisational psychosocial risk management activities.

Effect sizes are an important outcome of quantitative research, but few guidelines exist that explain how researchers can determine which effect sizes are meaningful. Psychologists often want to study effects that are large enough to make a difference to people's subjective experience. Thus, subjective experience is one way to gauge the meaningfulness of an effect. We propose and illustrate one method for how to quantify the smallest subjectively experienced difference—the smallest change in an outcome measure that individuals consider to be meaningful enough in their subjective experience such that they are willing to rate themselves as feeling different—using an anchor-based method with a global rating of change question applied to the positive and negative affect scale. We provide a step-by-step guide for the questions that researchers need to consider in deciding whether and how to use the anchor-based method, and we make explicit the assumptions of the method that future research can examine. For researchers interested in people's subjective experiences, this anchor-based method provides one way to specify a smallest effect size of interest, which allows researchers to interpret observed results in terms of their theoretical and practical significance.

Background This methodological study evaluated the psychometric properties of the Persian version of the questionnaire for patients’ satisfaction with physical therapy. Methods The Persian version of this questionnaire was prepared through forward and backward translation. Its reliability was examined by the intraclass correlation coefficient (ICC) and Cronbach’s alpha. Structural validity was evaluated through confirmatory factor analysis and assessing the correlation between the score of the questionnaire with the visual analog score (VAS) and global rating of change (GRC) scale. The correlation of the questionnaire was checked with PTPSQ. Results A total of 297 patients with musculoskeletal disorders at public physical therapy centers in Kerman, Iran, participated in this study and 40 participants, five to seven days later, answered the questionnaire again. Cronbach’s alpha coefficient was 0.95 and ICC was 0.88. The confirmatory factor analysis confirmed the four-factor solution. The correlation coefficients of the total score of the questionnaire with the VAS and GRC scores were -0.24 (P = 0.0001) and -0.32 (P = 0.0001), respectively. The correlation of this questionnaire with the other physiotherapy satisfaction questionnaire was good. Conclusions The Persian version of the 14 items on patient satisfaction with physical therapy questionnaire exhibited acceptable reliability and validity.

O objetivo deste trabalho é analisar a inserção do agronegócio brasileiro no comércio internacional de produtos agrícolas, buscando verificar o posicionamento geral do país e também dos produtos exportados, considerando o período compreendido entre o triênio 1999-2001 e o triênio 2009-2011. O desempenho é avaliado por meio da participação do país e de seus produtos nas importações mundiais. A Metodologia de análise é o modelo apresentado por Fajnzylber (1991). Os resultados mostram que o agronegócio brasileiro tem um posicionamento positivo, pois tanto a classificação global como também de parte significativa dos produtos de sua pauta de exportações se encontram em situação ótima, de acordo com o enfoque metodológico adotado., Revista de Ciências Agrárias, vol. 39 n.º 1 (2016)

Objective: To adapt an objective surgical assessment tool (OSATS) for adenotonsillectomy/tonsillectomy to Turkish., Methods: A prospective longitudinal validation study was planned. Adenoidectomy items were added to the tonsillectomy OSATS. Data were collected from 80 real-time observed surgical performances. To examine interrater reliability, 25 performances were simultaneously scored by two evaluators. In total, 105 test forms were filled, and reliability and validity studies were conducted., Results: Regarding reliability, Cronbach's alpha values were found to be >0.95, indicating very high internal consistency. For interrater reliability, a statistically significant positive correlation was found between the two evaluators' task-specific checklist (TSC) (p=0.0001; r=0.969) and global rating checklist (GRC) (p=0.0001; r=0.957). Between the TSC and GRC, there was a statistically significant positive correlation (p=0.0001; r=0.951). A model was developed using the generalizability theory, and the highest variances belonged to surgical processes. Regarding validity, the trainers' scores were significantly higher than the trainees' scores (p=0.0001). As surgical experience increased, both the TSC and the GRC scores significantly increased (p=0.0001). A trainee's first-ever 34 adenotonsillectomy performances were scored. A scatter plot of the obtained data showed that the trainee had gained more scores as the number of operations increased., Conclusion: The results indicate that adenotonsillectomy OSATS is a reliable and valid tool for the assessment of residents' surgical skills. This is the first OSATS developed for adenotonsillectomy in the Turkish literature. It is ready to be used in the operating room to provide constructive feedback and monitor the development of trainees' surgical skills in adenotonsillectomy., C1 [Kara, Cuneyt Orhan; Mengi, Erdem; Tumkaya, Funda; Ardic, Fazil Necdet] Pamukkale Univ, Dept Otorhinolaryngol, Sch Med, Denizli, Turkey., [Senol, Hande] Pamukkale Univ, Dept Biostat, Sch Med, Denizli, Turkey.

In healthcare, developing high procedural skill levels through training is a key factor for obtaining good clinical results on surgical procedures. Providing feedback to each student tailored to how the student has performed the procedure each time, improves the effectiveness of the training. Current state-of-the-art feedback relies on Checklists and Global Rating Scales to indicate whether all process steps have been performed and the quality of each execution step. However, there is a process perspective not successfully captured by those instruments, e.g., steps performed but in an undesired order, part of the process repeated an unnecessary number of times, or excessive transition time between steps. In this work, we propose a novel use of process mining techniques to effectively identify desired and undesired process patterns regarding rework, order, and performance, in order to complement the tailored feedback of surgical procedures using a process perspective. The approach has been effectively applied to analyze a real Central Venous Catheter installation training case. In the future, it is necessary to measure the actual impact of feedback on learning.

Simulation-based training can be used to evaluate communication and teamwork competencies among interdisciplinary teams and promote healthcare quality and safety (Rosen MA et al., Sim Health Care 3:33–41, 2008; Guise J et al., Sim Healthcare. 3:217–223, 2008; Goffman D et al., Semin perinatol. 37:140–142, 2013). Simulation has been used as an instructional design method to teach medical students, residents, and subspecialty fellows (Medical simulation in medical education: results of an AAMC survey. /https://www.aamc.org/download/259760/data/medical simulation in medical education an aamc survey.pdfS; Deering S et al., Semin Perinatol. 37:143–14, 2013). It is also recognized as a method of evaluating professional competence for practicing physician (Sidi A et al., J Grad Med Educ 489–494, 2014; Chiu M et al., Can J Anesth. 63:1357–1363, 2016). Simulation-based training is most effective when it is designed to successfully capture the knowledge, skills, and clinical reasoning that underlie expected behavior within the framework of the competencies assessed. Accurate and objective evaluation of trainee performance depends on the assessment method and the assessment tools selected. In this chapter, we will review a commonly recommended framework for evaluating the quality of an assessment tool. We end the chapter with examples of checklists and global rating scales, two common assessment tools that can be used during simulation-based training to determine if the expected clinical and procedural performance outcomes for learners have been achieved.

Rationale The ability to acquire the best evidence efficiently is an important competency for busy health-care professionals who must make decisions quickly. Aims and objectives We aimed to develop and validate a scale for measuring evidence-searching capability. Methods We first developed a scale for measuring evidence-searching capability by using the modified Delphi technique. Seven experts commented on a draft 33-item scale on a 5-point scale. All items rated less than three by any expert were removed. The items were modified or merged considering experts' feedback. When all items were rated greater than or equal to 3 by all experts with an interquartile range of less than or equal to 1, a consensus on the scale was achieved with the content validity constructed. We performed a pilot test and a formal test, and evaluated the inter-rater, intra-rater, and internal reliability by calculating the intra-cluster correlation coefficient (ICC), kappa correlation coefficient, and Cronbach's α. Results We developed a scale consisted of 15 items and a global rating score, with an average scale-content validity index of 0.98. In the pilot test, the ICC for inter-rater and intra-rater reliability was 0.91 and 1, respectively. The Cronbach's α was 0.90. In the formal test, the ICC for inter-rater reliability ranged from 0.61 to 1 and the weighted kappa correlation coefficient ranged from 0.27 to 1, with a Cronbach's α of 0.97. Conclusions This study is the first to develop and validate a scale for measuring evidence-searching skills through a systematic approach. The scale is composed of 15 items and a global rating score that can be easily used in objective assessment of knowledge-acquiring ability.

Background Though peer review is at the heart of scholarly publishing, peer review reports are not commonly investigated. We aimed to analyse the quality and structure of review reports submitted to the Indian Journal of Psychological Medicine (IJPM). Methods We systematically analysed the structure, tone, and quality of peer review reports of all original articles submitted to the journal between January 1, 2018 to May 15, 2020. Quality assessment was done using the 8-item Review Quality Instrument (RQI). Results A total of 527 review reports from 291 original articles were analysed. More than two-thirds of review reports were provided as inline comments (n = 368, 69.8 %). Most of the review reports were not well-structured; only a few provided a summary (n = 64, 13.2 %) or divided the comments into major and minor ones (n = 12, 2.5 %). Nearly a quarter had negative wordings (n = 117, 24.1 %) and a minority had a frankly unprofessional tone (n = 43, 8.8 %). The global rating was “poor” (n = 266, 50.5 %) or “below average” (n = 203, 38.5 %) for most reports. Conclusion Most of the peer reviews submitted to the IJPM were not structured and obtained low scores on the RQI domains. Concerted efforts are needed to improve the quality of peer reviews and to provide training for reviewers.