Search

AbstractObjectiveMethodResultsConclusions\nHIGHLIGHTSSuicide risk assessments are obligatory when patients express a death wish in clinical practice. Yet, suicide risk estimates based on unguided risk assessments have been shown to be of low reliability. Since generalizability of previous studies is limited, the current study aimed to assess inter-rater and intra-rater reliability of risk estimates conducted by psychotherapists and psychology students using written case vignettes.In total, N = 256 participants (psychology students, psychotherapists) were presented with 24 case vignettes describing patients at either low, moderate, severe or extreme risk of suicide. Participants were asked to assign a level of risk to each single vignette at a baseline assessment and again at a follow-up assessment two weeks later.Risk estimates showed a low inter-rater reliability, both for students (AC1 = .35) and for psychotherapists (AC1 = .44). Intra-rater reliability was moderate for psychotherapists (AC1 = .59) and rather low for psychology students (AC1 = .47). In general, intra- and intra-rater reliability were highest for vignettes displaying “low” and “extreme” risk.The results highlight that the reliability of unguided suicide risk assessments is questionable. Standardized risk assessment protocols are therefore recommended. Nonetheless, even reliable risk estimation does not imply predictive validity of risk estimates for future suicidal behavior.Suicide risk estimates have been shown to be of low reliabilitySuicide risk estimates by psychotherapists and students also showed low inter-rater and intra-rater reliability in the current studyReliable risk estimation does not imply predictive validity of risk estimates for future suicidal behavior [ABSTRACT FROM AUTHOR]

AbstractAimMethodsResultsConclusionThe Test of Gross Motor Development Third Edition (TGMD-3) is used to assess the development of fundamental movement skills in children from 3 to 10 years old. This study aimed to evaluate the intra-rater, inter-rater, and test-retest reliability and to determine the minimal detectable change (MDC) value of the TGMD-3 in children with developmental coordination disorder (DCD).The TGMD-3 was administered to 20 children with DCD. The child’s fundamental movement skills were recorded using a digital video camera. Reliability was assessed at two occasions by three raters using the generalizability theory.The TGMD-3 demonstrates good inter-rater reliability for the locomotor skills subscale, the ball skills subscale, and the total score (φ = 0.77 − 0.91), while the intra-rater reliability was even higher (φ = 0.94 − 0.97). Test-retest reliability was also shown to be good (φ = 0.79-0.93). The MDC95 was determined to be 10 points.This study provides evidence that the TGMD-3 is a reliable test when used to evaluate fundamental movement skills in children with DCD and suggests that an increase of 10 points represents a significant change in the motor function of a child with DCD. [ABSTRACT FROM AUTHOR]

Introduction: Many assessment methods are used in physiotherapy to analyze the fitness level and injury risk in athletes, and to determine the general health status and the effectiveness of the treatment applied in patients. Considering the need for telehealth use, it is essential to determine the usability of tests performed as tele-assessment. This study aimed to examine the intra-rater reliability, validity, and feasibility of the tele-assessment version of core strength and endurance and functional capacity assessments in healthy individuals. Methods: "Curl-up," "Modified Push-up," "Plank," and "Lateral Bridge" tests were used for core strength and endurance performance assessment, and "Timed Up and Go," "30 second Sit to Stand," and "Functional Reach Test" tests were used for functional capacity assessment in healthy individuals. Participants were evaluated first by tele-assessment, then one hour later on the same day, all assessments were repeated face-to-face procedures by the researchers thus the validity of the tele-assessment method was determined. All tests were applied as tele-assessment one week later to determine the intra-rater reliability of the tele-assessment method. The system usability scale was applied to evaluate the usability of our tele-assessment method. Results: Eighty healthy people were enrolled. Intraclass correlation coefficients ranged between 0.91 and 0.97 for core performance tests and between 0.95 and 0.97 for functional tests. All tele-assessment versions of the core performance and functional tests were highly correlated with the face-to-face versions. Discussion: Core strength-endurance and functional tests performed via tele-assessment were reliable, valid, and feasible for practically measuring the performance of healthy young adults. This study supports the tele-assessment versions of these tests. Trial registration number and trial register: Clinicaltrials.gov: NCT04899804 [ABSTRACT FROM AUTHOR]

Background The modified Thomas test (MTT) is commonly used to assess the flexibility of hip musculature, including the iliopsoas, rectus femoris, and tensor fascia latae. This measurement is important to include in a comprehensive musculoskeletal examination. However, existing research shows conflicting results regarding its reliability, particularly due to variations in controlling pelvic tilt during testing, which may lead to inaccurate measurements of hip extension when quantifying the test outcomes. Purpose/Hypothesis This study aimed to evaluate the intra- and inter-rater reliability of the Modified Thomas Test (MTT) in assessing hip flexor length using a goniometer. It was hypothesized that controlling for pelvic tilt would enhance the reliability of these measurements. Study Design Intra- and inter-rater reliability study Methods Sixty-four healthy individuals were recruited to participate in this study. The MTT was performed twice on each leg by both an experienced and a student physical therapist. Blinded goniometric measurements for hip extension range of motion (ROM) in the MTT position were taken with neutral pelvic tilt being enforced via palpation. A double-blind protocol was used where both examiners were unaware of each other's measurements and the goniometer was covered to blind the measuring therapist to the values as well. ROM values were entered into a Microsoft Excel spreadsheet and quantified using SPSS software. Statistical analysis included calculating Intraclass Correlation Coefficients (ICCs) and Standard Errors of Measurement (SEMs) using SPSS software. Results The study included 64 participants (mean age = 23.7 ± 4.34 years). The MTT demonstrated high intra-rater reliability (ICC = 0.911) and inter-rater reliability (ICC = 0.851). The SEMs indicated minimal variability around the mean scores. The average hip extension ROM measured was 5.43± 9.73 degrees. Conclusion These results suggest that the MTT is a reliable tool for assessing hip flexor length in clinical practice, particularly when pelvic tilt is controlled. These results have important implications for accurately testing orthopedic limitations that can contribute to low back, hip, and knee pain. Level of Evidence 3 [ABSTRACT FROM AUTHOR]

Abstract The study assessed the impact of procedural errors on the remote dielectric sensing system (ReDS), a non-invasive lung fluid assessment technology, in an Asian cohort. Healthy volunteers underwent ReDS measurements following manufacturer’s instructions, with two consecutive measurements one minute apart. A subset of 20 participants had modified procedure settings. Reliability was measured using intraclass correlation coefficient (ICC). The study included 86 healthy volunteers, and all ReDS measurements fell within the recommended normal range. The intra-rater reliability of ReDS measurements was excellent, with an ICC of 0.968. Among the subset of 20 subjects, deviations in height and weight did not significantly affect ReDS values. However, deviations in chest size by ± 3 cm had a noticeable impact on ReDS measures, and incorrect station selection led to fluctuations in ReDS readings. In conclusion, the ReDS system demonstrated excellent intra-rater reliability and applicability in an Asian cohort. Procedural errors, such as chest size measurement and station selection, significantly influenced ReDS measurements. Adherence to standardized operating procedures is crucial to ensure accurate and consistent results. These findings highlight the importance of adherence to manufacturer instructions when utilizing ReDS for lung fluid assessment, thereby enhancing its reliability and clinical applicability.

Background: Chronic leg ulcers present a global challenge in healthcare, necessitating precise wound measurement for effective treatment evaluation. This study is the first to validate the "split-wound design" approach for wound studies using objective measures. We further improved this relatively new approach and combined it with a semi-automated wound measurement algorithm. Method: The algorithm is capable of plotting an objective halving line that is calculated by splitting the bounding box of the wound surface along the longest side. To evaluate this algorithm, we compared the accuracy of the subjective wound halving of manual operators of different backgrounds with the algorithm-generated halving line and the ground truth, in two separate rounds. Results: The median absolute deviation (MAD) from the ground truth of the manual wound halving was 2% and 3% in the first and second round, respectively. On the other hand, the algorithm-generated halving line showed a significantly lower deviation from the ground truth (MAD = 0.3%, p < 0.001). Conclusions: The data suggest that this wound-halving algorithm is suitable and reliable for conducting wound studies. This innovative combination of a semi-automated algorithm paired with a unique study design offers several advantages, including reduced patient recruitment needs, accelerated study planning, and cost savings, thereby expediting evidence generation in the field of wound care. Our findings highlight a promising path forward for improving wound research and clinical practice. [ABSTRACT FROM AUTHOR]

Purpose: To investigate power Doppler (PD) activity and tendon structure (between the injured and contralateral limb) in patients with unilateral patellar tendinopathy (PT) using ultrasonography (US). Secondly, the aim was to determine the intra‐rater reliability of the PD activity and tendon structure. Methods: This study analyzed US baseline data from 57 male participants with symptomatic unilateral PT who had been enrolled in one of two randomized clinical trials. Data were analyzed to examine if systematic differences existed between injured and contralateral limbs using Fiji ImageJ. Results: The PD activity of the symptomatic tendon was larger 25.6 (Q1 = 14.9; Q3 = 41.6) mm2 than the asymptomatic 0 (Q1 = 0.0; Q3 = 0.0) mm2 (p < 0.001). There was a significantly greater tendon thickness at the proximal (2.5 mm 95% CI [2.0; 3.0]), mid (0.8 mm 95% CI [0.5; 1.1]), and distal (0.2 mm 95% CI [0.1; 0.4]) part of the tendon for the symptomatic compared to the asymptomatic tendon. Intra‐rater reliability for PD activity and tendon structure ranged from moderate‐to‐excellent (0.74; 0.99). Conclusion: These results provide mean estimates for tendon thickness of symptomatic and asymptomatic tendons, that can be used for clinicians to reliably estimate pathological tendon thickness. [ABSTRACT FROM AUTHOR]

Background and Objectives: Muscle properties are critical for performance and injury risk, with changes occurring due to physical exertion, aging, and neurological conditions. The MyotonPro device offers a non-invasive method to comprehensively assess muscle biomechanical properties. This systematic review evaluates the reliability of MyotonPro across various muscles for diagnostic purposes. Materials and Methods: Following PRISMA guidelines, a comprehensive literature search was conducted in Medline (PubMed), Ovid (Med), Epistemonikos, Embase, Cochrane Library, Clinical trials.gov, and the WHO International Clinical Trials platform. Studies assessing the reliability of MyotonPro across different muscles were included. A methodological quality assessment was performed using established tools, and reviewers independently conducted data extraction. Statistical analysis involved summarizing intra-rater and inter-rater reliability measures across muscles. Results: A total of 48 studies assessing 31 muscles were included in the systematic review. The intra-rater and inter-rater reliability were consistently high for parameters such as frequency and stiffness in muscles of the lower and upper extremities, as well as other muscle groups. Despite methodological heterogeneity and limited data on specific parameters, MyotonPro demonstrated promising reliability for diagnostic purposes across diverse patient populations. Conclusions: The findings suggest the potential of MyotonPro in clinical assessments for accurate diagnosis, treatment planning, and monitoring of muscle properties. Further research is needed to address limitations and enhance the applicability of MyotonPro in clinical practice. Reliable muscle assessments are crucial for optimizing treatment outcomes and improving patient care in various healthcare settings. [ABSTRACT FROM AUTHOR]

Background: An essential factor in the validity of motor evoked potential (MEP)s recorded by transcranial magnetic stimulation (TMS) over multiple times is their test-retest reliability which to a large extent depends on the accuracy and competence of the assessor (intra-rater reliability). However, intra-rater reliability is infrequently reported in TMS studies suggesting that this is rarely done. Objectives: This study was conducted to determine the intra-rater within and between-session reliability of a newly trained TMS assessor prior to a main TMS study and report on the methodology used to encourage similar practice. Methods: Fourteen (10 males, 4 females; mean age: 32 ± 5.8 years) participants took part in the study. Motor evoked potentials were elicited from a relaxed, right first dorsal interosseous (FDI) muscle three times (T1, T2 and T3) across two testing sessions at least 48 hours apart. During the first session, MEPs were recorded twice (T1 and T2) within an interval of 20 minutes to determine the within (intra) session reliability of the assessor. During the second session, a single measurement was carried out (T3) which was compared to T1 to determine the inter-session reliability. Results: Repeated measure analysis of variance (ANOVA) did not reveal significant difference in the amplitude of the MEPs obtained across the three time periods (P = 0.196) demonstrating agreement in the MEPs and hence the reliability of the assessor. Additionally, the intraclass correlation coefficient (ICC) between T1 and T2; and T1 and T3 were 0.952 (P < 0.001) and 0.833 (P = 0.001) respectively further indicating the within and between sessions reliability of the assessor. Conclusions: The agreement between the three measured MEPs amplitude and the significant ICC demonstrates the reliability of the assessor in this study to use TMS for research. We suggest that the intra-rater reliability of new TMS operators should be established using the methodology in this report prior to main TMS studies. [ABSTRACT FROM AUTHOR]

Abstract Ensuring consistent and reliable scoring is paramount in education, especially in performance-based assessments. This study delves into the critical issue of marking consistency, focusing on speaking proficiency tests in English language learning, which often face greater reliability challenges. While existing literature has explored various methods for assessing marking reliability, this study is the first of its kind to introduce an alternative statistical tool, namely the gauge repeatability and reproducibility (GR&R) approach, to the educational context. The study encompasses both intra- and inter-rater reliabilities, with additional validation using the intraclass correlation coefficient (ICC). Using a case study approach involving three examiners evaluating 30 recordings of a speaking proficiency test, the GR&R method demonstrates its effectiveness in detecting reliability issues over the ICC approach. Furthermore, this research identifies key factors influencing scoring inconsistencies, including group performance estimation, work presentation order, rubric complexity and clarity, the student’s chosen topic, accent familiarity, and recording quality. Importantly, it not only pinpoints these root causes but also suggests practical solutions, thereby enhancing the precision of the measurement system. The GR&R method can offer significant contributions to stakeholders in language proficiency assessment, including educational institutions, test developers and policymakers. It is also applicable to other cases of performance-based assessments. By addressing reliability issues, this study provides insights to enhance the fairness and accuracy of subjective judgements, ultimately benefiting overall performance comparisons and decision making.

Ensuring consistent and reliable scoring is paramount in education, especially in performance-based assessments. This study delves into the critical issue of marking consistency, focusing on speaking proficiency tests in English language learning, which often face greater reliability challenges. While existing literature has explored various methods for assessing marking reliability, this study is the first of its kind to introduce an alternative statistical tool, namely the gauge repeatability and reproducibility (GR&R) approach, to the educational context. The study encompasses both intra- and inter-rater reliabilities, with additional validation using the intraclass correlation coefficient (ICC). Using a case study approach involving three examiners evaluating 30 recordings of a speaking proficiency test, the GR&R method demonstrates its effectiveness in detecting reliability issues over the ICC approach. Furthermore, this research identifies key factors influencing scoring inconsistencies, including group performance estimation, work presentation order, rubric complexity and clarity, the student's chosen topic, accent familiarity, and recording quality. Importantly, it not only pinpoints these root causes but also suggests practical solutions, thereby enhancing the precision of the measurement system. The GR&R method can offer significant contributions to stakeholders in language proficiency assessment, including educational institutions, test developers and policymakers. It is also applicable to other cases of performance-based assessments. By addressing reliability issues, this study provides insights to enhance the fairness and accuracy of subjective judgements, ultimately benefiting overall performance comparisons and decision making. [ABSTRACT FROM AUTHOR]

Purpose: To evaluate the effect of EVO+ Visian Implantable Collamer Lens (ICL) implantation on the iridocorneal angle pigmentation and structures. Methods: Myopic refractive surgery candidates who underwent posterior chamber EVO+ ICL implantation were evaluated preoperatively and 3 and 6 months postoperatively. High-resolution images of the iridocorneal angle (nasal, superior, temporal and inferior quadrants) were acquired during gonioscopy. A masked observer, blinded to study visits and patients' information, evaluated the angle width, apparent iris root insertion, iris configuration, and trabecular meshwork pigmentation according to the Spaeth scale in two different occasions. The intra-rater reliability was estimated using the weighted Gwet's Agreement Coefficient (AC2). Differences between visits were analyzed using the Cochran Q test or the Friedman test. Results: Twenty-one patients (13 females and 8 males) aging 31.3 ± 6.3 years old were recruited. The intra-rater reliability of gonioscopy assessment was excellent for every parameter assessed (AC2 ≥ 0.97). No significant differences were found among the study visits for any quadrant in the angle width (p ≥ 0.74), apparent iris root insertion (p ≥ 0.22), iris configuration (p ≥ 0.21) and trabecular meshwork pigmentation (p ≥ 0.24). Mean pigmentation of trabecular meshwork for the four quadrants did not vary either among visits (p = 0.25). Conclusions: The EVO+ ICL implantation after uneventful procedures appears not to clinically affect the iridocorneal angle pigmentation and/or structures during a short-medium follow-up using gonioscopy assessment. The intra-rater reliability of glaucoma specialists assessing iridocorneal angle structures using gonioscopy images is very high. [ABSTRACT FROM AUTHOR]

The assessment of music performance by experts is vital for the measurement of performers' achievements both in the course of their training and in the context of competitions. More than 350 international piano competitions are held every year, in which the main task of the jury is to judge performances as they take place in the moment, without taking into account any other information such as performers' previous competition successes. A vast body of evidence shows that many factors influence such judgments. In this article, I discuss these factors, as first identified in early and comparatively little-known research by Maria Manturzewska, the first Polish music psychologist (1930–2020). The principal purpose of her studies was to examine the inter- and intra-rater reliability of judges' scores in the 6th International Fryderyk Chopin Piano Competition (IFCPC), held in Warsaw in 1960, and in an experiment in which the judges were, respectively, 28 competition jurors and a panel of 10 experts (2 competition jurors, a participant in the competition, and 8 further outstanding pianists). Based on the analysis of the jurors' scores in the competition and those of the experts, Manturzewska found low values for both inter- and intra-rater reliability: different judges assessed the same performances in different ways, as did experts hearing the same performance on two occasions. Manturzewska proposed several explanations for the discrepancies between judges' assessments. Not only have her students and collaborators continued her work by testing and verifying some of her hypotheses, but more recent research conducted outside Poland has also done so. In conclusion, although some kinds of objective measurement can be made using new tools and technologies, human judges with their own competences, ability to evaluate, value systems, preferences, and a sense of fairness and morality will always be needed for reliable and accurate assessments. [ABSTRACT FROM AUTHOR]

Introduction: The Violent Extremism Risk Assessment - Version 2 Revised (VERA-2R) is an evidence-based structured professional judgement (SPJ) tool for ideologically motivated violence. Use of the tool can help professionals in risk assessment and risk management of suspected and convicted terrorists and violent extremists at different stages within the criminal process. It is important that the tool leads to reliable and valid risk assessments. Methods: The present study aimed to establish the reliability of the VERA-2R, focusing on the interrater- and intrarater reliability. Trained researchers assessed the interrater- and intrarater reliability, respectively in 30 and 33 cases of Dutch convicted terrorist offenders, on the basis of extensive judicial files. Results: In general, the average amount of agreement on the indicators and structured risk judgements can be classified as good to excellent, for both the interrater- and intra-rater reliability. However, six indicators were found to have low reliability. Above clarifying the reliability of the VERA-2R, this study also showed how the interrater- and intrarater reliability of a SPJ tool can be investigated with trained assessors based on judicial files. This can be of added value, because existing reliability studies often use case vignettes, have a small sample size and/or do not include a stringent training program. Discussion: With this research, we can make more substantiated recommendations on how the VERA-2R can be improved. This could lead to more accurate risk assessments and risk management strategies. However, in order to develop a more reliable and valid VERA-2R instrument, the remaining psychometric properties must also be investigated and published. That will be discussed in a future article. [ABSTRACT FROM AUTHOR]

Abstract Background In research designs that rely on observational ratings provided by two raters, assessing inter-rater reliability (IRR) is a frequently required task. However, some studies fall short in properly utilizing statistical procedures, omitting essential information necessary for interpreting their findings, or inadequately addressing the impact of IRR on subsequent analyses’ statistical power for hypothesis testing. Methods This article delves into the recent publication by Liu et al. in BMC Cancer, analyzing the controversy surrounding the Kappa statistic and methodological issues concerning the assessment of IRR. The primary focus is on the appropriate selection of Kappa statistics, as well as the computation, interpretation, and reporting of two frequently used IRR statistics when there are two raters involved. Results The Cohen’s Kappa statistic is typically utilized to assess the level of agreement between two raters when there are two categories or for unordered categorical variables with three or more categories. On the other hand, when it comes to evaluating the degree of agreement between two raters for ordered categorical variables comprising three or more categories, the weighted Kappa is a widely used measure. Conclusion Despite not substantially affecting the findings of Liu et al.?s study, the statistical dispute underscores the significance of employing suitable statistical methods. Rigorous and accurate statistical results are crucial for producing trustworthy research.

Background and Objectives: Muscle properties are critical for performance and injury risk, with changes occurring due to physical exertion, aging, and neurological conditions. The MyotonPro device offers a non-invasive method to comprehensively assess muscle biomechanical properties. This systematic review evaluates the reliability of MyotonPro across various muscles for diagnostic purposes. Materials and Methods: Following PRISMA guidelines, a comprehensive literature search was conducted in Medline (PubMed), Ovid (Med), Epistemonikos, Embase, Cochrane Library, Clinical trials.gov, and the WHO International Clinical Trials platform. Studies assessing the reliability of MyotonPro across different muscles were included. A methodological quality assessment was performed using established tools, and reviewers independently conducted data extraction. Statistical analysis involved summarizing intra-rater and inter-rater reliability measures across muscles. Results: A total of 48 studies assessing 31 muscles were included in the systematic review. The intra-rater and inter-rater reliability were consistently high for parameters such as frequency and stiffness in muscles of the lower and upper extremities, as well as other muscle groups. Despite methodological heterogeneity and limited data on specific parameters, MyotonPro demonstrated promising reliability for diagnostic purposes across diverse patient populations. Conclusions: The findings suggest the potential of MyotonPro in clinical assessments for accurate diagnosis, treatment planning, and monitoring of muscle properties. Further research is needed to address limitations and enhance the applicability of MyotonPro in clinical practice. Reliable muscle assessments are crucial for optimizing treatment outcomes and improving patient care in various healthcare settings.

Background: In research designs that rely on observational ratings provided by two raters, assessing inter-rater reliability (IRR) is a frequently required task. However, some studies fall short in properly utilizing statistical procedures, omitting essential information necessary for interpreting their findings, or inadequately addressing the impact of IRR on subsequent analyses' statistical power for hypothesis testing. Methods: This article delves into the recent publication by Liu et al. in BMC Cancer, analyzing the controversy surrounding the Kappa statistic and methodological issues concerning the assessment of IRR. The primary focus is on the appropriate selection of Kappa statistics, as well as the computation, interpretation, and reporting of two frequently used IRR statistics when there are two raters involved. Results: The Cohen's Kappa statistic is typically utilized to assess the level of agreement between two raters when there are two categories or for unordered categorical variables with three or more categories. On the other hand, when it comes to evaluating the degree of agreement between two raters for ordered categorical variables comprising three or more categories, the weighted Kappa is a widely used measure. Conclusion: Despite not substantially affecting the findings of Liu et al.?s study, the statistical dispute underscores the significance of employing suitable statistical methods. Rigorous and accurate statistical results are crucial for producing trustworthy research. [ABSTRACT FROM AUTHOR]

IntroductionThe Violent Extremism Risk Assessment - Version 2 Revised (VERA-2R) is an evidence-based structured professional judgement (SPJ) tool for ideologically motivated violence. Use of the tool can help professionals in risk assessment and risk management of suspected and convicted terrorists and violent extremists at different stages within the criminal process. It is important that the tool leads to reliable and valid risk assessments.MethodsThe present study aimed to establish the reliability of the VERA-2R, focusing on the interrater- and intrarater reliability. Trained researchers assessed the interrater- and intrarater reliability, respectively in 30 and 33 cases of Dutch convicted terrorist offenders, on the basis of extensive judicial files.ResultsIn general, the average amount of agreement on the indicators and structured risk judgements can be classified as good to excellent, for both the interrater- and intra-rater reliability. However, six indicators were found to have low reliability. Above clarifying the reliability of the VERA-2R, this study also showed how the interrater- and intrarater reliability of a SPJ tool can be investigated with trained assessors based on judicial files. This can be of added value, because existing reliability studies often use case vignettes, have a small sample size and/or do not include a stringent training program.DiscussionWith this research, we can make more substantiated recommendations on how the VERA-2R can be improved. This could lead to more accurate risk assessments and risk management strategies. However, in order to develop a more reliable and valid VERA-2R instrument, the remaining psychometric properties must also be investigated and published. That will be discussed in a future article.

Copyright of International Journal of Medicine & Science of Physical Activity & Sport / Revista Internacional de Medicina y Ciencias de la Actividad Física y del Deporte is the property of Revista Internacional de Medicina y Ciencias de la Actividad Fisica y del Deporte and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Background: Phase and step annotation in surgical videos is a prerequisite for surgical scene understanding and for downstream tasks like intraoperative feedback or assistance. However, most ontologies are applied on small monocentric datasets and lack external validation. To overcome these limitations an ontology for phases and steps of laparoscopic Roux-en-Y gastric bypass (LRYGB) is proposed and validated on a multicentric dataset in terms of inter- and intra-rater reliability (inter-/intra-RR). Methods: The proposed LRYGB ontology consists of 12 phase and 46 step definitions that are hierarchically structured. Two board certified surgeons (raters) with > 10 years of clinical experience applied the proposed ontology on two datasets: (1) StraBypass40 consists of 40 LRYGB videos from Nouvel Hôpital Civil, Strasbourg, France and (2) BernBypass70 consists of 70 LRYGB videos from Inselspital, Bern University Hospital, Bern, Switzerland. To assess inter-RR the two raters' annotations of ten randomly chosen videos from StraBypass40 and BernBypass70 each, were compared. To assess intra-RR ten randomly chosen videos were annotated twice by the same rater and annotations were compared. Inter-RR was calculated using Cohen's kappa. Additionally, for inter- and intra-RR accuracy, precision, recall, F1-score, and application dependent metrics were applied. Results: The mean ± SD video duration was 108 ± 33 min and 75 ± 21 min in StraBypass40 and BernBypass70, respectively. The proposed ontology shows an inter-RR of 96.8 ± 2.7% for phases and 85.4 ± 6.0% for steps on StraBypass40 and 94.9 ± 5.8% for phases and 76.1 ± 13.9% for steps on BernBypass70. The overall Cohen's kappa of inter-RR was 95.9 ± 4.3% for phases and 80.8 ± 10.0% for steps. Intra-RR showed an accuracy of 98.4 ± 1.1% for phases and 88.1 ± 8.1% for steps. Conclusion: The proposed ontology shows an excellent inter- and intra-RR and should therefore be implemented routinely in phase and step annotation of LRYGB. [ABSTRACT FROM AUTHOR]

Featured Application: The presented team handball PlayerScore can be applied to determine the individual player performance in team handball competition. In team handball, the individual match performance of each player is essential for winning games; however, a validated match analysis system is still lacking. Consequently, the aim of the study was to justify (1) the different relevant variables and their scoring within the individual match analysis (PlayerScore), (2) to determine the intra-rater reliability and validity of the PlayerScore, and (3) to determine the influence of the rater in relation to their degree of expertise level. Six games (three games each of Spain and Brazil, one game twice) of the 2021 World Championship were analyzed by six different raters. The PlayerScore was calculated for each field player of Spain and Brazil in all seven analyzed games. We found a high intra-rater reliability (ICC = 0.97) for the two rated games (Spain against Germany), a highly significant difference (p < 0.001) between the summarized team PlayerScore of Spain and Brazil, as well as significant differences (p < 0.001) for the factor "game" and "rater" (p < 0.05), but no significant interaction for "game × rater" (p = 0.90) in the two-way repeated measures ANOVA. We conclude that the PlayerScore is a reliable and valid rating tool to determine the individual players' performance in team handball; however, the raters should have sufficient experience in the different techniques and tactics in team handball. [ABSTRACT FROM AUTHOR]

Anteroposterior (AP) alignment assessment for nondisplaced femoral neck fractures is important for determining the treatment strategy and predicting postoperative outcomes. AP alignment is generally measured using the Garden alignment index (GAI). However, its reliability remains unknown. We compared the reliability of GAI and a new AP alignment measurement (valgus tilt measurement [VTM]) using preoperative AP radiographs of nondisplaced femoral neck fractures. The study was designed as an intra- and inter-rater reliability analysis. The raters were four trauma surgeons who assessed 50 images twice. The main outcome was the intraclass correlation coefficient (ICC). To calculate intra- and inter-rater reliability, we used a mixed-effects model considering rater, patient, and time. The overall ICC (95% CI) of GAI and VTM for intra-rater reliability was 0.92 (0.89–0.94) and 0.86 (0.82–0.89), respectively. The overall ICC of GAI and VTM for inter-rater reliability was 0.92 (0.89–0.95), and 0.85 (0.81–0.88), respectively. The intra- and inter-rater reliability of GAI was higher in patients aged <80 years than in patients aged ≥80 years. Our results showed that GAI is a more reliable measurement method than VTM, although both are reliable. Variations in patient age should be considered in GAI measurements. [ABSTRACT FROM AUTHOR]

Background: The goal of this study was to assess the reliability of electromyography and range of motion measurements obtained using a knee exercise monitoring system. This device was developed to collect data on knee exercise activities. Methods: Twenty healthy individuals performed isotonic quadriceps exercises in this study. The vastus medialis surface electromyography (sEMG) and range of motion (ROM) of the knee were recorded during the exercise using the isotonic knee exercise monitoring device, the Mobi6-6b, and a video camera system. Each subject underwent a second measuring session at least 24 h after the first session. To determine reliability, the intraclass correlation coefficients (ICCs) and standard error of measurement (SEM) at the 95% confidence interval were calculated, and a Bland-Altman analysis was performed. Results: For inter-rater reliability, the ICCs of the mean absolute value (MAV) and root mean square (RMS) of sEMG were 0.73 (0.49, 0.86) and 0.79 (0.61, 0.89), respectively. ROM had an ICC of 0.93 (0.02, 0.98). The intra-rater reliability of the MAV of the sEMG was 0.89 (0.71, 0.96) and the intra-rater reliability of RMS of the sEMG was 0.88 (0.70, 0.95). The ROM between days had an intra-rater reliability of 0.82 (0.54, 0.93). The Bland-Altman analysis demonstrated no systematic bias in the MAV and RMS of sEMG, but revealed a small, systematic bias in ROM (-0.8311 degrees). Conclusion: For sEMG and range of motion measures, the isotonic knee exercise monitoring equipment revealed moderate to excellent inter- and intra-rater agreement. However, the confidence interval of ROM inter-rater reliability was quite large, indicating a small agreement bias; hence, the isotonic knee exercise monitor may not be suitable for measuring ROM. This isotonic knee exercise monitor could detect and collect information on a patient's exercise activity for the benefit of healthcare providers. [ABSTRACT FROM AUTHOR]

COVID-19 is associated with musculoskeletal disorders. Ultrasound is a tool to assess muscle architecture and tendon measurements, offering an idea of the proportion of the consequences of the disease, since significant changes directly reflect the reduction in the ability to produce force and, consequently, in the functionality of the patient; however, its application in post-COVID-19 infection needs to be determined. We aimed to assess the intra- and inter-rater reliability of ultrasound measures of the architecture of the vastus lateralis (VL), rectus femoris (RF), vastus medialis (VM), gastrocnemius lateralis (GL), gastrocnemius medialis (GM), soleus (SO), and tibialis anterior (TA) muscles, as well as the patellar tendon (PT) cross-sectional area (CSA) in post-COVID-19 patients. An observational, prospective study with repeated measures was designed to evaluate 20 post-COVID-19 patients, who were measured for the pennation angle (θp), fascicular length (Lf), thickness, echogenicity of muscles, CSA and echogenicity of the PT. The intra-class correlation coefficient (ICC) and 95% limits of agreement were used. The intra-rater reliability presented high or very high correlations (ICC = 0.71–1.0) for most measures, except the θp of the TA, which was classified as moderate (ICC = 0.69). Observing the inter-rater reliability, all the evaluations of the PT, thickness and echogenicity of the muscles presented high or very high correlations. For the Lf, only the RF showed as low (ICC = 0.43), for the θp, RF (ICC = 0.68), GL (ICC = 0.70) and TA (ICC = 0.71) moderate and the SO (ICC = 0.40) low. The ultrasound reliability was acceptable for the muscle architecture, muscle and tendon echogenicity, and PT CSA, despite the low reliability for the Lf and θp of the RF and SO, respectively. [ABSTRACT FROM AUTHOR]

Abstract Background Given the potential benefits of introducing ultrasound in the clinical assessment of muscle disorders, this study aimed to assess the feasibility and reliability of measuring forearm muscle thickness by ultrasound in a geriatric clinical setting. Methods Cross-sectional pilot study in 25 participants (12 patients aged ≥ 70 years in an acute geriatric ward and 13 healthy volunteers aged 25–50 years), assessed by three raters. Muscle thickness measurement was estimated as the distance between the subcutaneous adipose tissue-muscle interface and muscle-bone interface of the radius at 30% proximal of the distance between the styloid process and distal insertion of the biceps brachii muscle of the dominant forearm. Examinations were repeated three times by each rater and intra- and inter-rater reliability was calculated. Feasibility analysis included consideration of technological, economic, legal, operational, and scheduling (TELOS) components. Results Mean muscle-thickness measurement difference between groups was 4.4 mm (95% confidence interval [CI] 2.4 mm to 6.3 mm], p

Health professionals like physiotherapists require additional choices in portable dynamometers to conduct evaluations pre- and post-intervention in order to accurately measure the efficacy of treatments and patient progression and to adjust rehabilitation goals. New dynamometers have arrived on the market, but there is no evidence for performance of the Kinvent. This study aimed to investigate intra- and inter-rater reliability (ICC one-or-two-way-random-model/Bland–Altman) of the Kinvent hand-held dynamometer in the muscle groups of the lower and upper limbs (COSMIN guidelines). The Kinvent showed a good to excellent intra- and inter-rater reliability for almost all the upper and lower limbs movements assessed. Ankle dorsiflexion was moderate in all assessments.

The visual scoring of gold standard polysomnography (PSG) is known to present inter- and intra-scorer variability. Previously, Somno-Art Software, a cardiac based sleep scoring algorithm, has been validated in comparison to 2 expert visual PSG scorers. The goal of this research is to evaluate the performances of the algorithm against a pool of scorers. Sixty PSG and actimetry recording nights, representative of clinical practice (healthy subjects and patients suffering from obstructive sleep apnea [OSA], insomnia or major depressive disorder), were scored by 5 different sleep scoring centers and by the Somno-Art Software. Intra-class correlation coefficient (ICC) and Wilcoxon Signed-Rank Test were calculated between each scorer and the average value of the 6 scorers, including Somno-Art Software. In addition, epoch-by-epoch agreement between scorers were analyzed. Somno-Art Software estimation of sleep efficiency, wake, N1+N2, N3 and REM sleep fit within the interscorer range for the full dataset and the subgroups, except for underestimating N3 sleep in OSA patients. Additionally, Somno-Art Software overestimated sleep latency compared to the average scoring for insomniacs (+4.7 ± 1.6min). On the full dataset, Somno-Art Software had good (0.75 < ICC<0.90) or excellent (ICC>0.90) ICC scores for all sleep parameters except N3 sleep (moderate score, 0.50 < ICC<0.75). For the 4-stages epoch-by-epoch agreement, Somno-Art Software was slightly below that of the visual scorers except for the healthy sub-group where an overlap was demonstrated. Somno-Art Software sleep scoring shows a good interscorer reliability in the range of the 5 visual polysomnography scorers. [ABSTRACT FROM AUTHOR]

Background  The craniovertebral angle (CV angle) measurement is a convenient and easy clinical method for the cervical posture assessment. "Markus Bader (MB) ruler software" assists to measure angles and distances on the computer screen. The MB-ruler protractor is almost transparent; hence, it can be easily used on the computer screen. Many studies in physiotherapy have used this software for postural assessment with the photographic method, but the reliability of this software is not available to our knowledge in the Indian population. Aim  The aim of this study was to find inter-rater and intra-rater reliabilities of "MB-Ruler software", by using the photographic method to measure the CV angle for the cervical postural assessment. Study Design  This is an observational study in healthy young population. Subjects and Method  Total 30 young student volunteers with no clinical symptoms were enrolled for this study. A single standard lateral view image in a sitting position was used for CV angle measurement with "MB ruler software." Three readings of the CV angle were taken by three different observers (A, B, and C) separately at intervals of time. The CV angle measured by three observers was compared. Results  Cronbach's ɑ coefficient value for intra-rater (ICC 0.999) (intra-class correlation coefficient) as well as inter-rater (ICC 0.892) reliability for measuring CV angle was very high. Conclusion  MB ruler demonstrates high intra-rater and inter-rater reliabilities and can be used for the photographic evaluation of posture by assessing the CV angle. [ABSTRACT FROM AUTHOR]

• Scapular mobility is uniquely challenging to assess and currently there is no reliable goniometric technique described in the literature to measure scapular protraction and retraction. • The current study examined intra and inter-rater reliability of a goniometric technique using the superior angle and acromion as landmarks to measure the position of the scapula relative to the frontal plane. • SEM for repeat measures by the expert and novice raters was < 4.5° and < 3.9°respectively, and SEM between evaluators was < 5.0°, demonstrating absolute reliability similar to that of other goniometric assessments. • Minimum detectable change (MDC) values as a percentage of the mean (% MDC), reflecting the degree of measurement error as a percentage of the available motion, ranged between 15.9-43.7% for intra-rater reliability and 21.9-52.8% for inter-rater reliability. Observational, descriptive design. Despite scapular mobility being essential for the completion of activities of daily living (ADLs), there is currently no established, a reliable goniometric technique to measure scapular protraction and retraction. A proposed method has shown clinically significant inter-rater reliability for a goniometric technique for these measurements. This observational descriptive study examined the intra-rater reliability of a goniometric technique to assess scapular protraction and retraction among a sample of healthy adults. An occupational therapist who is a certified hand therapist (CHT) and an occupational therapy student used goniometry to measure the neutral (resting), protracted, and retracted positions of the right and left scapula for a sample of healthy young adults (n = 54; a total of 108 data points for each measurement). These measurements were compared to analyze intra-rater and inter-rater reliability. For measurements of the scapula in neutral, protraction, and retraction, the standard error of measure (SEM) for repeat measures by the expert and novice raters was < 4.5° and < 3.9° respectively and ICC values ranged from poor to moderate (0.37-0.63). The SEM for measures between evaluators was < 5.0° and the ICC was poor (0.16-0.35). Minimum detectable change (MDC) values as a percentage of the mean (% MDC) ranged between 15.9 and 43.7% for intra-rater reliability and 21.9-52.8% for inter-rater reliability. The results of the study were mixed; variance of less than 5° for repeat measures by the same rater and measures between raters suggest clinically acceptable reliability. However, variance as a proportion of available motion (%MDC) demonstrates a broader range both above and below the threshold of 30% suggested for rehabilitation assessments. There are few efficient, reliable techniques to measure scapular mobility in clinical practice. The absolute reliability of goniometry to measure scapular protraction and retraction is similar to measurements of other joints. However, additional research and possible refinement of the technique is recommended to further address relative reliability and validity. [ABSTRACT FROM AUTHOR]

In team handball, the individual match performance of each player is essential for winning games; however, a validated match analysis system is still lacking. Consequently, the aim of the study was to justify (1) the different relevant variables and their scoring within the individual match analysis (PlayerScore), (2) to determine the intra-rater reliability and validity of the PlayerScore, and (3) to determine the influence of the rater in relation to their degree of expertise level. Six games (three games each of Spain and Brazil, one game twice) of the 2021 World Championship were analyzed by six different raters. The PlayerScore was calculated for each field player of Spain and Brazil in all seven analyzed games. We found a high intra-rater reliability (ICC = 0.97) for the two rated games (Spain against Germany), a highly significant difference (p < 0.001) between the summarized team PlayerScore of Spain and Brazil, as well as significant differences (p < 0.001) for the factor “game” and “rater” (p < 0.05), but no significant interaction for “game × rater” (p = 0.90) in the two-way repeated measures ANOVA. We conclude that the PlayerScore is a reliable and valid rating tool to determine the individual players’ performance in team handball; however, the raters should have sufficient experience in the different techniques and tactics in team handball.

PURPOSE: The aim of the study was to investigate the inter- and intra-rater reliability of the Modified Modified Ashworth Scale (MMAS) in the assessment of lower extremity spasticity in children with spastic cerebral palsy (CP). METHODS: Fifteen children (10 boys) with a mean age of 8.7±3.4 years participated. Two physiotherapists rated the spasticity of the hip adductors, knee extensors, and ankle plantar flexors for inter-rater reliability. Each child was examined again by one of the physiotherapists (same physiotherapist for all of the children) for intra-rater reliability (mean interval = 7 days). A random sequence of raters and muscles tested was applied. RESULTS: The reliability of the intraclass correlation coefficients (ICC) for individual muscle groups ranged between good to excellent (ICCagreement of 0.60–0.83). The ICC values for overall inter-rater (ICCagreement = 0.82) and intra-rater reliability (ICCagreement = 0.85) were excellent. CONCLUSION: The MMAS showed excellent reliability for the assessment of lower extremity muscle spasticity in children with cerebral palsy. However, an interpretation should be made with caution due to the small sample size and wide range of confidence interval values. [ABSTRACT FROM AUTHOR]

Background: We study dual-task performance with marker-free video recordings of Timed Up-and-Go tests (TUG) and TUG combined with a cognitive/verbal task (TUG dual-task, TUGdt).Research Question: Can gait parameters be accurately estimated from video-recorded TUG tests by a new semi-automatic method aided by a technique for human 2D pose estimation based on deep learning?Methods: Thirty persons aged 60-85 years participated in the study, conducted in a laboratory environment. Data were collected by two synchronous video-cameras and a marker-based optoelectronic motion capture system as gold standard, to evaluate the gait parameters step length (SL), step width (SW), step duration (SD), single-stance duration (SSD) and double-stance duration (DSD). For reliability evaluations, data processing aided by a deep neural network model, involved three raters who conducted three repetitions of identifying anatomical keypoints in recordings of one randomly selected step from each of the participants. Validity was analysed using 95 % confidence intervals (CI) and p-values for method differences and Bland-Altman plots with limits of agreement. Inter- and intra-rater reliability were calculated as intraclass correlation coefficients (ICC) and standard errors of measurement. Smallest detectable change was calculated for inter-rater reliability.Results: Mean ddifferences between video and the motion capture system data for SW, DSD, and SSD were significant (p < 0.001). However, mean differences for all parameters were small (-6.4%-13.0% of motion capture system) indicating good validity. Concerning reliability, almost all 95 % CI of the ICC estimates exceeded 0.90, indicating excellent reliability. Only inter-rater reliability for SW (95 % CI = 0.892;0.973) and one rater's intra-rater reliability for SSD (95 % CI = 0.793;0.951) were lower, but still showed good to excellent reliability.Significance: The presented method for extraction of gait parameters from video appears suitable for valid and reliable quantification of gait. This opens up for analyses that may contribute to the knowledge of cognitive-motor interference in dual-task testing. [ABSTRACT FROM AUTHOR]

Abstract Background The English version of the Chedoke Arm and Hand Activity Inventory is a validated, upper-limb measure with the purpose of assessing functional recovery of the arm and hand after a stroke. A German translation and cross-cultural adaptation was recently produced and demonstrated high validity, inter-rater reliability and internal consistency. As a follow-up, the present study evaluated the intra-rater reliability and responsiveness of the CAHAI-G for the long and all shortened versions. Methods The CAHAI-G and the Action Research Arm Test were assessed on three different measurement events: upon entry (ME1), two to 3 days after entry (ME2), and after three to 4 weeks (ME3). For the intra-rater reliability analysis, the ME1 CAHAI assessments were recorded on video and rated by three therapists to obtain the intraclass coefficients (ICC). The data of all three MEs were analysed in a group of stroke inpatients for the evaluation of responsiveness. To test for responsiveness, the CAHAI-G change data were compared to concurrent instruments: The Global Rating of Change-questionnaire and the Global Rating of Concept-questionnaire. Both served as external criteria. For all CAHAI-G versions (7, 8, 9 or 13 items), the same analysis procedures for the evaluation of the responsiveness parameter were performed. Results In total, 27 patients (9 females, age 63 ± 13.7) were enrolled in the study. The ICCs for the intra-rater reliability were calculated to be between 0.988 and 0.998 for all CAHAI versions. Responsiveness parameters were as follows from CAHAI-G 7 to 13: Minimal Detectable Change (MDC90) 5.3, 6.0, 6.1, 8.2; Pearson’s correlation coefficients CAHAI-Gs with ARAT 0.365, 0.409*, 0.500**, 0.597**. The Area und Under the Curve and the Minimal Clinical Important Difference values for all CAHAI-G versions and the three external criteria ranged between 0.483 to 0.603 and 2.5 to 9.0, respectively. Conclusion In addition to the high validity, inter-rater reliability and internal consistency, the CAHAI-G revealed high intra-rater reliability. The data also suggest an adequate responsiveness of the CAHAI-G versions 9 and 13.

Introduction: Reliability is an integral part of measuring the reproducibility of research information. Intra-cluster correlation coefficient (ICC) is one of the necessary indicators for reliability reporting, which can be misleading in terms of its diversity. The main purpose of this study was to introduce the types of reliability and appropriate ICC indices. Methods: In this tutorial article, useful information about the types of reliability and indicators needed to report the results, as well as the types of ICC and its applications were explained for dummies. Results: Three general types of reliability include inter-rater reliability, test-retest reliability, and intra-rater reliability was presented. 10 different types of ICC were also introduced and explained. Conclusion: The research results may be misleading if any of the reliability types and calculation criteria types are chosen incorrectly. Therefore, to make the results of the study more accurate and valuable. Medical researchers must seek help from relevant guidelines such as this study before conducting reliability analysis.

Anteroposterior (AP) alignment assessment for nondisplaced femoral neck fractures is important for determining the treatment strategy and predicting postoperative outcomes. AP alignment is generally measured using the Garden alignment index (GAI). However, its reliability remains unknown. We compared the reliability of GAI and a new AP alignment measurement (valgus tilt measurement [VTM]) using preoperative AP radiographs of nondisplaced femoral neck fractures. The study was designed as an intra- and inter-rater reliability analysis. The raters were four trauma surgeons who assessed 50 images twice. The main outcome was the intraclass correlation coefficient (ICC). To calculate intra- and inter-rater reliability, we used a mixed-effects model considering rater, patient, and time. The overall ICC (95% CI) of GAI and VTM for intra-rater reliability was 0.92 (0.89–0.94) and 0.86 (0.82–0.89), respectively. The overall ICC of GAI and VTM for inter-rater reliability was 0.92 (0.89–0.95), and 0.85 (0.81–0.88), respectively. The intra- and inter-rater reliability of GAI was higher in patients aged

COVID-19 is associated with musculoskeletal disorders. Ultrasound is a tool to assess muscle architecture and tendon measurements, offering an idea of the proportion of the consequences of the disease, since significant changes directly reflect the reduction in the ability to produce force and, consequently, in the functionality of the patient; however, its application in post-COVID-19 infection needs to be determined. We aimed to assess the intra- and inter-rater reliability of ultrasound measures of the architecture of the vastus lateralis (VL), rectus femoris (RF), vastus medialis (VM), gastrocnemius lateralis (GL), gastrocnemius medialis (GM), soleus (SO), and tibialis anterior (TA) muscles, as well as the patellar tendon (PT) cross-sectional area (CSA) in post-COVID-19 patients. An observational, prospective study with repeated measures was designed to evaluate 20 post-COVID-19 patients, who were measured for the pennation angle (θp), fascicular length (Lf), thickness, echogenicity of muscles, CSA and echogenicity of the PT. The intra-class correlation coefficient (ICC) and 95% limits of agreement were used. The intra-rater reliability presented high or very high correlations (ICC = 0.71–1.0) for most measures, except the θp of the TA, which was classified as moderate (ICC = 0.69). Observing the inter-rater reliability, all the evaluations of the PT, thickness and echogenicity of the muscles presented high or very high correlations. For the Lf, only the RF showed as low (ICC = 0.43), for the θp, RF (ICC = 0.68), GL (ICC = 0.70) and TA (ICC = 0.71) moderate and the SO (ICC = 0.40) low. The ultrasound reliability was acceptable for the muscle architecture, muscle and tendon echogenicity, and PT CSA, despite the low reliability for the Lf and θp of the RF and SO, respectively.

Objective: To examine the inter- and intra-rater reliability of the pull test in patients with Parkinson's disease (PD) using the extracted pull force., Methods: In this inter- and intra-rater reliability study, two raters performed a pull test on 30 patients with PD. The pull force was quantified using inertial sensors attached to the rater's right hand and the patient's lower trunk. In this study, the pull force was calculated as an extracted three-dimensional vector quantity, the resultant acceleration, and was expressed in m/s2. Inter- and intra-rater reliabilities were analyzed using the interclass correlation coefficient (ICC) for the pull force and Cohen's weighted kappa (κw) for the pull test score. Furthermore, Bland-Altman analysis was used to investigate systematic errors., Results: The inter- and intra-rater reliability of the pull force was very poor (ICC = 0.033-0.214). Bland-Altman analysis revealed no systematic errors in the pull forces between the two test points. Conversely, κw for the pull test scores ranged from 0.763 to 0.920, indicating substantial to almost perfect agreement., Conclusion: The pull test score was reliable despite variations in the quantified pull force for inter- and intra-rater reliability. Our findings suggest that the pull test is a robust tool for evaluating postural instability in patients with PD and that the pull force probably does not affect scoring performance.

Spasticity is one of the most common and disabling symptoms in Multiple sclerosis (MS). The is a clinical tool for assessing spasticity. This study aimed to investigate the inter- and intra-rater reliability of the modified Tardieu scale for assessing knee extensors spasticity in MS patients. Twenty-six patients with MS (12 females and 14 males) with a mean age of 40 ± 11.39 years participated in this study. The extensor muscles of both knees were evaluated using the MTS in two sessions. At first session, two examiners randomly assessed the knee extensor spasticity to study the inter-rater reliability and 3–4 days later the first examiner assessed the patients again, to determine intra-rater reliability. Intra-class Correlation Coefficient (ICC) analysis, two-way random effect model was used to determine the reliability of various components of the modified Tardieu scale. The inter-rater reliability for quality of muscle reaction of knee extensor muscles was very good (ICC = 0.89) and for the difference between the angle of muscle response (R1) and full range(R2) of movement (R2- R1), as spasticity intensity criterion, was good (ICC = 0.73). ICC values for R2-R1 and muscle response quality assessments by one rater were 0.73 and 0.82, respectively. The findings of the current study showed that the MTS has good to very good inter- and intra-rater reliability for assessing knee extensors spasticity in MS patients. [ABSTRACT FROM AUTHOR]

The study is an attempt to estimate the intra-rater reliability of students’ self-assessment of their writing performances and to find out whether rater training improves the reliability. The rater training used is adapted from the model developed by Herman, Aschbacher and Winters(1992). This quantitative study which employed equivalent time-samples design has two variables; they were the reliability coefficient of students’ self-assessment and the rater training. The data were collected by asking 25 students conducting self-assessment on their four writing tasks and analyzed using Spearman coefficient correlation. It was found that the rs were 0.798, 0.772, 0.699 and 0.637; it showed that the consistency within students in assessing their own writing was moderately high. However, the rs of the intra-rater reliability of the self-assessment after the treatments is not higher than that of other experience being available in the absence of the treatment. It is concluded that rater training has not been able to improve the intra-rater reliability.

• There is substantial agreement amongst professionals who interpret and perform prognostic SSEPs. • Agreement is lower in patients below 2 years of age, in those with poor outcome and when interpreting SSEPs as absent. • Clinicians using SSEPs to prognosticate poor outcome should be aware of the inter-rater variability amongst professionals. To measure inter- and intra-rater agreement in the interpretation of cortical somatosensory evoked potential (SSEP) components following paediatric cardiac arrest (CA) in multi-professional neurophysiology teams. Thirteen professionals blinded to patient outcome interpreted 96 SSEPs in paediatric patients 24-/48-/72-hours following CA. Of these, 34 were duplicates used to assess intra-rater agreement. Consistent interpretations (absent/present/indeterminate) between scientists (who record/identify SSEP components) and neurophysiologists (who provide prognostic SSEP interpretation) were expressed as percentages. Rates of agreement were calculated using Fleiss' kappa coefficient (K). Unanimous agreement between professionals was present in 40% (95%CI: 28–54%) of the interpreted SSEPs, with a K value of 0.62 (95%CI: 0.55–0.70) based on average agreement. Agreement was similar between neurophysiologists (K = 0.67; 95%CI: 0.57–0.77) and scientists (K = 0.62; 95%CI: 0.54–0.70) but lower in patients < 2 years old (K = 0.23; 95%CI: 0.14–0.33) and in those with poor outcome (K = 0.21; 95%CI: 0.07–0.35). No SSEP was unanimously interpreted as absent and 92% (95%CI: 89–95%) of duplicate SSEPs were interpreted consistently. Despite substantial agreement when interpreting prognostic SSEPs, this was significantly lower in children with poor outcome and of younger age. Clinicians using SSEPs in the intensive care unit should be aware of the inter-rater variability when interpreting SSEPs as absent. [ABSTRACT FROM AUTHOR]

Few studies have investigated the measurement error of the Mini-Mental State Examination (MMSE) in the same unit of measurement, also known as absolute reliability. This measurement can help determine whether an observed score change for an individual is likely to represent true change. The aim of this study was to investigate the absolute reliability of the MMSE among individuals with dementia that reside in nursing homes. Among 88 participants, 19 (21.6%) were men, 35 (39.8%) had Alzheimer's disease, 35 (39.8%) had vascular dementia, and the mean age was 84.0 years (range 65–98). The participants were tested and retested with the MMSE within 1–6 days. Both tests were administered by the same assessor at the same time of day. The mean MMSE score was 13.7 (range 0–28). The absolute difference between MMSE scores varied from 0 to 6 points, and the differences did not correlate with the corresponding score means (p = 0.874). The smallest detectable change (SDC) between two measurements was 4.00. The SDC was independent of depression, impaired vision and hearing, delirium within the last week, dementia type and age. However, the SDC was 5.56 among men and 3.50 among women (p = 0.003). In conclusion, for individuals with dementia that reside in nursing homes, it seems like their MMSE score needs to change by four or more points between two measurements in order for their score change to be reliably higher than the measurement error. [ABSTRACT FROM AUTHOR]

Even as new medical modalities, diagnostics, and technologies are rapidly changing healthcare, providing patients with safe, high-quality care remains the central focus. To provide safe patient care, healthcare providers are obligated to demonstrate and maintain the necessary competence. As more healthcare disciplines move toward a competency-based education model, it is essential to extend the competence verification from the academic educational level to the patient’s bedside. The nutrition-focused physical exam (NFPE) is a competency recently adopted by registered dietitian nutritionists (RDNs) for assessing patients’ nutritional status. Being a newly acquired skill, validated tools are required to measure NFPE competence during routine clinical practice. The Interactive Nutrition Specific Physical Exam Competency Tool (INSPECT) is a new tool developed specifically to observe and measure RDNs’ NFPE competence in clinical settings. The INSPECT was designed and validated for content using expert RDNs’ input in the first and second phases of the study. This current study aimed to assess the reliability of the INSPECT through multi-site observations by clinical supervisors evaluating RDNs’ NFPE competency during patient assessment. The INSPECT exhibited good inter-rater reliability (ICC = 0.78 for the first assessment and ICC = 0.68 for the second assessment), moderate to strong intra-rater reliability for 37 of 41 items (Spearman rho = 0.54 to 1.0), and excellent internal consistency (Cronbach’s α = 0.86 for the first assessment and α = 0.92 for the second assessment). In total, 10 out of the 11 INSPECT subsets showed good to excellent internal consistency (α ranging from 0.70 to 0.98). The results demonstrate that the INSPECT is a reliable tool, is stable over time, and has good agreement and excellent consistency between raters. The INSPECT can be a valuable tool to measure, promote and maintain RDNs’ NFPE competence in authentic acute care settings.

Purpose: Neck pain is a common musculoskeletal disorder. Therefore, establishing effective physical therapy for neck pain is one of the most important issues. In addition, in physical therapy for neck pain, it is important to evaluate the thoracic spine, which is an adjacent region of the neck. The lumbar-locked rotation test is designed to evaluate the rotational range of the thoracic spine. However, the reliability of the test when performed on patients with neck pain has not been confirmed., Objective: We aimed to determine the intra- and inter-rater reliability of the lumbar-locked rotation test in patients with neck pain., Methods: In this study involving 43 patients, two separate examiners measured thoracic spine rotation. Both examiners conducted three measurements for each side, before and after a five-minute interval. Reliability was assessed using various intra-class correlation coefficient (ICC) models., Results: The intra-rater reliability showed ICC values of 0.99 for both examiners. The inter-rater reliability showed ICC values of 0.98 for both right and left thoracic rotations., Conclusion: The findings strongly suggest that the lumbar-locked rotation test has high within-session intra- and inter-rater reliability for patients with neck pain. This test can be considered a reliable method of measuring the thoracic spine rotational range of motion in patients with neck pain in clinical practice., Competing Interests: The authors have declared that no competing interests exist., (Copyright © 2024, Yoshida et al.)