Search

Objective: To examine the sensitivity of split‐sample reliability estimates to the random split of the data and propose alternative methods for improving the stability of the split‐sample method. Data Sources and Study Setting: Data were simulated to reflect a variety of real‐world quality measure distributions and scenarios. There is no date range to report as the data are simulated. Study Design: Simulation studies of split‐sample reliability estimation were conducted under varying practical scenarios. Data Collection/Extraction Methods: All data were simulated using functions in R. Principal Findings: Single split‐sample reliability estimates can be very dependent on the random split of the data, especially in low sample size and low variability settings. Averaging split‐sample estimates over many splits of the data can yield a more stable reliability estimate. Conclusions: Measure developers and evaluators using the split‐sample reliability method should average a series of reliability estimates calculated from many resamples of the data without replacement to obtain a more stable reliability estimate. [ABSTRACT FROM AUTHOR]

The negative affective priming (NAP) task is a behavioral measure of inhibition of emotional stimuli. Previous studies using the NAP task have found that individuals with depression show reduced inhibition of negative stimuli, suggesting that inhibition biases may play a role in the etiology and maintenance of depression. However, the psychometric properties of the NAP task have not been evaluated or reported. In the present study, we report data on the association between NAP task performance and depression symptoms in three independent samples, and we evaluate the internal consistency and test–retest reliability of the NAP effect indices. The NAP effect for both negative and positive target words had poor internal consistency in all three samples, as well as poor 2-week (Study 2) and 6-month (Study 3) test–retest reliability. The internal consistency and test–retest reliability of response times (RT) for the individual trial types were moderate to high, as were the intercorrelations between trial types. This pattern of results indicates that overall RT is reliable but variance in RTs for the different trial types in the NAP task is indistinguishable from variance in overall RT. Depression symptom severity was not associated with the NAP effect for negative or positive target words in any of the samples, which could be due to the poor reliability of the NAP effect. Based on these findings, we do not recommend that researchers use the NAP task as a measure of individual differences in the inhibition of emotional stimuli. [ABSTRACT FROM AUTHOR]

AIM: To investigate the visual fatigue of patients with accommodative asthenopia and the difference in scores before and after treatment by using the asthenopia survey scale(ASS), and to evaluate its reliability, validity and responsiveness in this population.METHODS: A total of 112 patients with accommodative asthenopia were admitted to the department of ophthalmology, Shuguang Hospital, Shanghai University of Traditional Chinese Medicine and recruited online were investigated using the ASS questionnaire, and 48 subjects were randomly selected and retested 1wk later. A variety of reliability, validity and responsiveness indicators were used to evaluate the scale.RESULTS: The overall Cronbach's α coefficients of the scale entries was 0.91; The split-half reliability coefficients was 0.86; Repeated measurement correlation coefficient of the scale total score was 0.74, there was no significant difference before and after(P>0.05); The results of confirmatory factor analysis showed that the absolute fit index of the three-factor structural model was χ2/df

Background: In major depressive disorder (MDD), self-blame-related fMRI measures have shown the potential to be used as prognostic markers for recurrence risk. Like most potential fMRI markers, however, their reliability is unclear. Here, we probed the internal reliability of self-blame-related fMRI measures, as well as the impact of different modelling approaches on reliability metrics and validity. Methods: Internal consistency (i.e. split-half reliability) was calculated for blood oxygen level-dependent (BOLD) responses and psychophysiological interactions (PPI) related to self-blame-related biases in medication-free remitted MDD participants (n = 81) and healthy controls (n = 41). Trial-length was modelled using three durations (0, 2 and 5 s), which was convolved with the haemodynamic response function (HRF) with and without time and dispersion derivatives. Intraclass correlation coefficients (ICCs) were calculated for simple contrasts examining activation to self-blaming emotions and other-blaming emotions and the more complex contrast of the subtraction-based difference between self- and other-blaming emotions within the following a priori ROIs: right superior anterior temporal lobe seed region, anterior subgenual cingulate cortex, posterior subgenual cortex and right striatum / pallidum. Results: Across ROIs, we obtained fair reliability (ICC ≥ 0.40) for simple, but poor reliability (ICC

Aims and objectives/purpose/research questions: Language-switch costs, which denote worse performance in language-switch than in language-repetition trials, appear to be a robust finding in bilingual language switching. The aim of the present study was to examine the intraindividual reliability of language-switch costs by means of a number-naming task with German-English bilinguals. Design/methodology/approach: In a cued language-switching paradigm, participants (n = 36) switched between German and English. They performed a number-naming task in three different conditions: one-digit numbers; two-digit numbers with decade 10; and two-digit numbers with decade 20. Data and analysis: We examined the experimental effects with an analysis of variance (reaction time and error rate as the dependent variables), using trial language, language sequence and number condition as independent variables. In addition, we calculated the split-half reliability of language-switch costs (across all conditions) as well as the correlations of language-switch costs between the different conditions. Findings/conclusions: While significant language-switch costs emerged in all three number conditions, our results demonstrate a medium-sized correlation between the three experimental conditions. The split-half reliability shows a moderate to strong correlation between the odd- and even-numbered trials in the experiment. Originality: On the one hand, the present study extends the observation of language-switch costs from one-digit number naming to the more complex naming of two-digit numbers. On the other hand, and theoretically even more important, we explored the reliability of language-switch costs in a bilingual number-naming task by calculating both split-half reliability and correlations between different number conditions. Significance/implications: The results indicated that while language-switch costs are a robust experimental effect on the group level, they appear to be less well suited for correlational approaches. This also suggests that caution should be exerted when language-switch costs are used to diagnose the ability of an individual to perform language control. [ABSTRACT FROM AUTHOR]

The Wisconsin Card Sorting Test (WCST) represents the gold standard for the neuropsychological assessment of executive function. However, very little is known about its reliability. In the current study, 146 neurological inpatients received the Modified WCST (M-WCST). Four basic measures (number of correct sorts, categories, perseverative errors, set-loss errors) and their composites were evaluated for split-half reliability. The reliability estimates of the number of correct sorts, categories, and perseverative errors fell into the desirable range (rel ≥.90). The study therefore disclosed sufficiently reliable M-WCST measures, fostering the application of this eminent psychological test to neuropsychological assessment. Our data also revealed that the M-WCST possesses substantially better psychometric properties than would be expected from previous studies of WCST test-retest reliabilities obtained from non-patient samples. Our study of split-half reliabilities from discretionary construed and from randomly built M-WCST splits exemplifies a novel approach to the psychometric foundation of neuropsychology. [ABSTRACT FROM AUTHOR]

Introduction Many psychometric studies have scrutinized the dependability of different instruments for evaluating and treating autism using applied behavior analysis (ABA). However, there has been no exploration into the psychometric attributes of the Catalyst Datafinch Applied Behavior Analysis Data Collection Application, namely, internal consistency reliability measures. Materials and methods  Four datasets were extracted (n=100, 98, 103, and 62) from published studies at The Oxford Center, Brighton, MI, ranging from March 19, 2023, through January 8, 2024, using Catalyst Datafinch as the data collection tool. All data were gathered by Board Certified Behavior Analysts (BCBAs) and behavioral technicians and designed to replicate how practitioners collect traditional paper and pencil data. SPSS Statistics (v. 29.0) computed internal consistency reliability measures, including Cronbach's alpha, inter-item, split-half, and interclass correlation coefficients. Results  Dataset #1: Cronbach's alpha was 0.916 with seven items, indicating excellent reliability. Cronbach's split-half reliability for Part 1 was 0.777, indicating good reliability, and for Part 2 was 0.972, indicating excellent reliability. Guttman split-half coefficient was 0.817, indicating good reliability. Inter-item correlation coefficients ranged from 0.474 to 0.970. The average measures interclass correlation (ICC) was 0.916, indicating excellent reliability. Single measures (ICC) reliability was 0.609, indicating acceptable reliability. Dataset #2: Cronbach's alpha was 0.954 with three items, indicating excellent reliability. Cronbach's split-half reliability for Part 1 was 0.912, indicating excellent reliability, and for Part 2 was 0.975, indicating excellent reliability. Guttman split-half coefficient was 0.917, indicating excellent reliability. Inter-item correlation coefficients ranged from 0.827 to 0.977. Average measures (ICC) was 0.954, indicating excellent reliability. Single measures (ICC) reliability was 0.875, indicating good reliability. Dataset #3: Cronbach's alpha was 0.974 with three items, indicating excellent reliability. Cronbach's split-half reliability for Part 1 was 0.978, indicating excellent reliability. Split-half reliability for Part 2 was 0.970, indicating excellent reliability. Guttman split-half coefficient was 0.935, indicating excellent reliability. Inter-item correlation coefficients ranged from 0.931 to 0.972. The average measures (ICC) was 0.974, indicating excellent reliability. Single measures (ICC) reliability was 0.926, indicating excellent reliability. Dataset #4: Cronbach's alpha was 0.980 with 12 items, indicating excellent reliability. Cronbach's split-half reliability for Part 1 was 0.973, indicating excellent reliability. Split-half reliability for Part 2 was 0.996, indicating excellent reliability. Guttman split-half coefficient was 0.838, indicating good reliability. Inter-item correlation coefficients ranged from 0.692 to 0.999. The average measures (ICC) was 0.980, indicating excellent reliability. Single measures (ICC) reliability was 0.804, indicating good reliability. Conclusions These results suggest that Catalyst Datafinch demonstrates high internal consistency reliability when used with individuals with autism. This indicates that the application is reliable for collecting and analyzing behavioral data in this population. The ratings ranged from good to excellent, indicating a high consistency in the measurements., Competing Interests: The authors have declared that no competing interests exist., (Copyright © 2024, Peterson et al.)

Split-half reliability provides a method for estimating inter-item reliability of measures obtained from repeatedly administered items (or trials). We introduce RELEX; a freely available, Microsoft Excel-based software tool that randomly assembles test halves, yielding a sampling-based distribution of split-half reliability coefficients. Estimates of parallel, tau-equivalent, and congeneric split half reliability are reported. RELEX offers a graphical user interface, and provides freely eligible numbers of sampled splits as well as easy handling of missing values and variable trial or item numbers. Results are summarized in the form of a histogram, along with measures of its central tendency and uncertainty.

The article presents the methods of formation evaluation metasubject competences as a result of cognitive development of children and regulatory universal educational actions. For the first time are presented: method of "Discourse" (A.Z. Zak), associated with the formation of cognitive assessment and regulatory metasubject competences of graduates of primary school (at the decision of the plot and logical problems); the results of the tasks of this technique fifth graders (at the beginning of the school year) number of ordinary public schools and a gymnasium; Statistical analysis of the data (M.G. Sorokova), proving the reliability of the "Discourse" technique and its validity as a differential ability to detect differences in socio-demographic factor "category of students."

Self-administered computerized assessment has the potential to increase the reach of neuropsychological assessment. The present study reports the first split-half reliability estimates for a self-administered computerized variant of the Wisconsin Card Sorting Test (WCST), which is considered as a gold standard for the neuropsychological assessment of executive functions. We analyzed data from a large sample of young volunteers (N = 375). Split-half reliability estimates for perseveration errors, set-loss errors, and inference errors were all above 0.90. Split-half reliability estimates for response time measures on switch and repeat trials exceeded 0.95. Our results indicated sufficient split-half reliability for a self-administered computerized WCST, paving the way for an advanced digital assessment of executive functions. We discuss potential effects of test formats, administration variants, and sample characteristics on split-half reliability.

Many researchers have tried to predict semantic priming effects using a myriad of variables (e.g., prime-target associative strength or co-occurrence frequency). The idea is that relatedness varies across prime-target pairs, which should be reflected in the size of the priming effect (e.g., cat should prime dog more than animal does). However, it is only insightful to predict item-level priming effects if they can be measured reliably. Thus, in the present study we examined the split-half and test-retest reliabilities of item-level priming effects under conditions that should discourage the use of strategies. The resulting priming effects proved extremely unreliable, and reanalyses of three published priming datasets revealed similar cases of low reliability. These results imply that previous attempts to predict semantic priming were unlikely to be successful. However, one study with an unusually large sample size yielded more favorable reliability estimates, suggesting that big data, in terms of items and participants, should be the future for semantic priming research. [ABSTRACT FROM AUTHOR]

Executive function is vital for normal social, cognitive, and motor functions. Executive function decline due to aging increases the risk of disability and falls in older adults, which has become an urgent public health issue. Fast and convenient neuropsychological tools are thus needed to identify high-risk groups as early as possible to conduct a timely intervention. Card sorting tasks, such as Wisconsin Card Sorting Task (WCST) and its variants, are popular tools for measuring executive function. This study investigated the reliability of an open-source, self-administered, online, short-version card sorting task with a sample of young (n = 107, 65 females, age: M = 30.1 years, SD = 5.5 years) and elderly Chinese (n = 113, 53 females, age: M = 64.0 years, SD = 6.7 years). We developed an automated scoring and visualization procedure following the recent recommendations on scoring perseverative responses to make the results comparable to the standardized WCST. Reliability estimates of commonly used measures were calculated using the split-half method. All task indices' reliabilities were reasonably good in both old and young groups except for "failure-to-maintain-set." Elderly Chinese adults showed compromised task performance on all measures compared with the young Chinese adults at the group level. The R script of automated scoring and estimation of reliability is publicly available. [ABSTRACT FROM AUTHOR]

Reliabilities of scores for experimental tasks are likely to differ from one study to another to the extent that the task stimuli change, the number of trials varies, the type of individuals taking the task changes, the administration conditions are altered, or the focal task variable differs. Given that reliabilities vary as a function of the design of these tasks and the characteristics of the individuals taking them, making inferences about the reliability of scores in an ongoing study based on reliability estimates from prior studies is precarious. Thus, it would be advantageous to estimate reliability based on data from the ongoing study. We argue that internal consistency estimates of reliability are underutilized for experimental task data and in many applications could provide this information using a single administration of a task. We discuss different methods for computing internal consistency estimates with a generalized coefficient alpha and the conditions under which these estimates are accurate. We illustrate use of these coefficients using data for three different tasks. [ABSTRACT FROM AUTHOR]

Copyright of Mersin University Journal of the Faculty of Education / Mersin Üniversitesi Eğitim Fakültesi Dergisi is the property of Mersin University Faculty of Education and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Aim: The purpose of this study was to use a variety of methods to evaluate and cross-validate the reliability estimates of the quality of schoolwork performance measures in the School Version of the Assessment of Motor and Process Skills (School AMPS) (1). Methods: Split-half reliability was estimated based on School AMPS data from 6 194 students who had performed at least two school AMPS tasks. Two different split-half methods were used: splitting the School AMPS items vertically and splitting them horizontally. The Rasch equivalent of Cronbach's alpha for the full School AMPS evaluation and standard error of measurement ( SEm) were also considered. Results: All three methods yielded high reliability coefficient estimates ( r ≥ 0.70) and overall SEm was low. Conclusions: From a practice perspective, the results of this study are of clinical importance as they provide evidence that occupational therapists can have confidence in the consistency of the School AMPS measures when they are used in the process of making decisions about individual students, planning interventions, and later performing follow-up evaluations to measure the outcome. [ABSTRACT FROM AUTHOR]

The purpose of this study was to assess the internal consistency and stability of the Five-Point Test (FPT), developed by Regard, Strauss, and Knapp (1982). A test of non-verbal fluency, the FPT is an executive measure that may be particularly useful in the evaluation of individuals with presumed frontal lobe damage. In the internal consistency study, 209 healthy participants were administered the FPT. A split-half analysis revealed a correlation of. 80 for unique designs, and. 48 for perseverative errors. In the stability study, 142 healthy participants were administered the FPT on two occasions with a mean interval of 37.8 days. Across the two administration periods, a test-retest correlation of. 78 for unique designs and. 51 for perseverative errors was found. When the mean performances were compared across administration periods, significant differences were found for unique designs, but not for percentage of perseverative errors. Taken together, the results of the two studies suggest that the internal consistency and stability coefficients of the FPT are acceptable for unique designs but low for the percentage of perseverative errors. [ABSTRACT FROM AUTHOR]

The Meaningful Auditory Integration Scale (MAIS) is a parent-report questionnaire for assessing auditory behaviour in aurally habilitated children. This study addressed the reliability and convergent validity of three different language versions of the MAIS: English, German, and Polish. In total, 114 parents (English, n = 27; Polish, n = 37; German, n = 50) completed the MAIS preoperatively and at 6 months after cochlear implantation. Internal reliability (Cronbach's alpha) ranged from 0.92 to 0.95 preoperatively, and from 0.87 to 0.93 at 6 months. Split-half reliability was at least 0.90 preoperatively, and ranged from 0.76 to 0.89 at 6 months. Corrected item total correlation coefficients were significant (p < 0.05) for all items except for item 1, which showed poor correlations in the Polish version. Correlation of the MAIS with the Listening Progress Profile (LiP), as a measure for convergent validity, yielded coefficients between 0.81 and 0.73 preoperatively, and between 0.79 and 0.61 at 6 months. These findings demonstrate high reliability and convergent validity of the three MAIS versions. [ABSTRACT FROM AUTHOR]

Self-administered computerized assessment has the potential to increase the reach of neuropsychological assessment. The present study reports the first split-half reliability estimates for a self-administered computerized variant of the Wisconsin Card Sorting Test (WCST), which is considered as a gold standard for the neuropsychological assessment of executive functions. We analyzed data from a large sample of young volunteers (N = 375). Split-half reliability estimates for perseveration errors, set-loss errors, and inference errors were all above 0.90. Split-half reliability estimates for response time measures on switch and repeat trials exceeded 0.95. Our results indicated sufficient split-half reliability for a self-administered computerized WCST, paving the way for an advanced digital assessment of executive functions. We discuss potential effects of test formats, administration variants, and sample characteristics on split-half reliability. [ABSTRACT FROM AUTHOR]

Background: In major depressive disorder (MDD), self-blame-related fMRI measures have shown the potential to be used as prognostic markers for recurrence risk. Like most potential fMRI markers, however, their reliability is unclear. Here, we probed the internal reliability of self-blame-related fMRI measures, as well as the impact of different modelling approaches on reliability metrics and validity., Methods: Internal consistency (i.e. split-half reliability) was calculated for blood oxygen level-dependent (BOLD) responses and psychophysiological interactions (PPI) related to self-blame-related biases in medication-free remitted MDD participants (n = 81) and healthy controls (n = 41). Trial-length was modelled using three durations (0, 2 and 5 s), which was convolved with the haemodynamic response function (HRF) with and without time and dispersion derivatives. Intraclass correlation coefficients (ICCs) were calculated for simple contrasts examining activation to self-blaming emotions and other-blaming emotions and the more complex contrast of the subtraction-based difference between self- and other-blaming emotions within the following a priori ROIs: right superior anterior temporal lobe seed region, anterior subgenual cingulate cortex, posterior subgenual cortex and right striatum / pallidum., Results: Across ROIs, we obtained fair reliability (ICC ≥ 0.40) for simple, but poor reliability (ICC < 0.40) for more complex fMRI measures related to self-blame. Despite this low internal consistency of complex measures at the individual level, we observed robust activation at the group-level, reproducing previously published results., Conclusions: While simple BOLD contrasts had fair reliability, previously employed PPI models had poor reliability and simple connectivity measures lacked predictive validity. This calls for the development of functional connectivity measures that strike a better balance between reliability and validity for future clinical applications, which require robust measures at the individual rather than group-level., (Copyright © 2021 The Authors. Published by Elsevier Inc. All rights reserved.)