Search

Purpose: Item response theory (IRT) is a modern psychometric framework with several advantageous properties as compared with classical test theory. IRT has been successfully used to model performance on anomia tests in individuals with aphasia; however, all efforts to date have focused on noun production accuracy. The purpose of this study is to evaluate whether the Verb Naming Test (VNT), a prominent test of action naming, can be successfully modeled under IRT and evaluate its reliability. Method: We used responses on the VNT from 107 individuals with chronic aphasia from AphasiaBank. Unidimensionality and local independence, two assumptions prerequisite to IRT modeling, were evaluated using factor analysis and Yen's Q[subscript 3] statistic (Yen, 1984), respectively. The assumption of equal discrimination among test items was evaluated statistically via nested model comparisons and practically by using correlations of resulting IRT-derived scores. Finally, internal consistency, marginal and empirical reliability, and conditional reliability were evaluated. Results: The VNT was found to be sufficiently unidimensional with the majority of item pairs demonstrating adequate local independence. An IRT model in which item discriminations are constrained to be equal demonstrated fit equivalent to a model in which unique discrimination parameters were estimated for each item. All forms of reliability were strong across the majority of IRT ability estimates. Conclusions: Modeling the VNT using IRT is feasible, yielding ability estimates that are both informative and reliable. Future efforts are needed to quantify the validity of the VNT under IRT and determine the extent to which it measures the same construct as other anomia tests.

Purpose: To evaluate whether levels of cortisol, a stress-related hormone, predicted response to intensive speech-language intervention for individuals with chronic aphasia (IWA). Secondary analyses explored baseline cortisol levels, change following intervention, association between cortisol levels and aphasia severity, self-reported communicative distress, and chronic stress. Method: Afternoon salivary cortisol levels were measured in 14 IWA during the first and last weeks of a 4-week intensive speech-language intervention epoch. Behavioral outcome measures were collected pre- and postintervention. Results: Cortisol levels did not significantly predict treatment response in this sample of IWA, although a positive trend was present. Baseline cortisol levels were not abnormally elevated, did not change from pre- to postintervention, and were not significantly correlated with any of the behavioral outcome measures. Discussion: Although afternoon salivary cortisol levels did not robustly predict treatment response in this participant sample, future studies may be warranted that include IWA with elevated levels of cortisol at pretreatment.

Purpose: This meta-analysis synthesizes published studies using "treatment of underlying forms" (TUF) for sentence-level deficits in people with aphasia (PWA). The study aims were to examine group-level evidence for TUF efficacy, to characterize the effects of treatment-related variables (sentence structural family and complexity; treatment dose) in relation to the Complexity Account of Treatment Efficacy (CATE) hypothesis, and to examine the effects of person-level variables (aphasia severity, sentence comprehension impairment, and time postonset of aphasia) on TUF response. Method: Data from 13 single-subject, multiple-baseline TUF studies, including 46 PWA, were analyzed. Bayesian generalized linear mixed-effects interrupted time series models were used to assess the effect of treatment-related variables on probe accuracy during baseline and treatment. The moderating influence of person-level variables on TUF response was also investigated. Results: The results provide group-level evidence for TUF efficacy demonstrating increased probe accuracy during treatment compared with baseline phases. Greater amounts of TUF were associated with larger increases in accuracy, with greater gains for treated than untreated sentences. The findings revealed generalization effects for sentences that were of the same family but less complex than treated sentences. Aphasia severity may moderate TUF response, with people with milder aphasia demonstrating greater gains compared with people with more severe aphasia. Sentence comprehension performance did not moderate TUF response. Greater time postonset of aphasia was associated with smaller improvements for treated sentences but not for untreated sentences. Conclusions: Our results provide generalizable group-level evidence of TUF efficacy. Treatment and generalization responses were consistent with the CATE hypothesis. Model results also identified person-level moderators of TUF (aphasia severity, time postonset of aphasia) and preliminary estimates of the effects of varying amounts of TUF for treated and untreated sentences. Taken together, these findings add to the TUF evidence and may guide future TUF treatment-candidate selection.

Purpose: The purpose of this study was to develop and pilot a novel treatment framework called "BEARS" (Balancing Effort, Accuracy, and Response Speed). People with aphasia (PWA) have been shown to maladaptively balance speed and accuracy during language tasks. BEARS is designed to train PWA to balance speed-accuracy trade-offs and improve system calibration (i.e., to adaptively match system use with its current capability), which was hypothesized to improve treatment outcomes by maximizing retrieval practice and minimizing error learning. In this study, BEARS was applied in the context of a semantically oriented anomia treatment based on semantic feature verification (SFV). Method: Nine PWA received 25 hr of treatment in a multiple-baseline single-case series design. BEARS + SFV combined computer-based SFV with clinician-provided BEARS metacognitive training. Naming probe accuracy, efficiency, and proportion of "pass" responses on inaccurate trials were analyzed using Bayesian generalized linear mixed-effects models. Generalization to discourse and correlations between practice efficiency and treatment outcomes were also assessed. Results: Participants improved on naming probe accuracy and efficiency of treated and untreated items, although untreated item gains could not be distinguished from the effects of repeated exposure. There were no improvements on discourse performance, but participants demonstrated improved system calibration based on their performance on inaccurate treatment trials, with an increasing proportion of "pass" responses compared to paraphasia or timeout nonresponses. In addition, levels of practice efficiency during treatment were positively correlated with treatment outcomes, suggesting that improved practice efficiency promoted greater treatment generalization and improved naming efficiency. Conclusions: BEARS is a promising, theoretically motivated treatment framework for addressing the interplay between effort, accuracy, and processing speed in aphasia. This study establishes the feasibility of BEARS + SFV and provides preliminary evidence for its efficacy. This study highlights the importance of considering processing efficiency in anomia treatment, in addition to performance accuracy.

Purpose: Individuals with aphasia identify discourse-level communication (i.e., language in use) as a high priority for treatment. The central premise of most aphasia treatments is that restoring language at the phoneme, word, and/or sentence level will generalize to discourse. However, treatment-related changes in discourse-level communication are modest, are poorly understood, and vary greatly among individuals with aphasia. In response, this study consisted of a multilevel discourse analysis of archival, monologic discourse outcomes across two high-intensity Semantic Feature Analysis (SFA) clinical trials. Aim 1 evaluated changes in theoretically motivated discourse outcomes representing lexical-semantic processing, lexical diversity, grammatical complexity, and discourse informativeness. Aim 2 explored the potential moderating role of nonlanguage cognitive factors (semantic memory, divided attention, and executive function) on discourse outcomes. Method: This study was a retrospective analysis of archival monologic discourse outcomes after intensive SFA for n = 60 (Aim 1) and a subset n = 44 (Aim 2). Outcome measures included lexical-semantic processing (% semantic errors), lexical diversity (moving average type-token ratio), grammatical complexity (mean utterance length), and discourse informativeness (% correct information units). Bayesian generalized mixed-effects models were used to examine changes across four study time points: enrollment, entry, exit, and 1-month follow-up. Results: The present study found no evidence for meaningful or statistically reliable improvements in monologue discourse performance after SFA when measured using standard, general-topic discourse stimuli. There was weak and inconsistent evidence that nonlanguage cognitive factors may play a role in moderating treatment response. Conclusions: These findings indicate a clear need to pair theoretically informed treatments designed to facilitate generalization to discourse with intentional measurement paradigms designed to capture it. Furthermore, there is a clear need to examine how established treatments, restorative or compensatory, can better facilitate generalization to discourse-level communication. These priorities are critical for meaningfully improving everyday communication and reducing the profound communication and psychosocial consequences of aphasia. [ABSTRACT FROM AUTHOR]

Purpose: Aphasia is a language disorder caused by acquired brain injury, which generally involves difficulty naming objects. Naming ability is assessed by measuring picture naming, and models of naming performance have mostly focused on accuracy and excluded valuable response time (RT) information. Previous approaches have therefore ignored the issue of processing efficiency, defined here in terms of optimal RT cutoff, that is, the shortest deadline at which individual people with aphasia produce their best possible naming accuracy performance. The goals of this study were therefore to (a) develop a novel model of aphasia picture naming that could accurately account for RT distributions across response types; (b) use this model to estimate the optimal RT cutoff for individual people with aphasia; and (c) explore the relationships between optimal RT cutoff, accuracy, naming ability, and aphasia severity. Method: A total of 4,021 naming trials across 10 people with aphasia were scored for accuracy and RT onset. Data were fit using a novel ex-Gaussian multinomial RT model, which was then used to characterize individual optimal RT cutoffs. Results: Overall, the model fitted the empirical data well and provided reliable individual estimates of optimal RT cutoff in picture naming. Optimal cutoffs ranged between approximately 5 and 10 s, which has important implications for assessment and treatment. There was no direct relationship between aphasia severity, naming RT, and optimal RT cutoff. Conclusion: The multinomial ex-Gaussian modeling approach appears to be a promising and straightforward way to estimate optimal RT cutoffs in picture naming in aphasia. Limitations and future directions are discussed.

Purpose: The purpose of this study was to verify the equivalence of 2 alternate test forms with nonoverlapping content generated by an item response theory (IRT)--based computer-adaptive test (CAT). The Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996) was utilized as an item bank in a prospective, independent sample of persons with aphasia. Method: Two alternate CAT short forms of the PNT were administered to a sample of 25 persons with aphasia who were at least 6 months postonset and received no treatment for 2 weeks before or during the study. The 1st session included administration of a 30-item PNT-CAT, and the 2nd session, conducted approximately 2 weeks later, included a variable-length PNT-CAT that excluded items administered in the 1st session and terminated when the modeled precision of the ability estimate was equal to or greater than the value obtained in the 1st session. The ability estimates were analyzed in a Bayesian framework. Results: The 2 test versions correlated highly (r = 0.89) and obtained means and standard deviations that were not credibly different from one another. The correlation and error variance between the 2 test versions were well predicted by the IRT measurement model. Discussion: The results suggest that IRT-based CAT alternate forms may be productively used in the assessment of anomia. IRT methods offer advantages for the efficient and sensitive measurement of change over time. Future work should consider the potential impact of differential item functioning due to person factors and intervention-specific effects, as well as expanding the item bank to maximize the clinical utility of the test.

Purpose: In this study, we investigated the agreement between the 175-item Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996) and a 30-item computer adaptive PNT (PNT-CAT; Fergadiotis, Kellough, & Hula, 2015; Hula, Kellough, & Fergadiotis, 2015) created using item response theory (IRT) methods. Method: The full PNT and the PNT-CAT were administered to 47 participants with aphasia in counterbalanced order. Latent trait-naming ability estimates for the 2 PNT versions were analyzed in a Bayesian framework, and the agreement between them was evaluated using correlation and measures of constant, variable, and total error. We also evaluated the extent to which individual pairwise differences were credibly greater than 0 and whether the IRT measurement model provided an adequate indication of the precision of individual score estimates. Results; The agreement between the PNT and the PNT-CAT was strong, as indicated by high correlation (r = 0.95, 95% CI [0.92, 0.97]), negligible bias, and low variable and total error. The number of statistically robust pairwise score differences did not credibly exceed the Type I error rate, and the precision of individual score estimates was reasonably well predicted by the IRT model. Discussion: The strong agreement between the full PNT and the PNT-CAT suggests that the latter is a suitable measurement of anomia in group studies. The relatively robust estimates of score precision also suggest that the PNT-CAT can be useful for the clinical assessment of anomia in individual cases. Finally, the IRT methods used to construct the PNT-CAT provide a framework for additional development to further reduce measurement error.

Evidence suggests that communication-based aphasia group treatment may increase formal assessment scores and improve features of discourse production. The real-life interactions which occur during group treatment may increase use of grammatically complete and relevant utterances. To examine the effects of communication-based group treatment on production of complete utterances during structured and conversational discourse in adults with chronic aphasia. We analyzed structured and conversational discourse samples from 23 adults with chronic aphasia originally collected by Elman and Bernstein-Ellis (1999a). Bayesian generalized linear mixed-effects models were used to evaluate changes in use of complete utterances and its two components, syntactic completeness and relevance, at treatment exit and 4-6 weeks post-treatment. Results are presented with reference to the region of practical equivalence (ROPE, i.e., the range of effect sizes small enough to ignore) and in terms of the probability, derived from Bayesian model posterior distributions, that the effect in question exceeded, fell within, or fell below the ROPE. At the group level, syntactic completeness improved in structured discourse, but syntactic completeness, relevance, and complete utterances did not change during conversational discourse. Individual response to treatment varied and participants with mild aphasia and without concomitant apraxia of speech were more likely to demonstrate post-treatment change in complete utterances. Use of syntactically complete utterances during structured discourse increased at treatment exit and follow-up. Participants, especially those with severe aphasia and apraxia of speech, may have changed in ways not revealed by the complete utterance measure. [ABSTRACT FROM AUTHOR]

Purpose: The purpose of this study was to develop a computerized adaptive test (CAT) version of the Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996), to reduce test length while maximizing measurement precision. This article is a direct extension of a companion article (Fergadiotis, Kellough, & Hula, 2015), in which we fitted the PNT to a 1-parameter logistic item-response-theory model and examined the validity and precision of the resulting item parameter and ability score estimates. Method: Using archival data collected from participants with aphasia, we simulated two PNT-CAT versions and two previously published static PNT short forms, and compared the resulting ability score estimates to estimates obtained from the full 175-item PNT. We used a jackknife procedure to maintain independence of the samples used for item estimation and CAT simulation. Results: The PNT-CAT recovered full PNT scores with equal or better accuracy than the static short forms. Measurement precision was also greater for the PNT-CAT than the static short forms, though comparison of adaptive and static nonoverlapping alternate forms showed minimal differences between the two approaches. Conclusion: These results suggest that CAT assessment of naming in aphasia has the potential to reduce test burden while maximizing the accuracy and precision of score estimates.

Purpose: The purpose of this study is to investigate the structure and measurement properties of the Aphasia Communication Outcome Measure (ACOM), a patient-reported outcome measure of communicative functioning for persons with aphasia. Method: Three hundred twenty-nine participants with aphasia responded to 177 items asking about communicative functioning. The data were analyzed using a categorical item factor analysis approach. Validity of ACOM scores on the basis of their convergence with performance-based, clinician-reported, and surrogate-reported assessments of communication was also assessed. Results: Fifty-nine items that obtained adequate fit to a modified bifactor measurement model and functioned similarly across several demographic and clinical subgroupings were identified. The factor model estimates were transformed to item response theory graded response model parameters, and the resulting score estimates showed good precision and moderately strong convergence with other measures of communicative ability and functioning. A free software application for administration and scoring of the ACOM item bank is available from the first author. Conclusions: The ACOM provides reliable measurement of patient-reported communicative functioning in aphasia. The results supported the validity of ACOM scores insofar as (a) factor analyses provided support for a coherent measurement model, (b) items functioned similarly across demographic and clinical subgroups, and (c) scores showed good convergence with measures of related constructs.

Purpose: In this study, we investigated the fit of the Philadelphia Naming Test (PNT; Roach, Schwartz, Martin, Grewal, & Brecher, 1996) to an item-response-theory measurement model, estimated the precision of the resulting scores and item parameters, and provided a theoretical rationale for the interpretation of PNT overall scores by relating explanatory variables to item difficulty. This article describes the statistical model underlying the computer adaptive PNT presented in a companion article (Hula, Kellough, & Fergadiotis, 2015). Method: Using archival data, we evaluated the fit of the PNT to 1- and 2-parameter logistic models and examined the precision of the resulting parameter estimates. We regressed the item difficulty estimates on three predictor variables: word length, age of acquisition, and contextual diversity. Results: The 2-parameter logistic model demonstrated marginally better fit, but the fit of the 1-parameter logistic model was adequate. Precision was excellent for both person ability and item difficulty estimates. Word length, age of acquisition, and contextual diversity all independently contributed to variance in item difficulty. Conclusions: Item-response-theory methods can be productively used to analyze and quantify anomia severity in aphasia. Regression of item difficulty on lexical variables supported the validity of the PNT and interpretation of anomia severity scores in the context of current word-finding models.

Purpose: Small-N studies are the dominant study design supporting evidencebased interventions in communication science and disorders, including treatments for aphasia and related disorders. However, there is little guidance for conducting reproducible analyses or selecting appropriate effect sizes in small-N studies, which has implications for scientific review, rigor, and replication. This tutorial aims to (a) demonstrate how to conduct reproducible analyses using effect sizes common to research in aphasia and related disorders and (b) provide a conceptual discussion to improve the reader’s understanding of these effect sizes. Method: We provide a tutorial on reproducible analyses of small-N designs in the statistical programming language R using published data from Wambaugh et al. (2017). In addition, we discuss the strengths, weaknesses, reporting requirements, and impact of experimental design decisions on effect sizes common to this body of research. Results: Reproducible code demonstrates implementation and comparison of within-case standardized mean difference, proportion of maximal gain, tau-U, and frequentist and Bayesian mixed-effects models. Data, code, and an interactive web application are available as a resource for researchers, clinicians, and students. Conclusions: Pursuing reproducible research is key to promoting transparency in small-N treatment research. Researchers and clinicians must understand the properties of common effect size measures to make informed decisions in order to select ideal effect size measures and act as informed consumers of small-N studies. Together, a commitment to reproducibility and a keen understanding of effect sizes can improve the scientific rigor and synthesis of the evidence supporting clinical services in aphasiology and in communication sciences and disorders more broadly. [ABSTRACT FROM AUTHOR]

Purpose: The purpose of this study was to identify the most appropriate item response theory (IRT) measurement model for aphasia tests requiring 2-choice responses and to determine whether small samples are adequate for estimating such models. Method: Pyramids and Palm Trees (Howard & Patterson, 1992) test data that had been collected from individuals with aphasia were analyzed, and the resulting item and person estimates were used to develop simulated test data for 3 sample size conditions. The simulated data were analyzed using a standard 1-parameter logistic (1-PL) model and 3 models that accounted for the influence of guessing: augmented 1-PL and 2-PL models and a 3-PL model. The model estimates obtained from the simulated data were compared to their known true values. Results: With small and medium sample sizes, an augmented 1-PL model was the most accurate at recovering the known item and person parameters; however, no model performed well at any sample size. Follow-up simulations confirmed that the large influence of guessing and the extreme easiness of the items contributed substantially to the poor estimation of item difficulty and person ability. Conclusion: Incorporating the assumption of guessing into IRT models improves parameter estimation accuracy, even for small samples. However, caution should be exercised in interpreting scores obtained from easy 2-choice tests, regardless of whether IRT modeling or percentage correct scoring is used. (Contains 7 tables and 3 figures.)

Purpose: The purposes of this investigation were to examine the construct dimensionality and range of ability effectively measured by 28 assessment items obtained from 3 different patient-reported scales of communicative functioning, and to provide a demonstration of how the Rasch approach to measurement may contribute to the definition of latent constructs and the development of instruments to measure them. Method: Item responses obtained from 421 stroke survivors with and without communication disorders were examined using the Rasch partial credit model. The dimensionality of the item pool was evaluated by (a) examining correlations of Rasch person ability scores obtained separately from each of the 3 scales, (b) iteratively excluding items exceeding mean square model fit criteria, and (c) using principal-components analysis of Rasch model residuals. The range of ability effectively measured by the item pool was examined by comparing item difficulty and category threshold calibrations to the distribution of person ability scores and by plotting the modeled standard error of person ability estimates as a function of person ability level. Results: The results indicate that most assessment items fit a unidimensional measurement model, with the notable exception of items relating to the use of written communication. The results also suggest that the range of ability that could be reliably measured by the current item pool was restricted relative to the range of ability observed in the patient sample. Conclusions: It is concluded that (a) a mature understanding of communicative functioning as a measurement construct will require further research, (b) patients with stroke-related communication disorders will be better served by the development of instruments measuring a wider range of communicative functioning ability, and (c) the theoretical and methodological tools provided by the Rasch family of measurement models may be productively applied to these efforts. (Contains 4 tables and 6 figures.)

Evidence regarding the effect of conversationally based communication group treatment on discourse production in aphasia is limited. Given the rich, complex communication experiences provided in these groups, it seemed plausible that participation in them could result in improvement of simpler aspects of discourse production. To examine the effects of group communication treatment on the informativeness and efficiency of structured and conversational discourse tasks in adults with chronic aphasia. The data for this study were discourses elicited prospectively from 23 adults with chronic aphasia by Elman & Bernstein-Ellis (1999a) as part of their randomised controlled trial, but never previously transcribed or analyzed. We evaluated changes in discourse informativeness and efficiency at treatment exit and follow-up with Bayesian generalised linear mixed-effects models. Individual effect sizes at exit and follow-up were estimated and a region-of-practical-equivalence approach was used to evaluate whether the posterior distributions at each timepoint for each participant were clinically meaningful. Results at the group level revealed that structured discourses became more informative and efficient after treatment, and that this improvement was at least maintained at follow-up. Informativeness of conversational discourse did not change from treatment entry to exit, but there was modest evidence of improvement at follow-up. There was no evidence of change to the efficiency of conversational discourse at either timepoint. There was wide variability in individual response to the treatment. Participation in conversation-based communication group treatment was associated with more informative and efficient structured discourse production and modestly improved informativeness in conversational discourse. Examining individual responses to treatment provided additional insight about the group-level outcomes and provided some clues about factors that might have influenced performance for some of the participants. [ABSTRACT FROM AUTHOR]