Search

The present study represents a novel method not yet used in the quantitative intersectionality literature -- the CT-C(M-1) model (Eid et al., 2003) -- for measuring and understanding the similarities and uniquenesses among intersectional subgroups. Intersectionality is a conceptual framework from which to investigate and remedy the ways in which oppression manifests at the intersections of socio-politico-geo-temporal power structure contexts and individuals' interwoven experiences of racism, sexism, and other forms of marginalization (Cho et al., 2013). Specifically, we describe and illustrate the usefulness of the CT-C(M-1) model in intersectionality research through estimation of the latent variable structure of two school climate variables (engagement and support) using data from N = 165 schools in which Black non-Hispanic students' experience is centered as the reference category, and which other race-ethnicity subgroups are compared. Consistent with prior research, our substantive findings indicated that, while a large share of commonality among subgroups was observed, Black Hispanic students experienced school climate differently from the other groups. This analytic tool adds to the growing set of quantitative methods that can aid in advancing the second goal of intersectionality research -- intervening in the status quo for true transformational change.

The National Association of School Psychologists emphasizes consultation, collaboration, and family-school-community collaboration as domains within school psychology practice. Family engagement is defined as caregiver participation in children's learning and instruction. While research outlines best practices in engaging and collaborating with families across multitiered systems of support (MTSS), there has been little examination of the current practices of school psychologists within this domain. This study investigated family engagement practices in schools, school psychologists' involvement in these practices, and their perceived effectiveness. Results indicated that while family engagement and problem-solving consultation methods were used within MTSS, school psychologists were less involved in implementation. Implications and future research are discussed.

The Home Learning Environment (HLE) focuses on everyday learning habits in families to support the development of children's early cognitive competencies. A growing number of studies have assessed the HLE by using different conceptual approaches and various assessment methods, often focusing on either the home literacy environment or the home numeracy environment. However, it is still unclear whether the 2 dimensions of the HLE are separable constructs and which assessment method is best suited for assessing the HLE, making it difficult to interpret and compare different study results. In the current study, we used multitrait-multimethod (MTMM) analyses to compare 3 common methods for assessing the HLE and their relations to various outcomes of kindergarten children. Our sample consisted of 2 independent cohorts of children (N[subscript 1] = 190, N[subscript 2] = 310) with an average age of 61 months (SD = 4.6). In both cohorts, the MTMM matrix showed a substantial effect of common methods and indicated a 1-dimensional HLE construct indicated most strongly by the children's book title recognition test (TRT-VS). Even when controlling for child and family characteristics (i.e., intelligence and socioeconomic status), the HLE was statistically significantly related to both the mathematical competencies and the linguistic competencies of the child in a structural equation model. This pattern of results was found consistently across both cohorts. We discuss the significance of a 1-dimensional HLE construct and its different assessment methods in the light of implications for future research and interventional approaches.

The objectives in this research were to determine the construct validity of both an adapted self-confidence questionnaire, the Personal Evaluation Inventory (PEI), developed by Shrauger and Schohn, and a Generalised Anxiety Disorder (GAD) questionnaire, which was adapted from the Taylor Manifest Anxiety Scale. The research was conducted in two girls' primary schools in Saudi Arabia to collect relevant data on the convergent and discriminant validity of the GAD and PEI questionnaires using the Multi-Trait Multi-Method (MTMM) matrix to prove construct validity. Sixty students and two teachers filled in questionnaires, with each student evaluating themselves and, then, their peers. The teachers evaluated themselves, their self-confidence and generalised degree of their anxiety disorder. The results were that the MTMM analysis supported, to a large extent, both convergent and discriminant validity of the analysed data from students and teachers on two traits (self-confidence and generalised anxiety disorder) and across three methods of measurement (self-reporting, peer-rating and teacher-rating). The results were that the Mono-Trait Mono-Method coefficients were relatively high, and there was relative strength in the Hetero-Trait Mono-Method coefficients. The Hetero-Trait Mono-Method coefficients were reasonable for self-confidence and for the generalised anxiety disorder questionnaires, but teacher-ratings for both traits were unexpected. Furthermore, the Hetero-Trait Hetero-Method coefficients were not constant and showed an unstable variance. In conclusion, the PEI and GAD questionnaires possess acceptable construct validity, but that the teacher-ratings for both the PEI and the GAD questionnaires needed modification in order to attain the desirable construct validity.

The aim of this paper is the validation of measurement tools which assess critical thinking and creativity as general constructs instead of subject-specific skills. Specifically, this research examined whether there is convergent and discriminant (or divergent) validity between measurement tools of creativity and critical thinking. For this purpose, the multi-trait and multi-method matrix suggested by Campbell and Fiske (1959) was used. This matrix presented the correlation of scores that students obtain in different assessments in order to reveal whether the assessments measure the same or different constructs. Specifically, the two methods used were written and oral exams, and the two traits measured were critical thinking and creativity. For the validation of the assessments, 30 secondary-school students in Greece and 21 in England completed the assessments. The sample in both countries provided similar results. The critical thinking tools demonstrated convergent validity when compared with each other and discriminant validity with the creativity assessments. Furthermore, creativity assessments which measure the same aspect of creativity demonstrated convergent validity. To conclude, this research provided indicators that critical thinking and creativity as general constructs can be measured in a valid way. However, since the sample was small, further investigation of the validation of the assessment tools with a bigger sample is recommended.

This paper examines how different test methods may tap different aspects of second language knowledge. It employs multiple-choice (MC) and constructed response (CR) items which yield distinct or convergent information in the computer delivered testing of English in its presentation of this factor. In order to examine the effects of test method, a CFA approach to a multitrait-multimethod design was adopted to examine the convergent as well as discriminant validity of the two skill area traits and the two test methods. After comparing the hypothesized model with a nested series of more restrictive models using X[superscript 2] difference tests (Byrne, 2006; Widaman, 1985), based on the information from the path coefficients, and also using principal factor analyses, it was found that: 1) the skills are factorially separable, yet, highly correlated, 2) MC and CR methods are separable, and very weakly correlated, 3) there is thus a method effect, and 4) the MC method may not be suitable in sampling the unique ability characteristics of reading and listening skills.

Cultural differences between faculty and their students can create important challenges that affect the quality and efficacy of online teaching and learning. The objectives of this study were to: (a) create and pilot test an assessment for online faculty to measure culturally responsive teaching knowledge (CRT) and culturally responsive educational practices (CREP) in teaching and advising students of color, military students, LGBTQ students, religious minority students, and international students; (b) describe participants' CRT knowledge, value, and CREP; and (c) examine differences between their knowledge and their practices. The combined pilot and main study sample was comprised of 47 completed surveys. Internal consistency reliability was high for all subscales (0.895-0.970); subscale intra-class correlation coefficients ranged from 0.526-0.833. The pilot round revealed strong face and content validity. Campbell-Fiske multitrait-multimethod matrix generated evidence of construct validity. Within-group comparisons of subscale scores using Wilcoxon Signed-ranks test revealed some significant differences between perceived knowledge and practice. Mann Whitney U test did not reveal significant differences in subscale scores or overall score by sector, degree level taught, or gender. The results have important implications for faculty training, professional development, mentoring, and support. Faculty who teach online who understand and value culturally responsive pedagogy and have the knowledge and skills to implement best practices in meeting the needs of diverse learners will enhance both teaching and learning. Culturally responsive knowledges and practice are particularly important as online programs are becoming ubiquitous across traditional institutions of higher education and their core faculty are teaching an increasingly diverse student body. [Paper presented at the Special Interest Group on Online Teaching and Learning (SIG-OTL), American Educational Research Association (AERA) Centennial Annual Meeting (100th, Washington, D.C., April 8-12, 2016).]

This report presents results on a parallel implementation of the expectation-maximization (EM) algorithm for multidimensional latent variable models. The developments presented here are based on code that parallelizes both the E step and the M step of the parallel-E parallel-M algorithm. Examples presented in this report include item response theory, diagnostic classification models, multitrait-multimethod (MTMM) models, and discrete mixture distribution models. These types of models are frequently applied to the analysis of multidimensional responses of test takers to a set of items, for example, in the context of proficiency testing. The algorithm presented here is based on a direct implementation of massive parallelism using a paradigm that allows the distribution of work among a number of processor cores. Modern desktop computers as well as many laptops are using processors that contain 2-4 cores and potentially twice the number of virtual cores. Many servers use 2, 4, or more multicore #central processing units (CPUs), which brings the number of cores to 8, 12, 32, or even 64 or more. The algorithm presented here scales the time reduction in the most calculation-intense part of the program almost linearly for some problems, which means that a server with 32 physical cores executes the parallel-E step algorithm up to 24 times faster than a single-core computer or the equivalent nonparallel algorithm. The overall gain (including parts of the program that cannot be executed in parallel) can reach a reduction in time by a factor of 6 or more for a 12-core machine. The basic approach is to utilize the architecture of modern CPUs, which often involves the design of processors with multiple cores that can run programs simultaneously. The use of this type of architecture for algorithms that produce posterior moments has straightforward appeal: The calculations conducted for each respondent or each distinct response pattern can be split up into simultaneous calculations.

The purposes of this study were to develop a model to measure the belief in Buddhism of junior high school students at Chiang Rai Buddhist Scripture School, and to determine construct validity of the model for measuring the belief in Buddhism by using Multitrait-Multimethod analysis. The samples were 590 junior high school students at Buddhist Scripture School who selected using the multi-stage random sampling. Three-Choice Situational Buddhist Belief Test (Sbbd) and Five-Scale Buddhist Belief Test (Rbbd) were used for data collection. Data were analyzed by descriptive statistics, CFA, and MTMM. The findings showed that 1) the results of the second order confirmatory factor analysis for measuring the belief in Buddhism were correlated with the empirical data at a good level (Sbbd: X[superscript 2]/df = 1.141, p-value = 0.146 and Rbbd: X[superscript 2]/df = 1.071, p-value = 0.287), and 2) multitrait-multimethod analysis had construct validity at a good level (X[superscript 2] = 33.664, df = 26, p-value = 0.144, CFI = 0.999, TLI = 0.998, RMSEA = 0.022, SRMR = 0.032, X[superscript 2]/df = 1.294). Both types of tests used for measuring the belief in Buddhism had convergent validity at a high level, discriminative validity at a moderate level, and reliability at a high level.

This study examined the convergent validity of the "Supports Intensity Scale-Adult Version" (SIS-A; Thompson et al., 2015a) and "Supports Intensity Scale-Children's Version" (SIS-C; Thompson et al., 2016a). Data from SISOnline (n = 129,864) for the SIS-A and from the SIS-C standardization sample (n = 4,015) were used for analyses. Using a pseudo multitrait-multimethod model, we estimated observed support needs scores as shared trait (support needs concept) and method (type, frequency, and daily support time) variances. Overall, trait variances more strongly influenced support needs scores than method variances, supporting the convergent validity of both versions of SIS. Findings also suggested that each of three methods of measuring support needs uniquely contributed to observed support needs ratings although different patterns existed between the SIS-A and SIS-C. [This paper was published in "American Journal on Intellectual and Developmental Disabilities" (EJ1119260).]

Measuring and understanding the nature of informant/rater effects and differences (Level 1) on a common trait when the target of measurement is at the organizational level (Level 2) involves a number of methodological considerations. Although previous research has discussed single-level latent variable applications of the correlated trait-correlated method minus one (CT-C(M-1)) model for multitrait-multimethod (MTMM) designs, only recently was the CT-C(M-1) model derived for multilevel data structures. The current paper examines research on the development of multilevel latent variable measurement models when both structurally different and interchangeable raters are involved in measuring Level 2 constructs. In so doing, we review and articulate often overlooked design and interpretive considerations unique to these models. We describe these elements within the context of a real-world multilevel CT-C(M-1) latent variable model that measures school climate for 294 public high schools using individual-level ratings from students (N = 58,018), teachers (N = 12,236), and health professionals (N = 742).

ELTeach is an online professional development program developed by Educational Testing Service (ETS) in collaboration with National Geographic Learning. The ELTeach program consists of two courses: English-for-Teaching and Professional Knowledge for English Language Teaching (ELT). Each course includes a coordinated assessment leading to a score report and certificate for teachers of English as a foreign language (EFL).The Test of English-for-Teaching ("TEFT"™), the assessment component of the English-for-Teaching course, measures EFL teachers' command of English for teaching English in classroom settings, as presented in the course. In this study, we examined the internal structure of the TEFT assessment. Results of the analyses demonstrated the role of both skill and content in representing the test's internal structure. The final parcel model had a higher-order general factor and four first-order factors corresponding to reading, writing, listening, and speaking. The findings support the current score reporting practice, that is, to report a total scaled score along with score information on language skills and on language use in specific content areas.

The validity of the computer-based language test is possibly affected by three factors: computer familiarity, audio-visual cognitive competence, and other discrepancies in construct. Therefore, validating the equivalence between the paper-and-pencil language test and the computer-based language test is a key step in the procedure of designing a computer-based language test. By taking the test on Essentials of English-Speaking Countries as the case study, this paper elucidates the three-step model of validating the equivalence of the two types of test: investigating computer familiarity, assessing the impact of audio-visual cognitive competence, and examining other discrepancies in construct. The model proposed by this paper can offer some methodological insights on the way to establishing the validation model of the equivalence between the paper-and-pencil language test and the computer-based language test.

As the field of higher education has used several methods to assess college teaching, the validation of these measures and how they relate to each other has been sparse. This study drew on data from a multi-institutional study of 587 college courses to examine the efficacy of two methods of measuring college teaching (class observation and syllabus analysis) in understanding six college teaching constructs. Results of the multi-trait multi-method (MTMM) modeling analysis revealed a confluence of theory and method in validation: observation was better suited for situated teaching theories that are embedded within disciplines and learners, while syllabus analysis was stronger for robust constructs (such as time on task) that examine coursework across contexts.

The authors' purpose was to evaluate the multiple-gate Preschool Behavior Screening System (PBSS) for identifying children's social, emotional, and behavioral (SEB) difficulties. The sample included 122 triads of preschool children between 3 and 5 years old and their parents and teachers. The teacher and parent samples were predominantly women and European American with a substantial African American representation. The design involved numerous indicators of the reliability of scores and the validity of ensuing inferences. Scores from the Behavior Assessment System for Children, Second Edition were used as criterion measures for scores from the PBSS. The PBSS scores were found reliable across scales with Cronbach's alpha values ranging from 0.85 to 0.97. Agreement between teachers and parents was consistent with findings from previous research. Analyzed within a Multitrait-Multimethod Matrix, correlational evidence supported inferences about externalizing difficulties, prosocial behavior, and total problems, as well as internalizing difficulties to a lesser degree. Receiver-operating characteristic curve analyses supported the PBSS across a range of scales and cut scores. Findings support the use of the PBSS as one method for identifying preschool children who may be at risk for SEB difficulties and underscore the importance of using multiple informants in the assessment of SEB difficulties.

The purpose of this study is to determine the literacy skill level of the adults who participated in the basic level literacy programs when they complete the program and to investigate the relation of the determined level of literacy skill with socio-economic features of adults. Research is a descriptive study using mixed-method design. Study is conducted in four different cities (i.e., Sanliurfa, Ankara, Istanbul, Diyarbakir). As a result of the study, 85% of course participants cannot reach desired reading level. Moreover, while it is expected that course participants read 90-100 words in two minutes, it is seen that only 13.4% of them reached this level. The rate of writing sufficiency is only 26.4%. As a whole, the literacy skill level of younger adults is higher. The achievement of participants who are working in an income generating job about reading and writing skills is higher than the others. Adults who have attended primary education were also more successful than the others. Another differential feature of successful adults is that they are more linked with the urban life. Having more literate adults around them and the usage of written communication could be effective for the development of their literacy levels. (Contains 1 table.)

School-level contextual factors have been found to influence reports of school climate. The purpose of the current study was to evaluate the extent to which these associations are related to the school climate traits being measured or the methods (i.e., informants) used to obtain them. Data from a multilevel multitrait-multimethod (MTMM) design in which structurally different and interchangeable students (n = 45,641) and teachers (n = 12,808), residing within 302 high schools, responded to items measuring four dimensions of school climate were evaluated through a multilevel correlated trait-correlated method (CT-CM) latent analysis that allowed for the estimation of both school-level trait and informant-based method factors. The resulting trait and method factors were regressed on several school-level contextual variables. Results indicated that the percentage of students receiving free and reduced-price meals (FRPM) in schools was associated with both school climate traits and informant-based method factors, school size and the percentage of minority students in schools were associated with some traits, and school size was associated with student method effects. Findings support the use of controlling for school-level contextual factors in school climate research.

We offer a new conceptualization and measurement models for constructs at the group-level of analysis in small group research. The conceptualization starts with classical notions of group behavior proposed by Tönnies, Simmel, and Weber and then draws upon plural subject theory by philosophers Gilbert and Tuomela to frame a new perspective applicable to many forms of small group behavior. In the proposed measurement model, a collective property is operationalized as shared interpersonal action that explicitly allows us to control for systematic (method) error and random error. Group members act as key informants of group properties and processes and are treated as methods in a multitrait multimethod setting to validate our models. The models are applied to data of 277 three-person groups to develop and illustrate new procedures for ascertaining variation in measures due to hypothesized construct(s), method error, and random error. Implications and guidelines for small group research are discussed.

This paper examines the underlying assumptions that competency-based frameworks are based upon in relation to leadership development. It examines the impetus for this framework becoming the prevailing theoretical base for developing leaders and tracks the historical path to this phenomenon. Research suggests that a competency-based framework may not be the most appropriate tool in leadership development across many organisations, despite the existence of these tools in those organisations, and reasons for this are offered. Varying approaches to developing effective leaders are considered and it is suggested that leading is complex as it requires both competencies and qualities in order for a person to be an effective leader. It is argued that behaviourally-based competencies only cater to a specific part of the equation when they relate to leadership development.

The present study investigated the factor structure of a field trial sample of the Test of English as a Foreign Language™ Internet-based test (TOEFL® iBT). An item-level confirmatory factor analysis (CFA) was conducted for a polychoric correlation matrix of items on a test form completed by 2,720 participants in the 2003-2004 TOEFL iBT Field Study. CFA-based multitrait-multimethod (MTMM) analyses for the Reading and Listening sections showed that the language abilities assessed in each section were essentially unidimensional, while the factor structure of the entire test was best represented by a higher-order factor model with a general factor (English as a second language/English as a foreign language ability) and four group factors for reading, listening, speaking, and writing. The integrated Speaking and Writing tasks, which require language processing in multiple modalities, well defined the target modalities (speaking and writing). These results broadly support the current reporting of four scores corresponding to the modalities and a total score, as well as the test design where the integrated tasks contribute only to the scores for the target modalities.

The main purpose of the study was to investigate the distinctness and reliability of analytic (or multitrait) rating dimensions and their relationships to holistic scores and "e-rater"® essay feature variables in the context of the TOEFL® computer-based test (CBT) writing assessment. Data analyzed in the study were analytic and holistic essay scores provided by human raters and essay feature variable scores computed by "e-rater" (version 2.0) for two TOEFL CBT writing prompts. It was found that (a) all of the six analytic scores were not only correlated among themselves but also correlated with the holistic scores, (b) high correlations obtained among holistic and analytic scores were largely attributable to the impact of essay length on both analytic and holistic scoring, (c) there may be some potential for profile scoring based on analytic scores, and (d) some strong associations were confirmed between several "e-rater" variables and analytic ratings. Implications are discussed for improving the analytic scoring of essays, validating automated scores, and refining "e-rater" essay feature variables.

This paper uses mixed methods to document how a new charter school's curriculum facilitates in the development of a learning community. The study highlights diverse curricular practices that often do not take place in traditional public schools. The school's philosophy and curriculum, which is based in social justice, provides a unique environment where students have the opportunity to better understand the world around them while simultaneously building relationships with their classmates and teachers. Utilizing a methodological combination of observations, informal and formal interviews, document analysis, and a short survey, the paper provides a rich description of the implementation of a charter school's curriculum steeped in critical pedagogy.

The correlated trait-correlated method (CTCM) model for the analysis of multitrait-multimethod (MTMM) data is known to suffer convergence and admissibility (C&A) problems. We describe a little known and seldom applied reparameterized version of this model (CTCM-R) based on Rindskopf's reparameterization of the simpler confirmatory factor analysis model. In a Monte Carlo study, we compare the CTCM, CTCM-R, and the correlated trait-correlated uniqueness (CTCU) models in terms of C&A, model fit, and parameter estimation bias. The CTCM-R model largely avoided C&A problems associated with the more traditional CTCM model, producing C&A solutions nearly as often as the CTCU model, but also avoiding parameter estimation biases known to plague the CTCU model. As such, the CTCM-R model is an attractive alternative for the analysis of MTMM data.

Most researchers agree that psychological/educational tests are sensitive to multiple traits, implying the need for a multidimensional item response theory (MIRT). One limitation of applying a MIRT in practice is the difficulty in establishing equivalent scales of multiple traits. In this study, a new MIRT linking method was proposed and evaluated by comparison with two existing methods. The results showed that the new method was more acceptable in transforming item parameters and maintaining dimensional structures. Limitations and cautions in using multidimensional linking techniques were also discussed. (Contains 8 figures and 3 tables.)

The No Child Left Behind (NCLB) Act of 2001 requires that public schools adopt research-supported programs and practices, with a strong recommendation for randomized controlled trials (RCTs) as the "gold standard" for scientific rigor in empirical research. Within that policy framework, this paper compares the relative utility of federally-recommended RCT versus the demonstrated "extended term mixed-method" (ETMM) designs as options for monitoring effects of novel programs in real-time field settings. Guided by the program's theory of action, a year-long, two-phase study was conducted to monitor the context, processes and early outcomes of an after-school supplemental program in a New York elementary school. In both phases, the design combined a matched-groups, quasi-experiment with qualitative classroom observations and descriptive surveys. Early findings showed some positive, albeit "gross" program effects. Although findings are tentative, the ETMM approach enhanced interpretations by shedding light on relevant environmental variables, causes for program instabilities and sample attrition, and factors affecting treatment fidelity and scaling-up of the program beyond the pilot year. (Contains 3 figures, 18 tables, and 1 footnote.)

Quality assurance and evaluation research, like other fields of social research and its application, are confronted with a series of problems. In the present paper, I want first to give a list of such problems, although necessarily incomplete. It is then claimed that while there is no "perfect" solution to these problems, critical multiplism may be a set of approaches which might attenuate the problems or at least make them more visible so that one can deal with them. Critical multiplism is a general methodological framework which cannot be fully accounted for here; rather, I want to present some major issues and on their base make some suggestions about how to conduct social research. (Contains 3 footnotes, 1 table, and 3 figures.)

This paper focuses on doctoral assessment as an area that has been relatively neglected in higher education research. It then describes and justifies a mixed-method approach to the study of PhD examination processes and outcomes in Australia. The design is reported of a study including candidate and candidature information for approximately 800 PhD students across all discipline areas at eight Australian universities, some examiner information, and the 2100 examiner reports on their theses. Examination process, outcome and discourse are discussed in relation to the study design and a number of research questions to be investigated. The sampling method and data collection are described. A particular focus of this paper is how the categories were developed, tested and refined for coding the texts of the examiner reports. The overall aims of the study are to contribute new knowledge about doctoral study and provide a firm empirical foundation for enhancing research performance. (Contains 2 figures.)

This paper describes the validation of actual and preferred forms of a new classroom environment instrument--the Technology-Rich Outcomes-Focused Learning Environment Inventory (TROFLEI)--using multitrait-multimethod modelling. The 80-item TROFLEI assesses 10 classroom environment dimensions: Student Cohesiveness, Teacher Support, Involvement, Investigation, Task Orientation, Cooperation, Equity, Differentiation, Computer Usage and Young Adult Ethos. A sample of 1,249 high school students from Western Australia and Tasmania responded to actual and preferred forms of the TROFLEI. Separate exploratory factor analyses for the actual and preferred forms supported the 10 scale "a priori" structure of the instrument. The use of multitrait-multimethod modelling with the 10 scales as traits and the two forms of the instrument as methods supported the TROFLEI's construct validity. The results of this research provide strong evidence of the sound psychometric properties of this new learning environment instrument. (Contains 7 tables and 1 figure.)

What role can online communities play in meeting the informal learning needs of a professional association? This article presents the results of an interpretive study of the experiences of coordinators of Alberta Community Adult Learning Councils who participated in an online community of practice designed to support informal workplace learning. Through active participation and peripheral "lurking," newcomers were oriented into the skills and culture of the practice, and experienced practitioners gained new insights into their own professional identities and the meaning of their work. Telling their stories helped to develop not only identity as individual practitioners, but also served to reconstruct the identity of the collective community on an ongoing basis. Motivations to participate included an opportunity to learn new skills and work practices, a means of social and professional connection to colleagues, and a mechanism to reduce the isolation that was inherent in the job function and geographical location. The role of the online moderator was identified as critical in sustaining the online community over an extended period and enhancing the learning function.

This paper is about the structural equation modelling of quantitative measures that are obtained from a multiple facet design. A facet is simply a set consisting of a finite number of elements. It is assumed that measures are obtained by combining each element of each facet. Methods and traits are two such facets, and a multitrait-multimethod study is a two-facet design. We extend models that were proposed for multitrait-multimethod data by Wothke (1984;1996) and Browne (1984, 1989, 1993), and demonstrate how they can be fitted using standard software for structural equation modelling. Each model is derived from the model for individual measurements in order to clarify the first principles underlying each model. (Contains 1 footnote and 1 figure.)

Data were collected about 95 fourth-grade and fifth-grade children from three sources (teachers, parents, and children) on four constructs related to aggression (proactive aggression, reactive aggression, callous-unemotional traits, and anger dysregulation) by using a multitrait-multimethod design. Two goals were addressed through a correlated-trait-correlated-method analysis: (a) Investigate the overlap versus distinctiveness of proactive aggression, reactive aggression, callous-unemotional traits, and anger dysregulation by examining the correlations among their trait factors. (b) Investigate the extent to which the informant who assessed children's aggression was associated with the rating given. Moderate-to-strong trait factor correlations emerged among all pairs of latent trait factors, suggesting at best modest discriminant validity among the constructs. The informant played a significant role for almost all ratings; however, teachers' ratings were the most strongly linked to method variance, and children's ratings provided the most convergent validity with other sources. Findings are discussed in terms of enhanced assessment of childhood aggression.

Purpose: Standardized clinical assessment of dysarthria is essential for management and research. We present a new, fully standardized dysarthria assessment, the Bogenhausen Dysarthria Scales (BoDyS). The measurement model of the BoDyS is based on auditory evaluations of connected speech using 9 scales (traits) assessed by 4 elicitation methods. Analyses of the BoDyS' reliability and construct validity were performed to test this model, with the aim of gauging the auditory dimensions of speech impairment in dysarthria. Method: Interrater agreement was examined in 70 persons with dysarthria. Construct validity was examined in 190 persons with dysarthria using a multitrait-multimethod design with confirmatory factor analysis. Results: Interrater agreement of < 1 on a 5-point scale was found in 91% of cases across listener pairs and scales. Average reliability was 0.85. Inspection of the multitrait-multimethod matrix pointed at a high convergent and discriminant validity. Modeling of the BoDyS trait and method factors using confirmatory factor analysis yielded high goodness of fit. Model coefficients confirmed high discriminant and convergent validity and revealed meaningful relationships between scales and methods. Conclusions: The 9 auditory scales of the BoDyS provide a reliable and valid profile of dysarthric impairment. They permit standardized measurement of clinically relevant dimensions of dysarthric speech.

Students' self-efficacy expectations (SEE) in mathematics are associated with their engagement and learning experiences. Going beyond previous operationalisations of SEE we propose a new instrument that takes into account not only "facet-specificity" (expectations related to particular competences or skills) and "strength" (confidence of the expectations), but also "level" (perceived task difficulty) of these expectations as proposed by Bandura ("Self-efficacy: The exercise of control," W. H. Freeman & Co, New York, 1997; "Self-efficacy beliefs of adolescents," Information Age Publishing, Greenwich, 2006). In particular, we included level-specific items referring to perceived difficulty on a subsequent national test in mathematics. In total 756 Norwegian grade 5, 8, and 9 students completed the "Self-Efficacy Gradations of Difficulty Questionnaire." We fitted plausible multitrait-multimethod models using structural equation models. The best fitting model included three factors representing levels of perceived difficulty, and a-priori specified correlated uniquenesses representing four facets. The facets related to problem solving or students' self-regulation skills during the test in order to accomplish the following: (1) complete a certain number of problems, (2) solve tasks of a certain challenge, (3) concentrate, and (4) not give up for a certain amount of time. The results indicated that three correlated constructs representing levels of SEE are associated with scores on national tests in mathematics, and that the strongest association is between national test scores and medium level SEE. Taking level (difficulty) into account broadens our understanding of the self-efficacy construct, and allows investigation into differential relationships between SEE and performance.

The use of multiple informants is common in assessments that rely on the judgments of others. However, ratings obtained from different informants often vary as a function of their perspectives and roles in relation to the target of measurement, and causes unrelated to the trait being measured. We illustrate the usefulness of a latent variable multilevel multitrait-multimethod measurement model for extracting trait factors from reports of school climate obtained by students (N = 45,641) and teachers (N = 12,808) residing within 302 high schools. We then extend this framework to include assessments of linkages between the resulting trait factors and potential outcomes that might be used for addressing questions of substantive interest or providing evidence of concurrent validity. The approach is illustrated with data obtained from student and teacher reports of two dimensions of school climate, student engagement, and the prevalence of teasing and bullying in their schools.

Multisite trials, in which individuals are randomly assigned to alternative treatment arms within sites, offer an excellent opportunity to estimate the cross-site average effect of treatment assignment (intent to treat or ITT) "and" the amount by which this impact varies across sites. Although both of these statistics are substantively and methodologically important, only the first has been well studied. To help fill this information gap, we estimate the cross-site standard deviation of ITT effects for a broad range of education and workforce development interventions using data from 16 large multisite randomized controlled trials. We use these findings to explore hypotheses about factors that predict the magnitude of cross-site impact variation, and we consider the implications of this variation for the statistical precision of multisite trials.

Many recent studies have aimed to demonstrate that specific types of reading comprehension depend on different underlying cognitive abilities. In these studies, it is often implicitly assumed that reading comprehension is a multidimensional construct. The general aim of this study was to examine the dimensionality of a large pool of reading comprehension items differing according to text and question type. The items were administered to 996 fourth-grade children. We used multitrait, multimethod modeling to test for the existence of specific text and question types. In addition, the correlations of factor scores, reflecting the different measures of reading comprehension, with word reading speed, vocabulary, and working memory were examined. Confirmatory factor analyses revealed that the specific measures of comprehension, differing according to text and question type, hardly reflected systematic variation, after a general factor of reading comprehension was taken into account. Reading comprehension items thus largely reflect a common factor. Factor scores that were supposed to reflect specific comprehension factors were not reliable and were hardly related to word reading speed, vocabulary, and working memory.

This study examined the convergent validity of the "Supports Intensity Scale-Adult Version" (SIS-A; Thompson et al., 2015a) and "Supports Intensity Scale-Children's Version" (SIS-C; Thompson et al., 2016a). Data from SISOnline (n = 129,864) for the SIS-A and from the SIS-C standardization sample (n = 4,015) were used for analyses. Using a pseudo multitrait-multimethod model, we estimated observed support needs scores as shared trait (support needs concept) and method (type, frequency, and daily support time) variances. Overall, trait variances more strongly influenced support needs scores than method variances, supporting the convergent validity of both versions of SIS. Findings also suggested that each of three methods of measuring support needs uniquely contributed to observed support needs ratings although different patterns existed between the SIS-A and SIS-C.

The "Kobayashi Maru" is a training simulation that has its roots in the Star Trek series notable for its defining characteristic as a no-win scenario with no "correct" resolution and where the solution actually involves redefining the problem. Drawing upon these characteristics, we designed a board meeting simulation for an experiential course in nonprofit governance, which places students in a high-stakes decision-making situation closely modeled on real events. To do so, we uniquely integrated principles from acting literature with theory and research in training and development. The Kobayashi Maru Meeting is a simulation with high physical and psychological fidelity--that is, one that closely resembles the "look and feel" of real-world board governance. The topics are deliberately sensitive to personal, organizational, and societal values to create high engagement and deep learning and to highlight the importance of good governance for organizational leadership. Results from multisource, multimethod data suggest that the simulation enhanced students' decision making, critical thinking, and communication skills, as well as their ability to deal with their own and others' reactions in intense circumstances. Beyond board governance, the simulation creates an authentic learning experience that can be adapted to multiple learning contexts including leadership, ethics, decision making, and communication.

Sensory processing abnormalities in autism have largely been described by parent report. This study used a multi-method (parent-report and measurement), multi-trait (tactile sensitivity and attention) design to evaluate somatosensory processing in ASD. Results showed multiple significant within-method (e.g., parent report of different traits)/cross-trait (e.g., attention and tactile sensitivity) correlations, suggesting that parent-reported tactile sensory dysfunction and performance-based tactile sensitivity describe different behavioral phenomena. Additionally, both parent-reported tactile functioning and performance-based tactile sensitivity measures were significantly associated with measures of attention. Findings suggest that sensory (tactile) processing abnormalities in ASD are multifaceted, and may partially reflect a more global deficit in behavioral regulation (including attention). Challenges of relying solely on parent-report to describe sensory difficulties faced by children/families with ASD are also highlighted.