Search

The elicited imitation task (EIT), in which language learners listen to a series of spoken sentences and repeat each one verbatim, is a commonly used measure of language proficiency in second language acquisition research. The "TOEFL® Essentials"™ test includes an EIT as a holistic measure of speaking proficiency, referred to as the "Listen and Repeat" task type. In this report, we describe the design considerations that informed the development of the EIT for TOEFL Essentials. We also report the results of a series of investigations conducted during the prototyping and pilot phases of test development, which were undertaken with the goal of confirming task design specifications, evaluating scoring performance, and obtaining initial validity evidence to support score interpretation and use of the EIT in the TOEFL Essentials test. We found that task design variables generally performed as expected. The length of input sentence was strongly associated with performance (Pearson r = 0.88), consistent with the construct measured by the EIT, while other task variables not directly related to the EIT construct did not impact performance (e.g., graphics, speaker accent, and response time). Scorers drawn from TOEFL iBT test raters were able to score responses consistently with over 98% exact or adjacent interrater agreement on a 6-point scale, and scores on the pilot version of the EIT were highly reliable (Cronbach's [alpha] = 0.93 on the 15-item pilot version). Correlations between EIT scores and other measures were generally as expected: Correlations with other speaking tasks were high (0.78-0.84) and slightly to somewhat lower for other language measures (0.73 for writing, 0.68 for listening, and 0.57 for reading). Correlation with an independent measure of holistic language proficiency (C-test) was moderately high (0.69), as expected. We discuss the study findings in terms of the TOEFL Essentials test validity argument and point out limitations to the current results along with future research needs. Overall, we believe that the findings provide initial support to warrant the use of the EIT as operationalized in the TOEFL Essentials test.

The topic of "teacher professionalism" is one of the most crucial ones in quality education research. It has a potential to generate results that could inform and hence enhance the practice in classrooms. Thus, research in this field needs reliable instruments to measure the professional knowledge of our teachers to be able to generate reliable results for our research problems. Not many instruments have been developed with regard to this topic. At the same time, an adequate validation of the instrument developed is often missing (Schilling & Hill, 2007). Hence, in a bigger project "ProwiN" (German acronym for professional knowledge of science teachers), test instruments for measuring science teachers' pedagogical, pedagogical content and content knowledge (PK, PCK, and CK) were developed for the subjects biology, chemistry and physics. The present study tested the validity of some of these items which were used to measure the pedagogical content knowledge (PCK) of biology teachers. These items focused on measuring teachers' professional knowledge by analyzing 1) teachers' "knowledge about student understanding" (or lack of understanding) of several topics in biology and 2) "knowledge about instructional strategies" like the use of models or experiments. The content validity of these instruments was examined by think-aloud interviews with American and German Biology teachers (N = 11). This study shows a high content validity for these items. Furthermore, this paper demonstrates the scope for adapting the conceptual framework of these items to measure biology teachers' PCK in other countries.

Background: Fibromyalgia (FM), a disorder characterized by chronic widespread pain and tenderness, affects greater than five million individuals in the United States alone. Patients experience multiple symptoms in addition to pain, and among them, fatigue is one of the most bothersome and disabling. There is a growing body of literature suggesting that fatigue is a multidimensional concept. Currently, to our knowledge, no multidimensional Patient Reported Outcome (PRO) measure of FM-related fatigue meets Food and Drug Administration (FDA) requirements to support a product label claim. Therefore, the objective of this research was to evaluate qualitative and quantitative data previously gathered to inform the development of a comprehensive, multidimensional, PRO measure to assess FM-related fatigue in FM clinical trials.Methods: Existing qualitative and quantitative data from three previously conducted studies in patients with FM were reviewed to inform the initial development of a multidimensional PRO measure of FM-related fatigue: 1) a concept elicitation study involving in-depth, open-ended interviews with patients with FM in the United States (US) (N = 20), Germany (N = 10), and France (N = 10); 2) a cognitive debriefing and pilot study of a preliminary pool of 23 items (N = 20 US patients with FM); and 3) a methodology study that explored initial psychometrics of the item pool (N = 145 US patients with FM).Results: Five domains were identified that intend to capture the broad experience of FM-related fatigue reported in the qualitative research: the Global Fatigue Experience, Cognitive Fatigue, Physical Fatigue, Motivation, and Impact on Function. Seventeen of the original pool of 23 items were selected to best capture these five dimensions. These 17 items formed the basis of a newly developed multidimensional PRO measure to assess FM-related fatigue in clinical trials: the Multidimensional Daily Diary of Fatigue-Fibromyalgia-17 (MDF-Fibro-17).Conclusion: Qualitative analysis, and preliminary quantitative item level data, confirmed that FM-related fatigue is multidimensional and provided strong support for the content validity of the MDF-Fibro-17. The next stage was to quantitatively evaluate the measure to confirm the factor structure, psychometric properties, sensitivity to change, and meaningful change. This has been conducted and is being reported separately. [ABSTRACT FROM AUTHOR]