Search

Recently, psychometricians and researchers have voiced their concern over the exploration of language test items in light of Messick's validation framework. Validity has been central to test development and use; however, it has not received due attention in language tests having grave consequences for test takers. The present study sought to examine the validity of the University of Tehran English language proficiency test (UTEPT) using 6 aspects of Messick's validation framework (i.e., content, structural, external, generalizability, substantive, and consequential). To examine each validity dimension, the Rasch model was considered. To this end, the data of 394 test takers who had taken the test in 2023 were cleaned and analyzed for the assumptions of the Rasch model and then for the 6 aforementioned validity aspects. The results indicated that the structural (i.e., unidimensionality), generalizability (i.e., measurement invariance), and substantive aspects held. However, the evidence for the content, external, and consequential aspects of validity was less definitive. The implications of the findings for the validity of the test, for the realm of language testing, and for item developers and item writers will be discussed.

This study presents a modified version of the Korean Elicited Imitation (EI) test, designed to resemble natural spoken language, and validates its reliability as a measure of proficiency. The study assesses the correlation between average test scores and Test of Proficiency in Korean (TOPIK) levels, examining score distributions among beginner, intermediate, and advanced learner groups. Using item response theory (IRT), the study explores the influence of four key facets--learners, items, raters, and constructs--on performance evaluation. An explanatory item response modeling (EIRM) analysis identified linguistic factors impacting the EI test's performance. Notably, the study uncovered a robust positive correlation between EI test scores and TOPIK levels. Significant score disparities were observed between beginner and intermediate, as well as beginner and advanced, learner cohorts. An IRT-based exploration of each facet revealed that item difficulty was comparatively lower in contrast to learners' commendable performance, and raters exhibited a high degree of scoring consistency. The EIRM analysis underscores the significance of variables such as the number of syllables, vocabulary score, and content word density in influencing test performance.

As the use of process data in large-scale educational assessments is becoming more common, it is clear that data on examinees' test-taking behaviors can illuminate their performance, and can have crucial ramifications concerning assessments' validity. A thorough review of the literature in the field may inform researchers and practitioners of common findings as well as existing gaps. This literature review used topic modeling to identify themes in 221 empirical studies using process data in large-scale assessments. We identified six recurring topics: response time models, response time-general, aberrant test-taking behavior, action sequences, complex problem-solving, and digital writing. We also discuss the prominent theories used by studies in each category. Based on these findings, we suggest directions for future research applying process data from large-scale assessments.

The social appreciation of cultural heritage is a process where knowledge, values and relationships that people develop around their heritage assets converge. In the absence of validated instruments to measure this construct, this study has focused on designing and validating a "Measurement Scale for the Social Appreciation of Cultural Heritage" (MS-SACH) for high school students. The questionnaire's content has been approved by nine experts and applied to a pilot sample to assess its comprehension. For construct validation, the questionnaire was applied to a sample of 304 students. The confirmatory factor analysis allowed the retention of ten items grouped into three factors associated with the social appreciation of cultural heritage: (1) knowledge about cultural heritage; (2) values attributable to cultural heritage; and (3) heritage linkages. The results allow us to conclude that the scale developed presents evidence of validity and reliability, whose contribution can be of great value in studies for the evaluation of proposals related to the teaching of cultural heritage.

Large scale international assessments depend on invariance of measurement across countries. An important consideration when observing cross-national differential item functioning (DIF) is whether the DIF actually reflects a source of bias, or might instead be a methodological artifact reflecting item response theory (IRT) model misspecification. Determining the validity of the source of DIF has implications for how it is handled in practice. We demonstrate a form of sensitivity analysis that can point to model misspecification induced by item complexity as a possible cause of DIF, and show how such a cause of DIF might be accommodated through attempts to generalize the IRT model for the studied item(s) in psychometrically and psychologically plausible ways. In both simulated illustrations and empirical data from TIMSS 2011 and TIMSS 2019 4th and 8th Grade Math and Science, we have found that using a form of proposed IRT model generalization can substantially reduce DIF when IRT model misspecification is at least a partial cause of the observed DIF. By demonstrating item complexity as a possible valid source of DIF and showing the effectiveness of the proposed approach, we recommend additional attention toward model generalizations as a means of addressing and/or understanding DIF.

The current article evaluates a national English language proficiency test known as the "MSRT test" which is used to determine the eligibility of candidates for admission to and completion of higher education programs in Iran. Students in all majors take this standardized, high-stake criterion-referenced test to determine if they have obtained the minimum English proficiency level and can be graduated. The present paper seeks to examine the test and its psychometrics characteristics due to the significance of such a high-stakes examination that may have social and long-lasting effects on the participants. It is claimed that the test measures participants' "knowledge of language" for communication rather than their "knowledge about language" in a constrained context. As a result, the test dimensionality and validity are up for debate. It was found that fundamental revisions in terms of test format and content are required to improve the test quality. The current study examined the areas that are yet unexplored and attempts to describe the MSRT assessment comprehensively.

Background: Authentic situations are considered a source of learning due to their real world relevance. This can encourage learners to acquire new knowledge. Increasing digitisation and associated resources, such as professional development opportunities for teachers, technology tools, or digital equipment for schools enable the development and implementation of authentic assessments. The basic academic principles for acquiring economic literacy are already provided in lower secondary school. This article examines, using the example of a new authentic technology-based assessment (TBA)--Economic Literacy-Assessing the Status Quo in Grade 8 (ECON 2022)--, the processes involved in constructing a TBA. The purpose is to develop a curricular valid measurement instrument for surveying the current state of economic literacy in the 8th grade of a German federal state. This study explores which economic competencies students--typically between 14 and 15 years of age--possess in Grade 8, and what level of competence can therefore be expected of them at the beginning of a vocational training programme. The assessment is geared toward the curriculum of the subject of economics and is based on a domain model. This article presents the background and construction process for the development of ECON 2022 as a TBA. Methods: To check the validity of test construction with a focus on the implementation of the authentic assessment and an analysis of difficulty-generating characteristics, the ECON 2022 test items were validated with an expert survey (N = 25). The two-stage data analysis comprised a descriptive quantifying analysis of the rating from the difficulty-generating characteristics specificity, cognitive demand and modelling and the design criterion authenticity. A set of experts rated the criteria. The expert survey was then compared with a previously conducted rating by the research team. The analysis of free-text comments on individual items was carried out discursively and qualitatively by the research team. Both sources of information were used to adapt the test items to measured item difficulties from the field test. For this purpose, items of great difficulty were changed to slightly easier items. In this context, the paper focuses on two central research questions: - How does the authenticity of a test environment relate to difficulty-generating criteria at item level? - Does the authenticity of a test environment have an impact on test results? Results: Results are as follows. (1) The ECON 2022 assessment offers an example of a test design in which the use of TBAs can support innovative and interactive item development. (2) Using the ECON 2022 assessment enabled the implementation of an assessment of economic literacy using authentic situations and the mapping of different facets of economic competence. (3) The validation study showed that the actual item difficulty did not correlate significantly with the authenticity of the assessment, and authenticity thus did not contribute to item difficulty. Conclusion: The results of the study show that we were successful in developing an authentic TBA in ECON 2022. ECON 2022 allows us to examine economic literacy in schools with a high degree of curricular validity and relevance and to analyse what level of competence and knowledge can be expected of students when they enter a vocational training occupation.

Contribution: This article describes the process used to create a questionnaire to evaluate the usability of mobile learning applications (CECAM). The questionnaire includes specific questions to assess user interface usability and pedagogical usability. Background: Nowadays, mobile applications are expanding rapidly and are commonly used in educational institutions to support the learning and teaching process. But the possible deficient usability could decrease the utility of learning activities and the student's motivation. Therefore, careful planning and design by the developer are required, along with a usability evaluation of the applications. Research Questions: How could an instrument be developed to evaluate the usability of m-learning applications that combine technical and pedagogical aspects? How can the quality of the developed instrument be determined? Methodology: A structured questionnaire was created like a measuring tool to evaluate and design m-learning applications. Different statistical techniques, including reliability and validity assessments, were employed to evaluate the quality of the instrument, which is determined through the calibration of the CECAM survey. Findings: After the validity analysis of the questionnaire, a scale with 56 items was obtained, with an alpha reliability coefficient of 0.911 (an excellent measuring scale). It pretends to be used by teachers to design or evaluate m-learning applications, improve their usability, and enhance the students' learning experience.

The Meaningful Learning in the Laboratory Instrument (MLLI) was designed to measure students' expectations before and after their laboratory courses and experiences. Although the MLLI has been used in various studies and laboratory environments to investigate students' cognitive and affective laboratory expectations, the authors of the instrument reported a discrepancy between the intended factor structure of the MLLI and the factor structure suggested by the data collected in preliminary studies. Therefore, the aim of this study was to investigate the validity and reliability evidence related to data collected with the MLLI, especially that related to structural validity. Evidence to support structural validity would provide greater meaning for the reporting and interpretation of MLLI scores. In this study, two possible "a priori" models for the factor structure of data collected from multiple institutions with the MLLI were investigated using confirmatory factory analysis (CFA). This initial investigation found poor data-model fit for each of the two tested models. Cognitive interviews and free response items were then used to inform modifications to the two "a priori structures," and a third alternative structure, which included a negative method factor, was also investigated. Once a best fitting model was identified, further model revisions were informed by a combination of modification indices and qualitative data. Evidence of adequate-to-good data model fit was found for the final revised version of the MLLI, deemed the MLLIv2. Additionally, evidence of both internal structure validity and single administration reliability were found for each of the MLLIv2 factors. The structure of the data from these items leads to scale scores that likely represent student expectations that contribute to meaningful learning and student expectations that detract from meaningful learning. As the results of this study provide the first psychometrically supported scales for MLLI data, they have implications on the future reporting and analyses of MLLI scores.

Various factors such as regulatory body mandates, graduate employability challenges, decreasing student engagement and increasing academic misconduct in higher education have motivated universities to explore alternative approaches to teach and assess. Accordingly, the oral assessment has taken precedence in many contexts as a popular form of assessment. Although literature highlights the strengths of oral assessments, there are also reservations among some scholars as it is found to contribute to issues pertaining to validity and reliability. This paper reports on a systematic review undertaken in accordance with the "Preferred Reporting Items for Systematic Reviews and Meta-Analyses" (PRISMA) guidelines to explore the extent of oral assessment validity, reliability and capacity to address academic misconduct in higher education. A total of 2,657 journal articles from ERIC, Web of Science, Scopus and A+ Education databases were imported into "Covidence" for screening of titles, abstracts and full texts. Seventeen studies were deemed suitable for inclusion in this systematic review. The analysis identified that the validity, reliability and capacity of the oral assessment to reduce academic integrity breaches were dependent on whether it has been designed, scaffolded, and implemented well.

C-Test is a gap-filling test designed to measure first and second language proficiency. Over the past four decades, researchers have shown the fit of C-Test data to parametric item response theory (IRT) models, but no study so far has shown the fit of C-Tests to nonparametric IRT models. The purpose of this study is to contribute to the ongoing C-Test validation project by providing evidence of fit to the Mokken scale analysis as a widely used nonparametric IRT model. A six-passage C-Test battery was analyzed using the monotone homogeneity model and the double monotonicity model of Mokken. Unidimensionality was evaluated using the automatic item selection procedure. Findings showed that the C-Test passages form a strong unidimensional scale, fit well to the monotone homogeneity model, and, after deleting one item, have a sufficient fit to the double monotonicity model. The findings also indicated that the items form a hierarchy, and persons can be located on an ordinal scale using their C-Test sum scores. Implications of the study for C-Test validity and application are discussed.

With the already widespread nature of multiple-choice assessments and the increasing popularity of answer-until-correct, it is important to have methods available for exploring the validity of these types of assessments as they are developed. This work analyzes a 20-question multiple choice assessment covering introductory undergraduate chemistry topics which was given to students in an answer-until-correct manner. Response process validity was investigated through one-on-one think-aloud interviews with undergraduate chemistry students. Answer-until-correct validity was also explored using an analysis of partial credit assignments. Results indicated the convenience of the quantitative partial credit method came at great cost to the precision of validity issue detection and is therefore not a valid shortcut to more rich qualitative approaches. The repeated attempt processing issue detection (RAPID) method is a novel method developed as a combination of response process and answer-until-correct validity. Results from this new method revealed validity issues that were undetected from the use of either approach individually or in concert.

This study sought to propose the Undesirable Behavior Strategies Questionnaire--Physical Education (UBSQ-PE), based on Doyle, W. (1977). Paradigms for research on teacher effectiveness. "Review of Research in Education," 5(1), 163-198. classroom ecology paradigm, and validate its use within Physical Education. 289 physical educators participated in the study (80 of them in the pilot study) and completed a survey evaluating the strategies they used across the classroom ecology task systems. An exploratory factor analysis proposed a four-factor solution that measured teachers' strategies for "Differentiation of Instruction, Interaction for Students, Classroom Management," and "Interaction with Students" and explained a cumulative 47.45% of the total explained variance (KMO = 0.809, [chi-squared subscript 231] = 1,266, 684, p < 0.001). A confirmatory factor analysis indicated a good fit of the data to the four-factor solution ([chi-squared]/df ratio = 1.52, comparative fit index = 0.906, Tucker Lewis index = 0.890, standardized root mean square residual = 0.079, root mean square error of approximation = 0.05), providing evidence for the instrument's internal consistency. Multiple analyses of variance showed that there were differences in teachers' strategies based on their gender, age, studies, professional knowledge, and school level. The use of UBSQ-PE asserts that teachers use undesirable behavior (UB) strategies according to their personal and professional characteristics. Teacher education programs could use this information for supporting PE teachers to adjust their UB strategies in response to each classroom's interrelated task systems.

Daily living skills may affect repetitive behavior patterns in individuals with autism spectrum disorder (ASD). This study aimed to validate a scale to assess activities of daily living at home (SAADL) in children and adolescents with ASD. Content validity by expert judgment showed Aiken V values between 0.75 and 0.88. The exploratory factor analysis (EFA) showed factor loadings (0.62 to 0.89) and high communalities. The % explanation of variance for dimension 1 (personal care) was 51.5% and for dimension 2 (mobility in the home) was 66.3%. The Cronbach's reliability was 0.87 and the retest values reflected high values of precision and accuracy. The SAADL is valid and reliable and can be used to assess activities of daily living of self-care and mobility at home.

In response to the calls for making key stakeholders' perspectives relevant in the test validation process, the study discussed in this report sought test-taker feedback as part of collecting validity evidence and supporting the ongoing field testing efforts of the new "TOEFL ITP"® Speaking section. Specifically, I aimed to investigate the extent to which test takers' perceptions of the newly proposed ITP test tasks are in agreement with the intended characteristics and qualities of the tasks. In addition, I opted to gather insights into whether the speaking tasks are perceived as acceptable by its prospective test takers and also to identify any unwarranted challenges posed for them in completing the tasks. A two-part questionnaire was thus administered during field testing of the new speaking section, and resulting data were analyzed both quantitatively and qualitatively. Findings emerging from the questionnaire data suggest that test-taker perceptions can be used to provide support to corroborate the intended (or hypothesized) properties of the tasks, while pointing to several areas for further monitoring and improvement.

Based on the concept of sustainability and the aim of addressing China?s great need for talent with green skills, a green skills scale was designed with the help of Chinese university students. Students from four institutions of higher education who had agreed to the Principles for Responsible Management Education (PRME) participated as research subjects. With reference to the reviewed literature, a green skills scale was developed for Chinese university students. The scale was reviewed by experts and scholars and subjected to a reliability and validity analysis, and 900 students were selected to be included in the sample. The green skills of university students are divided into four dimensions: cognition of green knowledge, mastery of green skills, application of green skills, and green skills development, resulting in a total of 16 questions. The overall reliability of the scale is 0.845, the reliability of the dimensions lies between 0.813 and 0.880, the correlation coefficients between the dimensions lie between 0.185 and 0.497, and the root-mean-square values of the AVE of the corresponding coefficients are all larger than the correlation coefficients of the corresponding row and column variables, demonstrating that the green skills scale for university students has good reliability and validity.

While discussions on the efficiency and adequacy of distance education, which has passed through various stages from the 1700s to the present day, and its accessibility has become easier with the opportunities provided by technology, it has to be experienced by almost everyone after the pandemic in 2019. While there are various scale studies for distance education before 2019 in the literature, there is a need for an up-to-date scale with proven validity and reliability. Therefore, this study aimed to develop an up-to-date scale with proven validity and reliability for the use of researchers who aim to study this subject. The item pool of the scale was designed considering the attitude and its sub-components as positive and negative. The scale, which took its final form before the application with the help of an assessment and evaluation specialist and a language expert, was applied to a group of 341 students consisting of Gazi University Education Faculty, Haci Bayram Veli University Fine Arts Faculty, Haci Bayram Veli University Faculty of Literature. As a result of Exploratory Factor Analysis and Confirmatory Factor Analysis, a structure consisting of 30 items with six factors was revealed. Cronbach's alpha values of the factors were measured as 0.84, 0.71, 0.85, 0.70, 0.75, and 0.76, respectively. The total Cronbach's alpha value of the scale, which constituted the whole of the factors, was 0.92. As a result of the analyses performed at the end of this research, a valid and reliable scale was put forward to measure the attitudes toward online education. [Note: The publication month (June) shown on the PDF is incorrect. The correct month of publication is December.]

The purpose of this study was to adapt a survey developed by Chai, Jong et al. (2019) on technological pedagogical STEM (TP-STEM) knowledge into Turkish and to determine the knowledge of pre-service and in-service teachers about TP-STEM. The original survey consisted of four factors and a total of 17 items. These factors included Technological Pedagogical Knowledge in Science (TPSK), Technological Pedagogical Knowledge in Mathematics (TPMK), Technological Pedagogical Knowledge in Engineering (TPEK), and Integrative STEM (iSTEM) knowledge. The participants of the study were 520 pre-service and in-service teachers. The analysis showed that the model fit indices for the validity of the factor structures were acceptable with a value of RMSEA=0.0621 and showed excellent agreement with the values SRMR=0.0346, CFI =0.961, TLI=0.953. The Cronbach's alpha values for the factors ranged from 0.80 to 0.84 ([alpha]-value >0.70). These results mean that the survey adapted to Turkish language was reliable and valid for further research. The results showed that pre-service and in-service mathematics teachers had lower self-efficacy on the subfactors (TPSK, TPMK, TPEK, and iSTEM) than pre-service and in-service science and computer teachers.

Emotion knowledge has attracted scientific interest in recent years due to its relevance to children's adjustment. Although there is some controversy as to its definition and components, the term is often used to describe the set of abilities to process emotional information. We need rigorous tools to assess it in multiple languages and contexts. The Assessment of Children's Emotional Skills-ACES was administered to 255 Spanish 8- to-11-year-old children along with, measures of adjustment and language. The Spanish ACES confirmed its three-factor structure; emotion recognition in faces, behaviours and situational emotional knowledge, and a second-order factor (i.e., overall emotional knowledge). The validity of this Spanish version of ACES was demonstrated in emotion knowledge relating to receptive vocabulary, age, sex and adjustment, with age, language and sex moderating relations between emotion knowledge and adjustment. The Spanish ACES can contribute to a greater understanding of the development of emotion knowledge in Spanish-speaking children; and help evaluate social and emotional intervention programmes.

The purpose of this study was to investigate the validity of the vocabulary subsection of a high-stakes university entrance exam for Ph.D. programs using the argument-based approach. All the three different versions of the test administered in a period of five years and the responses of 12,500 test-takers were studied. The study focused on four inferences of domain definition, evaluation, generalization and explanation mainly using corpus linguistics, the Rash measurement model and factor analysis. The results indicated substantial threats to the validity of the test in terms of vocabulary choice, item difficulty, item discrimination, construct representation, and reliability.

Massive open online courses (MOOCs) have emerged as an affordable way to distribute knowledge and democratize education. The examination of online courses calls for theoretical models and instruments that contemplate its particularities. The community of inquiry (CoI) framework has been used in several studies to analyze the effectiveness of online education and hybrid education, including MOOCs. This study aimed to translate and validate the Community of Inquiry Survey instrument (Arbaugh et al., 2008) into Brazilian Portuguese, and used a two-stage methodological design for translating and validating a questionnaire. In the first stage, we carried out translation, back-translation, and cross-cultural adaptation. We translated the 34 items while maintaining the survey's original format. In the expert evaluation phase, all items were considered understandable and essential for inclusion in the Brazilian Portuguese version of the CoI instrument. In the second stage, a prospective cross-sectional study was conducted to validate the questionnaire, and data was collected from participants completing the "Nursing Assessment" MOOC available on the Lúmina platform. A total of 690 responses were gathered. The resulting instrument produced excellent results, and the three presences achieved high reliability indexes, clearly indicating their adequacy. Furthermore, this study proved the validation of the CoI instrument, maintaining the three-factor structure previously reported in the literature corresponding to the three presences: teaching, social, and cognitive presence. We recommend further studies to evaluate the need for excluding or altering cognitive presence items.

Although an accurate measure of vocabulary size is integral to understanding the proficiency of language learners, the validity of multiple-choice (M/C) vocabulary tests to determine this has been questioned due to users guessing correct answers which inflates scores. In this paper the nature of guessing and partial knowledge used when taking the Vocabulary Size Test (VST) is examined. The analysis evaluates the thought processes of test takers through think-aloud protocols and self-reports. This provides a taxonomy of seven types of guesses used by learners while taking the VST and measures the frequency of their occurrence. Based on qualitative coding, guesses are investigated to determine if they exhibit partial word knowledge which is relevant to the test construct. The findings suggest that both guesses resulting from the use of partial knowledge and random guesses are included in estimates and this has a detrimental effect on test accuracy. The paper concludes that the VST does not provide an accurate measure of the vocabulary necessary for reading due to guessing and the meaning-recognition format. It also suggests that the role of partial knowledge should be considered when producing more sophisticated vocabulary tests in the future.

The Pediatric Symptom Checklist-17 (PSC-17) is a popular screening instrument used by parents and clinicians to assess children's behavioral functioning. However, more schools are examining the potential of the PSC-17 as part of a Multi-Tier System of Support framework. To investigate the potential of the PSC-17 in the schools, a sample of 1,779 U.S. preschool and kindergarten-aged children rated by parents (n = 667) and teachers (n = 1,112) was used to assess the measurement invariance of the PSC-17 across respondent groups. Multiple-group Confirmatory Factor Analysis supported partial scalar invariance for the PSC-17, indicating functional equivalence across teacher and parent respondents. Latent mean testing revealed teachers rated children as exhibiting a lower level of Externalizing Problems relative to parents; however, no significant differences in children's Internalizing Problems and Attention Problems were found between teacher and parent ratings.

The objective of this study was to validate a bilingual Spanish-English version of the Vocabulary Size Test (VST) considering its potential use as a discriminator between learners in terms of language competence. This version was designed based on the two forms available on one of the creators' websites as well as considering practices recommended regarding the elimination of cognates and loans. A one-way ANOVA test was used to confirm the test's capacity to discriminate among learners of different linguistic competence. Additionally, Principal Axis Factoring (PAF) was conducted to revise the existence of only one underlying variable. As a result of this study, a VST version for Spanish speakers consisting of 9 vocabulary frequency levels is shared. This version is in line with validation standards put forward in previous research. It is expected that this instrument will help future studies that seek to measure Spanish speakers' competence in English as a foreign or second language without having to deal with the interference of other intervening factors.

Cloze tests are gap-filling tests designed to measure overall language ability and reading comprehension in a second language. Due to their ease of construction and scoring, cloze tests are widely used in the context of second and foreign language testing. Previous research over the past decades has shown the reliability and validity of cloze tests in different contexts. However, due to the interdependent structure of cloze test items, item response theory models have not been applied to analyze cloze tests. In this research, we apply a method to circumvent the problem of local dependence for analyzing cloze tests with the Rasch model. Using this method, we applied the Rasch model to a cloze test composed of eight passages each containing 8-15 gaps. Findings showed that the Rasch model fits the data and thus it is possible to scale persons and cloze passages on an interval unidimensional scale. The test had high reliability and was well-targeted to the examinees. Implications of the study are discussed.

This article describes the development of the Early Elementary Writing Rubric (EEWR), an analytic assessment designed to measure kindergarten and first-grade writing and inform educators' instruction. Crocker and Algina's (1986) approach to instrument development and validation was used as a guide to create and refine the writing measure. Study 1 describes the development of the 10-item measure (response scale ranges from 0 = "Beginning of Kindergarten" to 5 = "End of First Grade"). Educators participated in focus groups, expert panel review, cognitive interviews, and pretesting as part of the instrument development process. Study 2 evaluates measurement quality in terms of score reliability and validity. Data from writing samples produced by 634 students in kindergarten and first-grade classrooms were collected during pilot testing. An exploratory factor analysis was conducted to evaluate the psychometric properties of the EEWR. A one-factor model fit the data for all writing genres and all scoring elements were retained with loadings ranging from 0.49 to 0.92. Internal consistency reliability was high and ranged from 0.89 to 0.91. Interrater reliability between the researcher and participants varied from poor to good and means ranged from 52% to 72%. First-grade students received higher scores than kindergartners on all 10 scoring elements. The EEWR holds promise as an acceptable, useful, and psychometrically sound measure of early writing. Further iterative development is needed to fully investigate its ability to accurately identify the present level of student performance and to determine sensitivity to developmental and instruction gains.

The purpose of this research was to explore the factors associated with online learning anxiety by carefully designing, developing, and providing preliminary validity and reliability evidence of a scale to measure undergraduate students' online learning anxiety. We created a conceptual framework to organize the literature surrounding online learning anxiety and used this framework to develop an initial item pool of 30 items. The researchers recruited N = 297 undergraduate student participants from four public universities in the southeastern United States from whom we collected and analyzed data for descriptive statistics, internal consistency reliability, exploratory factor analysis, and correlational analysis. Following systematic analytic procedures, we arrived at a three-factor model explaining approximately 65% of the variability in these data and retained 24 items in the final model with minimal cross-loadings in the pattern matrix. We labeled the identified factors as (1) online learner feelings of negativity and inadequacy, (2) online learner apprehension towards personal communication, and (3) online learner discomfort with instructor capacity and communication. The final instrument was named the Online Learner Anxiety Scale (OLAS). Scores on the OLAS were correlated with five other measures hypothesized to relate to online learning anxiety thereby providing stronger construct validity evidence. The OLAS was found to produce reliable scores that can be validly inferred as measures of online learning anxiety among undergraduate students in institutions of higher education. These findings are discussed and framed in light of current literature on online learning and possible future research directions.

This study involved the creation and validation of a self-rated social presence measure. Study 1 utilized focus groups to create items. The focus group participants were presented with a set of items based upon past literature; through discussion of these items, a preliminary measure was created. Study 2 involved an exploratory factor analysis on the preliminary measure to eliminate items that did not work well with each other. This reduced the measure from 54 to 23 items. Study 3 validated a 21-item self-rated measure of creating a social presence (PSP), which can be used to determine if people have difficulty projecting themselves as real individuals willing to interact with other online communicators.

In today's digital age, tests are increasingly being delivered on computers. Many of these computer-based tests (CBTs) have been adapted from paper-based tests (PBTs). However, this change in mode of test administration has the potential to introduce construct-irrelevant variance, affecting the validity of score interpretations. Because of this, when scores from a CBT are to be interpreted in the same way as a PBT, evidence is needed to support the reliability and validity these scores (AERA et al. 2014). Numerous studies have investigated the impact of changing the mode of test delivery from paper to computer, not only in terms of their psychometric properties, but also with regard to possible sources of construct-irrelevant variance. This article summarizes the main lessons learned from mode effects studies in education over the past 30 years and discusses some of the questions remaining.

This study aimed to develop the "Lesson Study Model Perception Scale" (LSMPS) and to determine the psychometric properties of the scale. The study was designed as a survey model. A survey model is a research approach that aims to describe a situation as it is. When all the findings obtained to determine the psychometric properties of the scale developed within the scope of the research are evaluated together, it is investigated that the measurements obtained from the scale and the inferences to be made based on these findings are reliable and valid. Existing literature revealed that there are limited number of studies that determine how the LS model is perceived by teachers. It is seen that none of these studies attempted to develop a scale to measure teachers' perceptions of the LS model. The lack of a study in this context in the literature shows the originality of the study.

This study aimed to study the validity and reliability of the Uskudar Benevolence and Malevolency Scale (USBEMA). The sample consists of 1028 people from all over Turkey. The analysis made revealed that the scale consists of two factors. The first factor "Purpose Oriented" explains 35.2% of the variance and the second factor "Process Oriented" 7.9% of the variance. The two-factor structure, consisting of a total of 35 items in total, explained 50,6% of the total variance. In addition, two factors confirmed the USBEMA in the confirmatory factor analysis (Chi-square/degrees of freedom: 4.09; RMSEA: 0.079; NFI: 0.80; NNFI: 0.81; CFI: 0.84; GFI: 0.86; AGFI: 0.83). The internal consistency coefficient (a) of the scale was found to be 0.92. As a result of the study, it was understood that the scale is valid and reliable. Thus, A valid and reliable USBEMA Scale emerged, which was rated as "I do not accept", "I accept", "I accept but I cannot do it, I'll feel regret", and "I believe and implement with a sense of responsibility by planning" and "I always believe and implement with love" that psychometrically measures the benevolence/malevolence attitudes and perceptions of individuals.

Stress among teachers is a debilitating issue that has been plaguing the educational landscape in Malaysia for a long time. Past studies have investigated this issue, and most studies focused on identifying the causes of stress among teachers. However, recent studies have looked at stress as a process that involves a threat (stressors), an appraisal and defence mechanism against the threat (coping mechanism) and the breakdown of the defence (effect). Therefore, it is essential to look at teachers' stress from these three angles. This study aims to introduce a new stress index for Malaysian teachers that seeks to identify teachers' stressors, coping mechanisms and effects of stress. The Fuzzy Delphi method was employed in constructing the stress index as this method used both experts' opinions and analytics that produced an index with high validity and reliability. A proposed stress index with 101 items was evaluated by a panel of experts from linguistics medicine, psychology, linguistics and counselling. Their inputs were then analysed using the Fuzzy Delphi method. The items were then refined with experts' consultation. Furthermore, the finalised stress index proposed a new indicator of stressor, financial problem, an indicator absent in the previously developed stress index. This study has shown the effectiveness of using the Fuzzy Delphi method in developing a new stress index to help teachers, school administrators, and the Ministry of Education understand teacher stress in Malaysia.

The teaching profession in the researcher's context is becoming a machine-like job because teachers' followership i.e. competence, commitment, courage, and self-management, is least prioritized, and more focus is given only on their regularity and punctuality. Effective followership is desired among the teachers for the achievement of educational goals but ignored and resultantly, there is no scale on teachers' followership. Moreover, the followership measures particularly for teachers in the research literature are rare and the available measures lack recommended psychometric evidence. Therefore, the effort to develop a followership scale for teachers (FST) has been made. Through literature review, 55 items were constructed. Five items were discarded from the item pool having poor Content Validity Ratio i.e. below 0.42 that is estimated through the judgment of 14 experts. The items were administered using google form in the what's app groups of teachers and 346 school teachers working in high schools have responded. Exploratory factor analysis using varimax rotation and confirmatory factor analysis on AMOS-21 was performed for construct validation of the scale. The scale was finalized containing 16 items with no item representing less than 0.40 load. Subscales of followership indicate moderate correlation and the overall Cronbach alpha of the scale was estimated at 0.88. Finally, FST includes four subscales i.e. (a) Competence, (b) Commitment, (c) Courage, and (d) Self-management. The scale containing recommended psychometric properties is available for use.

Alienation is a psychological state in which a teacher feels and develops relatively loss of relationship with others. First and foremost, in the present study, it was found that the standardized Alienation Scale by Hardeo Ojha constructed in the year 2010 was appropriate in measuring alienation in general social context for adults above twenty one years of age and it required a few modifications to cater specifically in terms of secondary school teachers context which led to further developing and validating alienation scale on secondary school teachers. The psychometric properties of alienation of secondary school teachers were explored. The study assessed the level of alienation dimension-wise of two hundred secondary school teachers teaching in Punjab (India). The scale was made reliable and valid using Content validity, Exploratory Factor Analysis, Confirmatory Factor Analysis, Cronbach's alpha, Average Variance Extracted, Composite Reliability. The reliability statistics using Cronbach's alpha is greater than 0.80 of overall alienation scale ([alpha]=.878), social isolation ([alpha]=0.892), work alienation ([alpha]=0.869) and cultural estrangement ([alpha]=0.875) with average variance extracted greater than 0.40 and composite reliability greater than 0.70 for all the factors. Therefore, validity and reliability analysis showed that the scale possesses adequate homogeneity and internal consistency. Thus, the alienation scale can be considered appropriate to measure level of alienation of secondary school teachers. Secondly, for field testing the applicability of the developed and validated scale of alienation, it was administered on 500 secondary school teachers. The objective was to conduct comparative analysis of alienation and its dimensions with respect to gender. The findings revealed no significant difference with respect to gender in alienation, social isolation and cultural estrangement. Significant differences were found in case of work alienation wherein the female secondary school teachers were on the higher end in comparison to the male secondary school teachers.

During the COVID-19 pandemic, various misinformation circulating among the public has the potential to be a source of misconceptions. This study aimed to validate the online three-tier diagnostic instrument test for Virus and COVID-19 as well as analyze the level of misconceptions experienced by students. The Treagust model is used as a reference for the instrument development process. Three public high schools providing 187 second grade students who had previously studied Virus content were involved as research respondents, The data analysis techniques used were descriptive statistics. Construct and content validity obtained from expert validation reached 89.88% and 97.12%, respectively. The results of data analysis also inform that each item was valid and the instrument was reliable. The average percentage of students who experience misconceptions was 39.96%. The average percentage of students who understand the topic correctly was 32.10%. The percentage of lucky guess reached 14.83% and students with low level of knowledge reached 13.10%. The diagnostic test as overall, was successful in detecting students' misconceptions. To overcome misconceptions and improve the quality of learning during a pandemic, biology learning must apply creative learning models, methodologies, and teaching materials.

The aim of this study is to develop a rubric to assess mathematical reasoning competence. Since the aim is to assess a competency, the frameworks of the PISA exams in the literature, which give an important place to competencies, have been examined. Due to its focus and in-depth analysis of mathematical reasoning, each of the actions expected from the mathematical reasoning process in the PISA 2021 Mathematics Framework was handled as a criterion and a rubric was created. Data were collected from 30 mathematics teachers with the data collection tool consisting of questions that were stated to be suitable for assessing mathematical reasoning skills in the resource presented as a broadened perspective for the PISA 2021 Mathematics field. The collected data were analyzed using the finalized rubric. In reliability analysis; The "percentage of agreement" among researchers was used to determine the criteria included in the rubric and to be followed with the data collection tool, "Cohen's kappa coefficient" and "Krippendorff's alpha coefficient" methods were used for the agreement between the raters after the analysis of the collected data. The findings show that the Reasoning Competence Rubric (RCR), which consists of 12 criteria, is valid and reliable.

This study validated a scale of students' learning satisfaction (SLS) and self-regulated learning (SRL) strategies in MOOCs to measure the relationship between SRL and SLS based on Zimmerman's SRL theory and Piccoli's web-based Virtual Learning Environment (VLE) effectiveness theory. The process of developing the scale included literatures review, face and content validity, pilot study, data collection, and data analysis. An online self-reporting questionnaire was designed to include constructs such as goal setting (GS), task strategy (TS), self-evaluation (SE), help-seeking (HS), environment structuring (ES), time management (TM) and students' satisfaction (SA). All these constructs were assessed on a 5-point Likert scale. The data was collected from 333 Malaysian students in higher education, and it was analysed by using Partial Least Squares Structural Equation Modelling (PLS-SEM) technique. The resultant constructs have shown empirical result of good reliability and validity where the coefficients of Cronbach's alpha were ranged between 0.784 to 0.877, HTMT values were passed of 0.90 tests and factors loading were more than 0.7. The resulting constructs can be used by the practitioners, academicians, researchers, and policy makers to have a clear understanding on SLS and SRL in MOOC so that, a suitable, effective, positive, and meaningful MOOC learning environment could be established in the universities, higher education learning institutions and/or training organizations.

This research aimed to investigate animation-based achievement tests according to the item format, psychometric features, students' performance, and gender. The study sample consisted of 52 fifth-grade students in Samsun/Turkey in 2017-2018. Measures of the research were open-ended (OE), animation-based open-ended (AOE), multiple-choice (MC), and animation-based multiple-choice (AMC) achievement tests developed for the unit "Motions of the Earth" of the subject area "Earth and Universe" of the science course. Tests were compared to item difficulty, item discrimination, internal consistency levels, and students' performance and gender. Additionally, predicting students' science grades was tested by animation-based open-ended and multiple-choice tests. Paired sample t-tests, Pearson's correlation coefficient, Friedman test, Wilcoxon test, two-way mixed ANOVA tests, and multiple regression analysis were applied to analyze research data. Research results show no significant difference between animation-based and traditional tests' validity and reliability levels. Research result also shows that students' test scores were significantly higher at animation-based tests than traditional tests. Besides, students' test scores differed according to the animation type used in tests. However, gender was not a significant variable on students' test scores. Finally, despite using animation as a significant predictor of Science grades, just animation based multiple-choice test has significantly predicted students' Science grades. Future research can investigate variables that affect students' participation in animation-based tests and their opinions on animation-based tests. The finding regarding the animation type variable can be investigated in-depth in terms of the effect levels of the visual and auditory elements added to the animations by including an equal number of items on the test.

The purpose of this multiple study is to investigate the effect of a psychoeducational training program on adolescents' emotional awareness skills as well as adapting an emotional awareness instrument into the Turkish language and context. In this regard, this paper reports the findings from two studies. In Study I, the validity and reliability analyses were conducted to test the generalizability of the emotion awareness scale in a different context. The findings showed that the scale has a six-factor structure. In addition, the reliability of the scale was found satisfactory. In study II, an experimental research design with the randomized pre and post-test control group was conducted with the participation of adolescents. The experimental group attended a seven-session emotional awareness training while the control group attended regular guidance activities in the school. The findings indicated that the students in the experimental group had significantly higher scores of emotional awareness and verbal sharing of emotions, compared with the students in the control group. However, no significant difference was observed for differentiating emotions, not hiding emotions, bodily awareness, attending to others' emotions, and analyses of emotions. The findings were discussed considering the relevant literature and the cultural characteristics in emotional awareness, and recommendations were presented, accordingly.

Anxiety is an affective variable that impacts learning negatively by impeding cognitive progress. Anxiety is associated with learners' individual differences such as beliefs, attitudes, expectations, motivation, and emotions. It has been determined in many studies that anxiety affects language learning negatively and interferes with many types of learning. The purpose of this study is to develop a measurement tool that will determine the level of anxiety students experience or may experience during the learning process of grammar. The population of the study consists of Turkish Language Teaching students receiving education in the academic year of 2019-2020. The sample of the study comprised of 328 participants who were selected by the random sampling method among the students studying at public university in Turkey. When scale score ranges are examined, it is seen that high scores indicate high anxiety, and low scores indicate low anxiety. It was verified as a result of the analysis that the sub-factors of the scale are the components of this structure, which is called grammar anxiety, and that they form the determined structure together. It was concluded that the model and goodness of fit indices are at acceptable levels. The scale sub-factors of Cronbach's Alpha internal consistency coefficient were 0.92 for "emphasis on grammar learning", 0.87 for "learner beliefs" and 0.88 for "grammar learning experience". The internal consistency coefficient obtained for the whole scale was determined as 0.92. The obtained values also show that this scale is a reliable measurement tool for measuring grammar anxiety.

This study aims to develop a valid and reliable "Teacher Anxiety Scale for Organizing Trips to Out of School Learning Environments" to measure teachers' anxiety about out-of-school learning environments. This research focuses on the methodological validity and reliability of the scale. The study group of the research consisted of 394 teachers working in public and private schools affiliated to the Ministry of National Education in different cities of Turkey. In the development of the scale, the processes of literature review, creating an item pool, obtaining expert opinion, pilot test, creating the trial form, data collection, data analysis (exploratory factor analysis, repeated exploratory factor analysis, confirmatory factor analysis, item analysis, and reliability analysis), and creating the final form were followed. As a result of the analysis, a valid and reliable scale consisting of 28 items and four factors was created. The factors of the scale were named "Bureaucracy-Related (BR) Anxiety", "Safety Risks-Related (SRR) Anxiety", "Harm-Related (HR) Anxiety" and "Pedagogy-Related (PR) Anxiety", respectively. While the Cronbach's Alpha reliability coefficient for the total scale was determined as 0.944; the reliability coefficients for the factors were determined as BR Anxiety Cr [alpha]=0.868, SRR Anxiety Cr [alpha]=0.922, HR Anxiety Cr [alpha]=0.903, and PR Anxiety Cr [alpha]=0.952, respectively. The results indicate that the scale is quite reliable.

This study aimed to develop a scale to determine teacher candidates' perceptions of the European Union (EU). A descriptive survey method was used in the study, and the study group consisted of 908 teacher candidates attending the Kirsehir Ahi Evran University Faculty of Education in the fall semester of 2021-2022 academic year. The participants were selected using a convenient sampling technique. The Perception Scale of the European Union comprised 20 items and three factors: anxiety, contribution, and culture was employed as the main research tool. The Cronbach's Alpha internal consistency coefficient was calculated for the said scale as 0.746, which indicated scale reliability. Data were analyzed using percentage, frequency, mean, and standard deviation techniques. Additionally, t-test and ANOVA were performed to identify the differences between samples. The study results showed that the teacher candidates' anxiety levels for the European Union were high, and their perceptions of the "contribution" and "culture" factors were moderate. The analysis of perceptions of the EU and political literacy skills of the participants revealed a significant difference in the "contribution" factor but no significant difference in the "anxiety" and "culture" factors. According to the relationship between teachers' perceptions of the EU and their cultural literacy skills, there was a significant difference in the "culture" factor but no significant difference appeared in the "anxiety" and "contribution" factors.

The present study investigated the speaking module of TOEFL iBT with an emphasis on the dichotomy of independent and integrated tasks. The potential differences between the two speaking conditions were intended to be explored based on the oral performance elicited from a group of Iranian test takers. To collect the required data, a simulated version of TOEFL iBT characterized with all test rubrics as practiced by ETS was applied. A total number of 224 oral samples were transcribed and then quantified in terms of complexity and fluency by some internet-based programs including "Coh-Metrix 3" and "Syllable Counter." Moreover, the accuracy of each oral performance was manually computed for comparison purposes. The results revealed that the two speaking contexts were significantly different in terms of linguistic accuracy as well as two complexity measures representing latent semantic analysis and the mean number of modifiers per noun phrase. However, the two speech-eliciting contexts did not trigger any significant difference in the participants' oral language performance in terms of fluency and two other complexity measures, namely lexical diversity and left-embeddedness. The findings may have some implications for all TOEFL iBT stakeholders concerning the speaking subtest.

The present study aims to develop a valid and reliable pedagogical literacy scale based on teachers' views. It also aims to investigate the variations in pedagogical literacy levels of teachers based on gender, educational level, and professional seniority variables to test the functionality of the pedagogical literacy scale. The study was conducted with teachers employed in educational institutions at various levels in Malatya Province, Turkey. The exploratory factor analysis was conducted with 345 active teachers employed in Yesilyurt District in Malatya Province during the 2021-2022 academic year. Dynamic factor analysis (DFA) was conducted with 375 active teachers employed in Battalgazi District in Malatya Province during the 2021-2022 academic years, and the functionality of the scale was determined with 412 teachers employed in various districts in Malatya Province. In the study, the draft item pool included 37 items before the exploratory factor analysis. However, the experts suggested revision of three items and removal of four items, and the exploratory factor analysis (EFA) was conducted on 33 items. EFA results revealed that the scale included 4 factors in 25 total items. Confirmatory factor analysis (CFA) revealed that the 4-factor 25-item scale was acceptable based on excellent fit indices. The Cronbach Alpha coefficient of the scale that included 25 items and four factors was 901. The analysis was conducted to test the functionality of the scale and it was determined that the pedagogical literacy levels of the teachers were good. Furthermore, it was concluded that there were significant differences between the pedagogical literacy levels of the teachers based on gender, educational level and professional seniority variables.

The objective of this study was to examine possible antecedents and consequences of teachers' emotions in the classroom. Based on a cognitive-social perspective and self-determination theory, we examined the relationship between basic psychological needs (BPNs), teachers' class-related emotions and teachers' life satisfaction. A sample of 595 teachers from Andalusia (Spain) participated in an online survey. A structural equation model was tested, in which BPNs predicted teachers' emotions ([beta] = 0.69; p < 0.001 positive emotions and [beta] = -0.42; p < 0.001 negative emotions). In addition, BPNs ([beta] = 0.36; p < 0.001) and positive emotions ([beta] = 0.23; p < 0.001) predicted satisfaction with life. The results show that the fulfilment of work-related BPNs is important to generate positive emotions and well-being in teachers. In addition, the study is the first to provide extensive details on the psychometric properties for assessing teacher emotions with the Achievement Emotions Questionnaire - Teachers (AEQ-T) in a Spanish sample.