Search

Large-scale performance assessment programs are a longstanding reform tool. However, standard setting can be a challenge for assessment programs that use primarily non-standardized assessments. The purpose of this paper is to extend this field of research by explaining the standard setting methodology applied to one more recent instantiation of a state performance assessment program. The second purpose of this paper is to discuss the data quality control and quality assurance challenges experienced after five years of applying the standard setting method. Recognizing the burgeoning interest again in large-scale performance assessment programs, the goal and intended contribution of this paper is to inform future decisions about selecting appropriate standard setting methods and dealing with unanticipated challenges that may arise during implementation based upon the lessons learned from one program. It is likely that other large-scale performance assessment programs may face similar operational challenges, especially those that do not rely on standardized tests or standardized administration procedures to produce annual determinations of student proficiency or other scores used for accountability purposes. Assessment system designers can use the insights in this paper to consider standard setting methods and how those methods may need to be adapted to promote technical quality.

Scientific methods have received widespread attention in recent years. Based on the analytical framework derived from Brandon's matrix consisting of four categories of scientific methods, this paper aims to conduct a content analysis to examine how the diversity of scientific methods is represented in college entrance chemistry examination papers from three exam boards in China. It was found that the percentages of the four categories of scientific methods in the examination papers varied significantly from each exam board, highlighting an imbalanced representation of scientific methods. Furthermore, among the four categories of scientific methods, non-manipulative parameter measurement (Non-MPM) accounted for the largest proportion in each examination paper, while the proportion of manipulative hypothesis testing (MHT) was very small, indicating that the practical chemistry items in China are less experimental. At the end of this paper, the implications of the findings and suggestions for further studies are discussed.

The study explores the unaddressed washback on the English-speaking test of the Hong Kong Diploma of Secondary Education Examination (HKDSE-English Language Paper 4). The HKDSE is a university entrance exam in Hong Kong which includes a group discussion speaking test known as English Language Paper 4. To identify ongoing washback effects, including sociocultural perspectives, the study analyses ongoing student washback of test-specific and non-specific preparation and how the mediating factors affect these preparations. One-hundred and ninety-nine secondary 4th to 6th grade students provided the questionnaire data in two rounds. Results from exploratory factor analysis indicate six major types of test preparation, which are categorised as two test specific, three non-specific, and one integrated type of preparation. Mediating factors comprise five categories of learner characteristics and stakeholders in and out of school. The cluster analyses identified four students' groups intertwined with the involvement of washback and the mediating factors. A strong relationship between extrinsic factors and explicit learning and intrinsic factors and implicit learning was found. The study concludes that students' washback is affected by mediating factors aside from the test; learning method choice by students (e.g. learning from entertainment content) has a strong relationship with specific mediating factors (e.g. interest in English). [ABSTRACT FROM AUTHOR]

Analysis of programming process data has become popular in computing education research and educational data mining in the last decade. This type of data is quantitative, often of high temporal resolution, and it can be collected non-intrusively while the student is in a natural setting. Many levels of granularity can be obtained, such as submission, compilation, edit, and keystroke events, with keystroke-level logs being the most fine-grained of commonly used dataset types. However, the lack of open datasets, especially at the keystroke level, is notable. There are several reasons for this failing, with the most prominent being the challenges of deidentification that are peculiar to keystroke log data. In this paper, we present the public release of two fully deidentified keystroke datasets that are the first of their kind in terms of both event and metadata richness. We describe our collection technique and properties of the data along with deidentification techniques that, while not fully relieving researchers of significant effort, at least reduce and streamline manual work in hopes that researchers will release similar datasets in the future.

This paper deals with the results of students during the preparing courses of mathematics for entrance exams at an university. These courses take place every year and since the Covid-19 period, they are not only face-to-face, but also online. The study involves students of secondary schools who apply to the university of economics. This report compares results of tests of two different classes of these students and from different parts of mathematics. These tests are in the form of online quizzes. We do not prove if there are differences between the scores of students of short-time or long-time courses. In addition, we compare the results with the students from the year before. We also emphasize the more problematic topics of mathematics.

Contemporary technology, along with the tangible/virtual duality of present-day learner consciousness, are, in the authors' opinion, necessitating and enabling a fundamental educational transformation. A central feature of this development will be 'letting the learners lead', including (from secondary onwards) their identification of the content of their curriculum. This paper considers how this pedagogical innovation might operate in practice. Nine United Kingdom (specifically South Wales) senior secondary students were invited to decide what they would "like to learn in the coming academic year". The group then explored how their agreed set of subjects (including, for instance, Sign Language; History of Coal Mining in My Locality; Mental Health and Me; Being Ready for the Next Pandemic; Pantomime…) might best be learned. These approaches and findings were shared with, understood by, and generally supported by a group of upper secondary school students in Australia and, to a lesser extent, by another group in Bangladesh. Dialogue with a small number of professors and other United Kingdom professionals also took place. The tentative conclusions and implications are that (a) university entrance requirements are predominantly aimed at student selection rather than at covering necessary pre-tertiary knowledge; (b) if, somehow, the competitive element -- including the allocation of grades -- could be eliminated, then giving learners the opportunity to shape their curricula becomes feasible; (c) letting the learners lead is seen (by a sample of them) as realistic and (with some reservations) desirable; (d) much of this learning would involve on-line research through the confident utilisation of familiar devices and systems (i.e. how 'non-school' information is now acquired); (e) there would also be on-line and face-to-face discussions with experts and practitioners; (f) teachers, with radically restructured roles, could offer valuable coordination and encouragement; and (g) the school as 'place' (as opposed to 'process') may wither away as the transformation proceeds. The authors are keen that their provisional findings should be tested by educational planners and researchers in other educational situations across the world.

This meta-analysis assesses the predictive validity of the Graduate Record Examination (GRE) across outcome variables, including grade point average, for graduate students. In addition to aggregate effects, this paper also assessed changes in observed effects over time as related to increasing diversity in the graduate student population and as a function of gender and racial/ethnic composition of study samples. Framed using a lens of critical whiteness, this analysis examined n = 1,659 individual effects across k = 201 studies. Overall, 62.3% of reported effects were nonsignificant (i.e. no predictive value of GRE scores on student outcomes). Further, the magnitude of observed predictive relationships decreased significantly over time. The aggregate mean effect across all studies and outcomes was small, significant, and positive: GRE score predicted 3.24% of variance across measured outcomes, 4% of variance in overall GPA, and 2.56% of variance in first-year graduate GPA. Sample composition effects by race/ethnicity were notable under some conditions, but nonsignificant, with increasing proportions of people of Color within a study sample associated with poorer predictive validity for GPA. Likewise, the magnitude of negative effects where lower GRE scores predicted stronger student outcomes showed increasing trends from 0.16% of variance for all-white samples to 7.3% for samples comprised entirely of people of Color.

English language teachers' assessment ability to assess all areas of pupils' learning is important for comprehending how learners advance across the curriculum and guiding them in their development (Livingston & Hutchinson, 2016). The educational system of Iran decided to shift from traditional teaching methodologies toward communicative approaches. This conversion would not turn into reality unless teachers could apply it in practice. However, teachers have certain needs such as expanding and broadening contextual-related knowledge, cooperating with their colleagues, and developing their assessment literacy (Coombe, Vafadar, & Mohebbi, 2020). To identify the teachers' understandings of, practices in, and challenges of assessment, 15 English language headteachers (English language teachers who are responsible for moderating the group of the teachers' activities) participated in the interviews of this study, followed by questionnaires for exploring teachers' needs. The interviews were coded and content analyzed independently by the researcher and an expert in assessment. The main themes and needs were derived from the interview analyses and are presented in eight pivots. The findings of the questionnaires manifested the priorities that teachers felt concerning assessment literacy and classroom-based assessment needs. The paper will discuss the findings concerning assisting teachers' professional development in assessment literacy. Implications are also provided.

Since the restoration of the "Gaokao" (College Entrance Examination) in 1977 and the nationwide higher education expansion that started in 1998, millions of rural students have enrolled in urban universities. Though Chinese rural students' educational trajectories have received extensive attention, their subjective experiences in the rapidly changing context of the higher education system remain insufficiently represented in the literature. This paper utilises a life history approach to explore three successful rural students' journeys towards and through higher education, graduating from university in the 1980s, 1990s, and 2000s. Through interweaving the social context and individual accounts in this paper, I examine how educational aspirations (as individual choices and strategies) are shaped by changes in the social milieus in which one is situated and how rural students perceive classed differences in the context of an expanding urban-rural divide. I highlight the point that rural families' aspirations towards higher education are not simply personal preference but rather are constructed within the nexus of perceived limitations and opportunities and are complexly shaped by the social and economic context. I argue that, though the forms of symbolic domination might vary across the studied timeframe--from clothes to extra-curricular activities to comprehensive ability--the rural students' disadvantaged situation in the urban university remained largely unchanged.

Inclusion has been increasingly recognised as a global common goal in education. In China, inclusive education for children with special educational needs and disabilities is currently practised as 'Learning in Regular Classrooms' (LRC). However, not only has the inclusion policy frequently been criticised as failing to provide clear, systematic, or consistent strategies, but also the actual practices of LRC have also been found to merely concern children's physical integration into the mainstream settings. This discussion paper explores key structural barriers to inclusive education through reviewing policies within the wider education ecosystem that are relevant to the theme of inclusion but outside the LRC policy itself. The analysis illustrates how the complex and interlocked structural barriers embedded within the wider context of current education policies pose persisting constraints for inclusion to progress in China, and how these structural barriers unique to the Chinese education system also reflect the common 'wicked problems' for practising inclusive education globally. The paper discusses five main education structures: neo-liberal education policies, national college entrance examination system, teacher evaluation system, staffing quota system and the 'combine medicine and education' policy. The conclusion illustrates the complexity and main issues facing future inclusion policy reforms and highlights key objectives for policy change. It indicates that, for inclusion reforms to be effective, broader changes are needed within the wider education ecosystem.

This paper deals with a course of mathematics at an university in the Czech Republic. This course is for incoming students. It prepares students for math entrance exams. During the coronavirus pandemic period (2020-2021) we taught the course online. We had to solve the problem with communication with students and obtain feedback from them. For this reason, we used interactive quizzes. In this article, we compare the situation at other universities, and we introduce online quizzes. Further, we compare their success rate during the course. In addition, we present the results from the student´s course evaluation survey.

As testing programs transition from paper to online testing, they must study mode comparability to support the exchangeability of scores from different testing modes. To that end, a series of three mode comparability studies was conducted during the 2019-2020 academic year with examinees randomly assigned to take the ACT college admissions exam on paper or online. Unlike many prior mode comparability studies, examinees testing online earned higher scores on average than examinees testing on paper, especially on the English, reading, and writing tests. Much of the observed mode effects were accounted for by items near the ends of the tests. Subsequent examination of item omit rates, differential item functioning by mode, and change-point analyses supported the notion that the observed mode effects were related to differential speededness. Specifically, examinees testing online were slightly less speeded, and were therefore able to perform better on items near the ends of the tests. Practical implications for linking across modes, test design, and test administration conditions are discussed.

Purpose: The purpose of this paper is to explore the impact of a year-long writing intervention located in an urban high school in partnership with a university teacher education professor and the students. The goals were as follows: to increase student self-efficacy about writing overall; to increase the number of students who successfully improve scores on writing assessments; and to increase ACT exam scores, making students more qualified candidates for college admission. Design/methodology/approach: The main method used for the research is a quantitative longitudinal study method, which involves collecting from each student, at pre-intervention and post-intervention, a writing sample. There are 54 students in the study. Findings: Findings showed an increase in student proficiency overall, with an increase of 0.53 holistically. Effective writing instruction asks students to write often to develop their writing will and skill. There are many more factors beyond the teaching of "ACT prep" writing skills that must be considered, particularly when teaching marginalized groups. This study found that the following are important: building and sustaining positive rapport; using their language as part of the learning process; creating space for students to write regularly; writing on a variety of topics; reflecting on and monitoring their writing; and receiving timely and targeted feedback. Originality/value: This paper discusses the impact of the writing intervention and describes the practices that were a part of the intervention for marginalized students in an urban school to increase their writing scores.

Post-transformational neoliberal reforms in the Czech Republic have established a competitive quasi-market, in which -- at the lower-secondary level -- academic-track schools (grammar schools or gymnasia) compete for students in the regular-track schools. These reforms have also brought a rise of shadow education (private supplementary tutoring) provided by individuals and private companies. Even public schools are sometimes providers of shadow education. The study explores gymnasia principals' motivations to provide paid courses to prepare students for entrance examinations to their study programmes and the challenges they experience in doing so. The findings are drawn mainly from semi-structured interviews with principals and other school management members in eleven gymnasia. The study conceptualises such courses as a 'double-edged sword'. First, because it serves as a 'weapon' that academic schools use in the quasi-market competition to attract more (high-achieving) students from regular track schools, and second, because it may have both positive and negative consequences (e.g., for equity in education), depending on the conditions of its provision. The paper contributes to a wider scholarly literature by expanding the understanding of ways schools behave in the competitive educational quasi-markets and by opening a new branch of shadow education research.

The paper argues that contemporary test validity theory places the consequences of testing on the lives of all college applicants at the back of the test validation argument. It introduces the notion of test efficacy as a process to gather evidence on claims on consequences of testing on all college applicants that can be traced back to validity. The paper proposes a test efficacy framework to evaluate test efficacy claims on the impact of admission examinations on all college applicants (not just those attaining the admission standard).

The entrance exams to Japanese universities have long been criticised for creating negative "washback" on English education in high schools in Japan, as teachers have often focused their teaching efforts on teaching grammar to prepare students for these exams rather than helping students to develop competency in spoken English. Many government initiatives have attempted to reform English language education by encouraging a greater focus on speaking competency. This paper outlines the nature of the Japanese university entrance exam system, considers the phenomenon of washback, and critically reviews the body of empirical research related to washback in the Japanese context. A systematic review was conducted on articles published between 1993 and June, 2022. These articles were subjected to thematic analysis in which four broad categories were identified: learner behaviour; washback to teacher behaviour; affective factors; and washback specifically from the introduction of the listening test. Findings suggest that more empirical studies are required to understand the complex nature of washback from university entrance exams, and that future research is required to examine potential washback from four-skills testing and, in particular, from the inclusion of a speaking test.

This paper examines the issue of student choice across subject streams in recent government reforms to China's National College Entrance Exam (NCEE). Utilizing data from a case study of an urban high school in Beijing, it argues that student choice of subjects across previous streams of science or fine arts runs counter to existing institutional structures at the secondary and higher education level, consideration of pedagogical techniques that differ by streams, and a "bias" for science grounded in sociohistorical concerns of national development.

This paper reports the findings of a study that investigated why a large number of test-takers show a low performance in four language skills (listening, speaking, reading, and writing) offered in an entry test used by a College of Technology in Oman to place students in advanced diploma and bachelor's degree programs. The main research question was meant to find out what difficulties the test takers face in the three tests (Listening, reading, and writing) in two semesters. The study analyzed the entry test results for two semesters including survey results obtained from the test takers regarding the difficulties experienced by them in the three tests and an analysis of writing answer scripts. The findings indicate that most test takers were not qualified enough to pursue their studies in the advanced diploma or bachelor's degree programs due to their low performance in the entry test. The findings, moreover, suggest that the current placement test poses difficulties for most test-takers. Therefore, based on the findings, the possible reasons for the low performance of the test takers and the fairness of the entry test are explicitly discussed and finally, suggestions and recommendations for addressing the issue of the current entry test are offered.

Since its inception in 1949, over 1,500 studies have investigated the validity of the GRE General Test to predict its performance criteria in higher education (Klieger, Bridgeman, Tannenbaum, & Cline, 2016). The present review paper sought to examine the predictive validity of the GRE General Test. Factors affecting the predictive validity (e.g., range restriction, compensatory selection, criterion unreliability, substantive and artifactual moderators, bias in testing, coaching effects, socioeconomic status (SES), gender, and a host of other intervening factors such as motivation, communication skills, etc.) have been discussed. A brief overview of GRE revised General Test format is also presented. After an account of the related review of the literature, a critical commentary on the predictive validity of the GRE General Test has been discussed with an emphasis on the role of criterion unreliability and SES factor effects.

The Canadian Academic English Language (CAEL) assessment is a proficiency test for university admission. Most research has focused on the paper edition, not the recent computer edition (CE). Validity research on other proficiency tests found that relationships between proficiency scores and academic achievement measures can be tenuous because other factors intervene in students' chances of success. Research on academic success has examined academic acculturation as one factor. However, the impact of acculturation on academic achievement has not been examined alongside language proficiency test scores. The current study investigated this issue by focusing on students with a range of CAEL scores and examining the relationship between CAEL-CE scores, academic achievement, and academic acculturation. Forty-four students at a Canadian English-medium university participated in the study. Students' grade point averages were obtained, and an academic acculturation questionnaire was administered. A subset of participants took part in focus groups. The findings and their implications are discussed.

This study presents a systematic review of the literature on the predicting student retention in higher education through machine learning algorithms based on measures such as dropout risk, attrition risk, and completion risk. A systematic review methodology was employed comprised of review protocol, requirements for study selection, and analysis of paper classification. The review aims to answer the following research questions: (1) what techniques are currently used to predict student retention rates, (2) which techniques have shown better performance under specific contexts?, (3) which factors influence the prediction of completion rates in higher education?, and (4) what are the challenges with predicting student retention? Increasing student retention in higher education is critical in order to increase graduation rates. Further, predicting student retention provides insight into opportunities for intentional student advising. The review provides a research perspective related to predicting student retention using machine learning through several key findings such as the identification of the factors utilized in past studies and methodologies used for prediction. These findings can be used to develop more comprehensive studies to further increase the prediction capability and; therefore, develop strategies to improve student retention.

International English Language Testing System (IELTS) scores have been widely used as the language entry requirements for international students in many English-medium universities. However, research on the predictive validity of IELTS on subsequent academic performance has been inconclusive. Additionally, despite the proliferation of IELTS preparation courses, very few attempts have been made to explore the extent to which these courses support course takers' development of knowledge and skills needed for their subsequent university study. This paper investigated (a) the predictive validity of IELTS results on the subsequent academic performance of Vietnamese international students at UK universities, and (b) the extent to which an IELTS preparation course in Vietnam supported its course takers in their subsequent academic study in the UK. Questionnaires with 80 Vietnamese international students from 31 universities across the UK showed that there was a positive significant correlation between these students' IELTS scores and their academic results. Follow-up interviews with two undergraduates and two postgraduates who had previously studied in the same IELTS preparation course, but were studying different academic disciplines at different institutions in the UK, and analysis of the materials used in their IELTS preparation course and university courses revealed that the course positively supported these students' subsequent academic study, but certain skills and knowledge needed in their academic study were not effectively covered in the IELTS preparation course.

In this paper, we study how well SAT scores, high school GPA, and undergraduate precalculus grades relate to undergraduate calculus grades at a 4-year private nonprofit university that does not have a placement exam. In addition, we compare the success in calculus of two groups of students, both of whom require two semesters to pass calculus. The first group takes calculus twice (failing the first attempt) and the second group takes precalculus followed by calculus.

The effects of affirmative action on the incentives to human capital accumulation are ambiguous from a theoretical perspective and the scarce empirical evidence on the matter provides mixed results. In this paper, we address this issue by investigating the impacts of Brazil's Law of Quotas on the students' performance in the college entrance exam, the ENEM. We provide causal evidence that the law fostered incentives to pre-college human capital accumulation, inducing students to attain higher ENEM scores. Moreover, the effects of the quotas were greater in quantitative-intensive subjects and these impacts increased throughout the first years after the law's implementation.