Search

This research employed a mixed-methods approach to explore the best practices of ordinary national educational testing (O-NET) to improve the quality of basic education. The methodology was divided into four phases, the first of which was a survey and analysis of the current situation at ONET. The sample group was made up of representatives of the school under the office of the basic education commission (OBEC), primary education, comprising the school administrators and teachers teaching in four subjects. Phase 2, "multiple case study," is qualitative research by selecting specific case studies for 10 primary schools based on criteria for selection and collecting data through in-depth interviews. A semi-structured interview form was completed by 30 key informants. Phase 3, the synthesis (draft) of best practices, and phase 4, the checking and proposing of best practices by 25 assessors. The results showed the best practices in using the test results of O-NET to improve the quality of basic education were 4 components: i) school management (16 practice guidelines); ii) learning management (10 practice guidelines); iii) student promotion (6 practice guidelines); and iv) parent communications (3 practice guidelines). The best practices were in accuracy, propriety, feasibility, and utilization; all were at the highest level.

Though a substantial amount of research exists on imputing missing scores in educational assessments, there is little research on cases where responses or scores to an item are missing for all test takers. In this paper, we tackled the problem of imputing missing scores for tests for which the responses to an item are missing for all test takers. We considered three missing-data imputation methods--the median method, the item response theory (IRT) method, and the two-way method--for imputing scores. We compared the performance of these three imputation methods with respect to their accuracy in estimating scaled scores and test reliability for the aforementioned problem. Real data were used in the comparison. All three methods performed well in imputing scaled scores with negligible imputation error: The IRT method and the median method provided slightly more accurate scaled scores. The two-way method provided the most accurate reliability estimates. Recommendations for practice are provided.

The pervasive issue of cheating in educational tests has emerged as a paramount concern within the realm of education, prompting scholars to explore diverse methodologies for identifying potential transgressors. While machine learning models have been extensively investigated for this purpose, the untapped potential of TabNet, an intricate deep neural network model, remains uncharted territory. Within this study, a comprehensive evaluation and comparison of 12 base models (naive Bayes, linear discriminant analysis, Gaussian process, support vector machine, decision tree, random forest, Extreme Gradient Boosting (XGBoost), AdaBoost, logistic regression, k-nearest neighbors, multilayer perceptron, and TabNet) was undertaken to scrutinize their predictive capabilities. The area under the receiver operating characteristic curve (AUC) was employed as the performance metric for evaluation. Impressively, the findings underscored the supremacy of TabNet (AUC = 0.85) over its counterparts, signifying the profound aptitude of deep neural network models in tackling tabular tasks, such as the detection of academic dishonesty. Encouraged by these outcomes, we proceeded to synergistically amalgamate the two most efficacious models, TabNet (AUC = 0.85) and AdaBoost (AUC = 0.81), resulting in the creation of an ensemble model christened TabNet-AdaBoost (AUC = 0.92). The emergence of this novel hybrid approach exhibited considerable potential in research endeavors within this domain. Importantly, our investigation has unveiled fresh insights into the utilization of deep neural network models for the purpose of identifying cheating in educational tests.

In order to investigate the influence of separation of grade distributions and ratio of common items on the precision of vertical scaling, this simulation study chooses common item design and first grade as base grade. There are four grades with 1,000 students each to take part in a test which has 100 items. Monte Carlo simulation method is used to simulate the response matrices by self-made program in R 3.0. As the items are scored by 0/1, we select two-parameter logistic model of Item Response Theory and use BILOG-MG for concurrent calibration with EAP method. The Bias and RMSE are calculated as precision indicators. The results show that: (1) Estimation precision of item and ability parameters differs in different grades. For discrimination and difficulty parameters, estimation precision is higher as closer to the base grade and is lower with the increase of effect size. For the ability parameters, the estimation precision is high generally except for fourth grade which is much lower. The precision is best at 0.5 of effect size in general. (2) There is an interaction between the ratio of common items to total test and effect size. When the effect size is 0.5 and 1.0, estimation precision of each grade is most accurate at 30% of common-item ratio. When the effect size is 1.5, the estimation precision of difficulty parameters is best for first, second, and third grade at 30% of common-item ratio while grade 4 at 15% of common-item ratio. The ability parameters of all grades are all best estimated at 15% of common item ratio. There must be a trade-off between the estimation precision of ability parameters and item parameter if the common item ratio is at the range of 15% to 30%. (3) The choice of base grade affects the accuracy of vertical scaling. When the lower grade is selected as the base grade, if the number of consecutive cumulative conversions from the upper grade test score to the lower grade exceeds 2, there will be a large deviation. Therefore if the senior grade changes to the junior grade, it is suggested that the gap of grades should not exceed 2 grades. As a whole, the proportion of anchor items for vertical scaling is set at 30%, but it is better to set the proportion of anchor items as "variable" value (15%-30%) when considering the separation of grade distributions.

During the pandemic three widespread shifts in teaching cultures affected digital writing pedagogies: resilient teaching, open teaching, and inclusive teaching. Resilient teaching design emerged as a strategy to counter the unpredictability of public health policies on class delivery modes, and emphasised designing for maximising student interactions as a response. Open teaching started as a response to a lack of access to textbooks and evolved to transform functions normally reserved for teaching into learning activities. In addition, inclusive teaching practices, developed as a response to racial and social injustices, resulted in deliberate emphasis on class structure to incorporate all students. Although seemingly disparate and disconnected from the issues of technology that normally influence the teaching of digital writing, each shift focused on student needs and predict a future for digital writing pedagogy.

This study seeks to share teachers' and principals' voices from middle schools in the Southeastern United States who are attempting to help their students achieve academic success. Obtaining a snapshot of middle schools in the Southeastern United States provides opportunities to recognize trends and identify challenges about the current implementation of middle level programs and practices related to academics. The last large-scale survey was completed over 10 years ago (McEwin & Greene, 2010, 2011) and was focused on principal voices only. When attempting to assess the level of implementation of various school improvement efforts or to gain a deeper understanding of the challenges faced daily in the classroom, the voices of teachers and principals are powerful voices that must be heard. Among other results, educators called for more meaningful collaboration among elementary and middle schools to address gaps in content and skills, and a better focus on educating the whole child in their schools. Implications and future directions are discussed.

The problem of this research is how to use assessment for school learning and what are students' creative thinking skills? Research objectives: First, to uncover, analyze, identify, describe the learning assessment used by teachers and students of Nusantara Indah Sintang Senior High School. Second, to express, analyze and describe students' creative thinking skills. This study uses a descriptive qualitative research method of survey type. The population and sample are the teachers and students of Nusantara Indah Sintang Senior High School. Data collection tools in the form of questionnaires and interviews. Data analysis can be descriptive qualitative. The results of the study revealed that the use of assessment for learning in school was suboptimal or infrequent, both from teacher and student questionnaire data, as well as confirmed teacher and student interview data. Likewise, the creative thinking abilities of students who are also not optimally cheated or do it rarely can be seen from the teacher's questionnaire data - an average of 56.13 percent, while the students' questionnaire data, with an average of 1.66, do not agree with the category. In addition, it is confirmed by the interview data that the four indicators of creative thinking were not optimally enabled at school.

The majority of students from elementary to tertiary levels have misunderstandings and challenges acquiring various statistical concepts and skills. However, the existing statistics assessment frameworks challenge practice in a classroom setting. The purpose of this research is to develop and validate a statistical thinking assessment tool involving form one (Grade 7) students' statistical thinking. The SOLO model was applied to develop five testlet tasks. Each testlet task involved four components. This study employed the survey methodology to assess the statistical thinking of 356 form one students. Content validity was determined using the Content Validity Index (CVI). The construct validity was determined using Rasch analysis. The results demonstrated that the instrument for assessing the statistical thinking of the form one students was valid and trustworthy. This finding of the study also revealed new evidence that the instrument allowed the teachers to identify the students' progress effectively based on the hierarchical manner of item levels in the testlet format. The instrument was useful in identifying students' statistical thinking levels. The students' ability to respond appropriately to a task at a particular level reveals their degree of cognitive development. Testlet task was also easy to diagnose the strengths and weaknesses in learning statistics topics.

We explored three approaches to resolving or re-scoring constructed-response items in mixed-format assessments: rater agreement, person fit, and targeted double scoring (TDS). We used a simulation study to consider how the three approaches impact the psychometric properties of student achievement estimates, with an emphasis on person fit. We found that although TDS is not specifically designed to identify discrepant or misfitting score profiles, this approach can improve person fit for students who are also within a critical range of achievement. Our findings also suggest that TDS alone is not sufficient to provide a comprehensive picture of measurement quality because it does not directly consider person fit. We suggest that researchers and practitioners consider combining TDS with other approaches to score resolution. We consider the implications of our findings for research and practice.

Teachers can develop science literacy among pupils through Inquiry-Based Science Education. IBSE helps to evolve the inquiry skills necessary for understanding the world of science and nature. To be able to improve the level of inquiry skills systematically, we need to know the current level among pupils. For this reason, we designed a multiple-choice test that measures the level of inquiry skills among 13 to 15 years old pupils. The design of the test consisted of six steps: selection of inquiry skills, construction and administration of open-ended test, creation of a database of distractors, construction of multiple-choice test, pilot testing and application of optimised test in pedagogical research. The designed test can be considered a valid and reliable research instrument and the individual test items show a relatively high discriminating power, optimal level of difficulty and a high number of functional distractors. By analysing the most common incorrect answers, we were able to identify various difficulties pupils have, regarding the application of inquiry skills in solving items. These results will help us to design appropriate strategies for improving pupils' inquiry skills in science education.

Large-scale educational assessments are sometimes considered low-stakes, increasing the possibility of confounding true performance level with low motivation. These concerns are amplified in remote testing conditions. To remove the effects of low effort levels in responses observed in remote low-stakes testing, several motivation filtering methods can be used to purify the data. We estimated scores from assessment data collected remotely in Spring 2021 six ways, applying examinee-based filtering methods (filtering examinees based on total time) and response-based filtering methods (filtering responses using the effort-moderated IRT model), varying the thresholds selected to separate effortful and non-effortful responses. We compared the 2021 scores (estimated six ways) to those obtained in previous cohorts tested in person. The results support the use of motivation filtering regardless of which method is used. Practical implications regarding method and threshold selection are discussed.

Studies have shown how school professionals deal with large quantities of student test data and accountability policies in different ways. However, the subtle ways in which school leaders and teachers use their time, their power and their talk when dealing with external policy demands in daily practice is understudied. Using data from interviews with formal school leaders and video observations of leadership teams and teaching teams, this paper investigates how school leaders and teachers deal with external testing policy demands in a fairly new policy context of national testing in Norway. The study aims to explore micro-political aspects of the issue, by applying a theoretical framework on leadership, micro-policy actors and ideas about time. The paper contributes to the field of education policy and reform by introducing the role that time plays in micro-politics in general and how time issues contribute to the shaping of stakeholders' understanding of and relationship to external policy demands. Findings surrounding middle leaders as crucial in the micro-policy construction of time and space, suggesting a need for research aimed at understanding school leaders' and teachers' sharing experiences of time and the care for the collective self.

We draw upon Appadurai's 'scapes' and Latour's Actor Network Theory (ANT) to interrogate historical and spatial flows in relation to specific testing technologies. We reveal how testing systems, conceptualised as actor-networks, rearticulate colonial legacies of inequality which are intensified by new and emerging technologies. ANT helps trace social and relational interactions occurring in various national and global data spaces, and helps make sense of incessant transformations in education datascapes. Mapping actor-networks, and translations, help name and navigate neoliberal forces acting through colonial legacies via emerging educational datascapes. Our examination shows how educational technologies associated with standardised testing in northern and southern contexts -- specifically, England, Singapore, Bangladesh and Australia -- exacerbate structural inequalities. We consider how such technologies are actants and reflective of co-existing historical and temporal influences, and global cultural and spatial flows, allowing us to map the multiple ways in which educational technologies are manifested in an emerging education datascape.

This paper follows a line of logical argumentation to claim that what Samuel Messick conceptualized about construct validation has probably been misunderstood by some educational policy makers, practicing educators, and classroom teachers. It argues that, while Messick's unified theory of test validation aimed at (a) warning educational practitioners and policy makers of the undesirable social consequences of test use and (b) entreating educators and test developers to think of a facet-driven item-banking-based construct-specific criterion-referenced common metric for any construct of interest in educational and other settings, his message has been misunderstood as a plea for alternative ways of evaluation and specifically a qualitative shift in educational assessment. The paper (a) draws on the conceptual differences between 'test function' and other construct-irrelevant peripheral 'affordances' to which any test can be put, (b) argues that the moment of truth for the qualitative camp has arrived, and (b) invites everyone to admit that even if qualitative assessment, 'thick' descriptions of achievement, and differentiated portraits and profiles of student performance might be much thicker than traditional norm-referenced psychometric tests, they are no match for any minimalistic facet-driven criterion-referenced common metric, nor is any of them an option. The paper suggests that the right way out is through an iron-clad criterion-referenced Occam's-razor-proof common metric for each construct of interest, perhaps the only option that is sure to transform the soft science of educational assessment into a hard science of educational measurement.

Linking score scales across different tests is considered speculative and fraught, even at the aggregate level. We introduce and illustrate validation methods for aggregate linkages, using the challenge of linking U.S. school district average test scores across states as a motivating example. We show that aggregate linkages can be validated both directly and indirectly under certain conditions such as when the scores for at least some target units (districts) are available on a common test (e.g., the National Assessment of Educational Progress). We introduce precision-adjusted random effects models to estimate linking error, for populations and for subpopulations, for averages and for progress over time. These models allow us to distinguish linking error from sampling variability and illustrate how linking error plays a larger role in aggregates with smaller sample sizes. Assuming that target districts generalize to the full population of districts, we can show that standard errors for district means are generally less than 0.2 standard deviation units, leading to reliabilities above 0.7 for roughly 90% of districts. We also show how sources of imprecision and linking error contribute to both within- and between-state district comparisons within versus between states. This approach is applicable whenever the essential counterfactual question--"what would means/variance/progress for the aggregate units be, had students taken the other test?"--can be answered directly for at least some of the units.

Errors in examination papers and other assessment instruments can compromise fairness. For example, a history question containing an incorrect historical date could be impossible for students to answer. Incorrect instructions at the start of an examination could lead students to answer the wrong number of questions. As there is little research on this issue within the educational assessment community, we reviewed the literature on minimizing errors in other industries and domains, including aviation, energy, and medicine. We identified generalizable principles and applied them to our context of educational assessment. We argue that since assessment instrument construction is a complex system comprising numerous interacting components, a holistic approach to system improvement is required. Assessment instrument errors stem primarily from human failure. When human failure occurs, it is not good enough to suggest that 'to err is simply human'. Instead it is necessary to look deeper, evaluating the latent working conditions that underpin the efficacy of procedures, making the human failure more or less likely. Drawing from the aviation industry's ergonomic SHELLO model, we articulate and explore three of the most critical working conditions that relate to our context: (i) time pressure, (ii) workload and stress, and (iii) wider organizational culture, including good error data collection. We conclude with recommendations for best practice in minimizing errors in assessment instruments. A 'good' error culture should be promoted, which avoids blaming individuals. Errors should be acknowledged readily by all, and system owners should take a scientific approach to understanding and learning from them.

In times of crisis it is good to look back. Not only is it comforting, but better understanding of events in our past can inform decision-making and help us find direction at uncertain times. COVID-19 may have presented new challenges, but this exploration of historical disruptions to school exams highlights themes and a recognisable human spirit. For this study Gillian Cooke and Gill Elliott have identified five crisis events in the history of Cambridge Assessment and tracked responses to these circumstances through documents held in the corporate archive. The flu pandemic of 1918, two World Wars, Indian Partition and the Strikes of the 1970s all, in different ways, pushed the boundaries of examination administration. The search yields both short, and long-term responses and reveals just how much the impact of a crisis is connected to the society that deals with it.

The objective of this research is to study the results of activities to increase the scores of Ordinary National Education Test. Cluster; teachers of Phetchaburi Rajabhat University comparing the results of Ordinary National Education Test in 2017-2018 and studying the satisfaction of the activities. The target group is 49 schools in Phetchaburi and Prachuap Khiri Khan Provinces, data were analyzed using mean and standard deviation. The study found that the difference of the scores of the Ordinary National Education Test was higher in 32 schools and there is a difference in scores of Ordinary National Education Test tests lower by 2 schools, representing 94.12, with the satisfaction of the participation in the activity of increasing the basic educational testing at the basic level is at a high level with an average of 4.22, standard deviations 0.73, which the participants are satisfied with the process. The process of organizing activities was at the highest with an average of 4.28, standard deviations 0.76 and continues organizing activities to increase the scores of Ordinary National Education Test.

Technical difficulties and other unforeseen events occasionally lead to incomplete data on educational tests, which necessitates the reporting of imputed scores to some examinees. While there exist several approaches for reporting imputed scores, there is a lack of any guidance on the reporting of the uncertainty of imputed scores. In this paper, several approaches are suggested for quantifying the uncertainty of imputed scores using measures that are similar in spirit to estimates of reliability and standard error of measurement. A simulation study is performed to examine the properties of the approaches. The approaches are then applied to data from a state test on which some examinees' scores had to be imputed following computer problems. Several recommendations are made for practice.

Response time models (RTMs) are of increasing interest in educational and psychological testing. This article focuses on the lognormal model for response times, which is one of the most popular RTMs. Several existing statistics for testing normality and the fit of factor analysis models are repurposed for testing the fit of the lognormal model. A simulation study and two real data examples demonstrate the usefulness of the statistics. The Shapiro-Wilk test of normality and a "z"-test for factor analysis models were the most powerful in assessing the misfit of the lognormal model. [For the corresponding grantee submission, see ED603049.]

Pearson product-moment correlation coefficient between item g and test score X, known as item-test or item-total correlation ("Rit"), and item-rest correlation ("Rir") are two of the most used classical estimators for item discrimination power (IDP). Both "Rit" and "Rir" underestimate IDP caused by the mismatch of the scales of the item and the score. Underestimation of IDP may be drastic when the difficulty level of the item is extreme. Based on a simulation, in a binary dataset, a good alternative for "Rit" and "Rir" could be the Somers' "D": it reaches the ultimate values +1 and -1, it underestimates IDP remarkably less than "Rit" and "Rir", and, being a robust statistic, it is more stable against the changes in the data structure. Somers' "D" has, however, one major disadvantage in a polytomous case: it tends to underestimate the magnitude of the association of item and score more than "Rit" does when the item scale has four categories or more.

Often, important decisions regarding accountability and placement of students in performance categories are made on the basis of test scores generated from tests, therefore, it is important to evaluate the validity of the inferences derived from test results. One of the threats to the validity of such inferences is aberrant responding. Several person fit indices were developed to detect aberrant responding on educational and psychological tests. The majority of the person fit literature has been focused on creating and evaluating new indices. The aim of this study was to assess the effect of aberrant responding on the accuracy of estimated item parameters and refining estimations by using person fit statistics by means of simulation. Our results showed that the presence of aberrant response patterns created bias in the both b and a parameters at the item level and affected the classification of students, particularly high-performing students, into performance categories regardless of whether aberrant response patterns were present in the data or were removed. The results differed by test length and the percentage of students with aberrant response patterns. Practical and theoretical implications are discussed.

Kelley's Discrimination Index (DI) is a simple and robust, classical non-parametric short-cut to estimate the item discrimination power (IDP) in the practical educational settings. Unlike item-total correlation, DI can reach the ultimate values of +1 and -1, and it is stable against the outliers. Because of the computational easiness, DI is specifically suitable for the rough estimation where the sophisticated tools for item analysis such as IRT modelling are not available as is usual, for example, in the classroom testing. Unlike most of the other traditional indices for IDP, DI uses only the extreme cases of the ordered dataset in the estimation. One deficiency of DI is that it suits only for dichotomous datasets. This article generalizes DI to allow polytomous dataset and flexible cut-offs for selecting the extreme cases. A new algorithm based on the concept of the characteristic vector of the item is introduced to compute the generalized DI (GDI). A new visual method for item analysis, the cut-off curve, is introduced based on the procedure called exhaustive splitting.

Many test developers try to ensure the content validity of their tests by having external experts review the items, e.g. in terms of relevance, difficulty, or clarity. Although this approach is widely accepted, a closer look reveals several pitfalls need to be avoided if experts' advice is to be truly helpful. The purpose of this paper is to exemplarily describe and critically analyse widespread practices of involving experts to ensure the content validity of tests. First, I offer a classification of tasks that experts are given by test developers, as reported on in the respective literature. Second, taking an exploratory approach by means of a qualitative meta-analysis, I review a sample of reports on test development (N = 72) to identify the common current procedures for selecting and consulting experts. Results indicate that often the choice of experts seems to be somewhat arbitrary, the questions posed to experts lack precision, and the methods used to evaluate experts' feedback are questionable. Third, given these findings I explore in more depth what prerequisites are necessary for their contributions to be useful in ensuring the content validity of tests. Main results are (i) that test developers, contrary to some practice, should not ask for information that they can reliably ascertain themselves ("truth-apt statements"). (ii) Average values from the answers of the experts which are often calculated rarely provide reliable information about the quality of test items or a test. (iii) Making judgements about some aspects of the quality of test items (e.g. comprehensibility, plausibility of distractors) could lead experts to unreliable speculations. (iv)When several experts respond by giving incompatible or even contradictory answers, there is almost always no criterion enabling a decision between them. In conclusion, explicit guidelines on this matter need to be elaborated and standardised (above all, by the AERA, APA, and NCME "Standards").

Testing programs are often classified as high or low stakes to indicate how stringently they need to be evaluated. However, in practice, this classification falls short. A high-stakes label is taken to imply that all indicators of measurement quality must meet high standards; whereas a low-stakes label is taken to imply the opposite. This approach can result in inappropriate allocation of resources and inadequate attention to needed evidence. We argue that "stakes" are better thought of as a profile of consequences. We suggest generalizable criteria for evaluating and responding to stakes in testing, with applications to licensure, employment, and K-12 accountability testing.

The Mantel-Haenszel delta difference (MH D-DIF) and the standardized proportion difference (STD P-DIF) are two observed-score methods that have been used to assess differential item functioning (DIF) at Educational Testing Service since the early 1990s. Latentvariable approaches to assessing measurement invariance at the item level have been proposed and studied since then as well. Previous research showed that using the weighted sum score as the matching variable may close the gap between the MH D-DIF statistic based on observed scores and its counterpart based on latent ability. In this study, we show that weighting reduces the difference between the STD P-DIF statistic and its counterpart based on latent ability. In addition, we discuss the factors that influence the gap and examine approaches that may facilitate the use of weighted STD P-DIF.

Takers of educational tests often receive proficiency levels instead of or in addition to scaled scores. For example, proficiency levels are reported for the Advanced Placement (AP®) and U.S. Medical Licensing examinations. Technical difficulties and other unforeseen events occasionally lead to missing item scores and hence to incomplete data on these tests. The reporting of proficiency levels to the examinees with incomplete data requires estimation of the performance of the examinees on the missing part and essentially involves imputation of missing data. In this article, six approaches from the literature on missing data analysis are brought to bear on the problem of reporting of proficiency levels to the examinees with incomplete data. Data from several large-scale educational tests are used to compare the performances of the six approaches to the approach that is operationally used for reporting proficiency levels for these tests. A multiple imputation approach based on chained equations is shown to lead to the most accurate reporting of proficiency levels for data that were missing at random or completely at random, while the model-based approach of Holman and Glas performed the best for data that are missing not at random. Several recommendations are made on the reporting of proficiency levels to the examinees with incomplete data.

If it is not possible for one to critique other research and succeed--or even remain securely employed--in a research profession, how is the profession ever to rid itself of flawed, biased, or fraudulent research? Answer: it will not. Any community that disallows accusations of bad behavior condones bad behavior. Any community that disallows evidence of falsehoods condones falsehoods. In-group leaders can promote falsehoods as truth because a volunteer army of enablers protects them, stamping out any dissident embers as they appear. Most of those who recognize the falsehoods say nothing, given the rational fear of retribution and career stunting. Students, families, and US society as a whole pay the price for US education research's falsehoods, doggedly defended by an aggressive minority in-group, while most education researchers pay no price at all. The falsehoods persist because few of those who know they are false summon the courage to speak up. The implicit pact between the aggressive minority and the passive majority sustains an intellectual monoculture. Frustratingly, the mainstream education press tends to accept what the powerful elites present them at face value, as if US education research were just like that in any other subject field, such as physics or business administration. Indeed, some education reporters casually dismiss reports of censorship, thus ignoring (and supporting) the problem. They seem to assume that such widespread and successful information control could not be possible. (See, for example, Russo 2015.) Should society at large feel OK with this arrangement? In part, Scolding #9 concerned the work of arguably the country's most policy-influential scholar in educational testing. I maintain that his primary contribution to the research literature is not just highly misleading, but fraudulent--he makes claims that he must know are false, yet he recommends basing highly consequential public policies on his work. Yet Scolder #9 talked down his work as a "pet theory" that "he may go overboard on." In other words, not to worry, we can just hide this under the covers inside the profession's tent. Any effect on the public is no concern of ours. Education research is ours to own and manage as we please, and none of the public's business. Yet, the society of education research professionals also wants the public to believe that they can be trusted as arbiters of truth in education policy, even though, for some, truth ranks among the least of their interests. Outing truth requires free inquiry, open debate, vigorous discussion, and conflict, all anathema to those professionals more concerned with their personal career trajectories, the superficial appearance of decorum, or the preservation of appealing myths.

Assessment is one of the five functions of the educational psychologist's (EP's) role, yet there is a dearth of research exploring its distinctive contribution to school-based practice, and a lack of definition about what it is. In this study, the assessment practices of EPs were compared with those of other educational professionals who had achieved certification for competence in educational testing. Data were analysed using descriptive and inferential statistics, with patterns emerging which indicated that, while there is some overlap, assessment practices of EPs and other educational professionals often have different foci. Specifically, evidence suggests that EPs offer a broader, more holistic perspective with strong emphasis on the social and emotional wellbeing of the child. Additionally, other practitioners typically have a more school-orientated focus, for example, testing for exam access arrangements, reflecting their professional role. Implications are discussed, particularly in relation to the distinctive contribution to EP assessment.

Equating error is usually small relative to the magnitude of measurement error, but it could be one of the major sources of error contributing to mean scores of large groups in educational measurement, such as the year-to-year state mean score fluctuations. Though testing programs may routinely calculate the standard error of equating (SEE), the commonly used summary statistics of SEE are not direct quantifications of the impact of equating error on group means. This article proposed summary statistics that directly quantify the impact of equating error on group means or group mean differences and provided empirical and analytical methods to estimate these statistics. Examples based on empirical data were used to illustrate practical applications of these statistics.

Despite agreement about the central importance of validity for educational and psychological testing, consensus regarding the definition of validity remains elusive. Differences in the definition of validity are examined and reveals that a potential cause of disagreement stems from differences in word use and meanings given to key terms commonly employed when discussing validity. A proposal for the meaning and use of specific terms is offered and a framework that divides issues associated with validity is presented. Specifically, the framework divides what Messick termed "an integrated evaluative judgment" into three components: Instrument Validity, Verification of Interpretation and Decision, and Utility of Actions.

This paper presents a study on known approaches for quality assurance of educational test and test items. On its basis a comprehensive approach to the quality assurance of online educational testing is proposed to address the needs of all stakeholders (authors of online tests, teachers, students, experts, quality managers, etc.). According to the proposed approach is developed an original software application Test Quality Evaluation (TQE) for the automation of the stakeholders' activities for quality assurance of educational tests throughout the whole lifecycle. The application retrieves and provides analysis of data from online tests conducted and specially designed surveys for quality evaluation of educational tests by students and experts. It allows tracking and evaluating the quality of educational tests in real time and provides the related quantitative data in different levels of generalization--in the level of a separate educational test, of educational tests of an entire course, or educational tests of a subject area. The software application has been put under real-time testing for quality evaluation of educational tests, included in e-learning courses from different subject areas that prove its applicability.

The article discusses the methodological approach to the technology of the educational experiment performance, the ways of the research data processing by means of research methods and methods of mathematical statistics. The article shows the integrated use of some effective approaches to the training of the students majoring in "Technological education". The purpose of article is to check the efficiency of the created didactic support that helps to ground the students majoring in "Technological education" in the technologies of modern production at the university. It shows the features of introduction of the created didactic set of the hi-tech educational laboratory equipment based on digital technologies with program control into the educational process. The materials of the article can be useful to university lecturers, school teachers (teachers of Physics, Mathematics, Technology etc.) and students of teachers training and technical institutions, colleges, schools.

From 2006 to 2008, Educational Testing Service (ETS) produced a series of reports titled "A Culture of Evidence," designed to capture a changing climate in higher education assessment. A decade later, colleges and universities already face new and different challenges resulting from societal, technological, and scientific influences. This new series of reports describes current issues and trends in assessment and how they relate to the needs of higher education institutions. In this report, we discuss the shift from accountability-based assessment to improvement-based assessment. When conducted for improvement, assessment is more inclusive of institutional faculty and staff, more informative to colleges and universities, and--it is hoped--focused on enhancing instruction and learning. However, these positive outcomes do not come without challenges. Ultimately, assessment-for-improvement has implications for many of the people, processes, and tools used to measure student learning

A common interpretation of achievement test results is that they provide measures of achievement that are much like other measures we commonly use for height, weight, or the cost of goods. In a limited sense, such interpretations are correct, but some nuances of these interpretations have important implications for the use of achievement test results. This paper will contrast two different theoretical underpinnings for the interpretation of educational assessments results as measurements. One of these theoretical views comes from initial attempts in psychology to measure the amount of a trait that was exhibited by a person. The other theoretical view comes from early work in education to measure how much of a desired curriculum was acquired by students. At times, these views conflict with each other and lead test developers and policy makers to ask for the impossible. After summarizing the two theoretical positions, the areas of conflict will be discussed. Finally, some recommendations will be given for what can be done to clarify the issues and minimize the problems that result from using conflicting theoretical frameworks.

This study aims to identify to what degree undergraduate students are able to manage the exam process to be successful in exams. The study group of the research, which utilizes the survey model, consists of 350 students in total, 185 female and 165 male, attending 4 different teaching programs in Faculty of Education, Gazi University. "The Scale of Exam application techniques", developed by Erisen and Celikoz in 2004, was applied to the study group. This 5-point Likert scale consists of 40 items about techniques used by students (1) before the exam, (2) during the exam, and (3) at the end of the exam. Three factors of the scale explain .68 of the total variance and the Cronbach's Alpha Reliability Coefficient of the scale is .95. In the present study, we compared exam process management skills of students based on variables of gender, grade and academic achievement. It was observed that opinions of undergraduate students about their exam process management skills varied according to independent variables.