Search

This research employed a mixed-methods approach to explore the best practices of ordinary national educational testing (O-NET) to improve the quality of basic education. The methodology was divided into four phases, the first of which was a survey and analysis of the current situation at ONET. The sample group was made up of representatives of the school under the office of the basic education commission (OBEC), primary education, comprising the school administrators and teachers teaching in four subjects. Phase 2, "multiple case study," is qualitative research by selecting specific case studies for 10 primary schools based on criteria for selection and collecting data through in-depth interviews. A semi-structured interview form was completed by 30 key informants. Phase 3, the synthesis (draft) of best practices, and phase 4, the checking and proposing of best practices by 25 assessors. The results showed the best practices in using the test results of O-NET to improve the quality of basic education were 4 components: i) school management (16 practice guidelines); ii) learning management (10 practice guidelines); iii) student promotion (6 practice guidelines); and iv) parent communications (3 practice guidelines). The best practices were in accuracy, propriety, feasibility, and utilization; all were at the highest level.

Though a substantial amount of research exists on imputing missing scores in educational assessments, there is little research on cases where responses or scores to an item are missing for all test takers. In this paper, we tackled the problem of imputing missing scores for tests for which the responses to an item are missing for all test takers. We considered three missing-data imputation methods--the median method, the item response theory (IRT) method, and the two-way method--for imputing scores. We compared the performance of these three imputation methods with respect to their accuracy in estimating scaled scores and test reliability for the aforementioned problem. Real data were used in the comparison. All three methods performed well in imputing scaled scores with negligible imputation error: The IRT method and the median method provided slightly more accurate scaled scores. The two-way method provided the most accurate reliability estimates. Recommendations for practice are provided.

The pervasive issue of cheating in educational tests has emerged as a paramount concern within the realm of education, prompting scholars to explore diverse methodologies for identifying potential transgressors. While machine learning models have been extensively investigated for this purpose, the untapped potential of TabNet, an intricate deep neural network model, remains uncharted territory. Within this study, a comprehensive evaluation and comparison of 12 base models (naive Bayes, linear discriminant analysis, Gaussian process, support vector machine, decision tree, random forest, Extreme Gradient Boosting (XGBoost), AdaBoost, logistic regression, k-nearest neighbors, multilayer perceptron, and TabNet) was undertaken to scrutinize their predictive capabilities. The area under the receiver operating characteristic curve (AUC) was employed as the performance metric for evaluation. Impressively, the findings underscored the supremacy of TabNet (AUC = 0.85) over its counterparts, signifying the profound aptitude of deep neural network models in tackling tabular tasks, such as the detection of academic dishonesty. Encouraged by these outcomes, we proceeded to synergistically amalgamate the two most efficacious models, TabNet (AUC = 0.85) and AdaBoost (AUC = 0.81), resulting in the creation of an ensemble model christened TabNet-AdaBoost (AUC = 0.92). The emergence of this novel hybrid approach exhibited considerable potential in research endeavors within this domain. Importantly, our investigation has unveiled fresh insights into the utilization of deep neural network models for the purpose of identifying cheating in educational tests.

In order to investigate the influence of separation of grade distributions and ratio of common items on the precision of vertical scaling, this simulation study chooses common item design and first grade as base grade. There are four grades with 1,000 students each to take part in a test which has 100 items. Monte Carlo simulation method is used to simulate the response matrices by self-made program in R 3.0. As the items are scored by 0/1, we select two-parameter logistic model of Item Response Theory and use BILOG-MG for concurrent calibration with EAP method. The Bias and RMSE are calculated as precision indicators. The results show that: (1) Estimation precision of item and ability parameters differs in different grades. For discrimination and difficulty parameters, estimation precision is higher as closer to the base grade and is lower with the increase of effect size. For the ability parameters, the estimation precision is high generally except for fourth grade which is much lower. The precision is best at 0.5 of effect size in general. (2) There is an interaction between the ratio of common items to total test and effect size. When the effect size is 0.5 and 1.0, estimation precision of each grade is most accurate at 30% of common-item ratio. When the effect size is 1.5, the estimation precision of difficulty parameters is best for first, second, and third grade at 30% of common-item ratio while grade 4 at 15% of common-item ratio. The ability parameters of all grades are all best estimated at 15% of common item ratio. There must be a trade-off between the estimation precision of ability parameters and item parameter if the common item ratio is at the range of 15% to 30%. (3) The choice of base grade affects the accuracy of vertical scaling. When the lower grade is selected as the base grade, if the number of consecutive cumulative conversions from the upper grade test score to the lower grade exceeds 2, there will be a large deviation. Therefore if the senior grade changes to the junior grade, it is suggested that the gap of grades should not exceed 2 grades. As a whole, the proportion of anchor items for vertical scaling is set at 30%, but it is better to set the proportion of anchor items as "variable" value (15%-30%) when considering the separation of grade distributions.

During the pandemic three widespread shifts in teaching cultures affected digital writing pedagogies: resilient teaching, open teaching, and inclusive teaching. Resilient teaching design emerged as a strategy to counter the unpredictability of public health policies on class delivery modes, and emphasised designing for maximising student interactions as a response. Open teaching started as a response to a lack of access to textbooks and evolved to transform functions normally reserved for teaching into learning activities. In addition, inclusive teaching practices, developed as a response to racial and social injustices, resulted in deliberate emphasis on class structure to incorporate all students. Although seemingly disparate and disconnected from the issues of technology that normally influence the teaching of digital writing, each shift focused on student needs and predict a future for digital writing pedagogy.

This study seeks to share teachers' and principals' voices from middle schools in the Southeastern United States who are attempting to help their students achieve academic success. Obtaining a snapshot of middle schools in the Southeastern United States provides opportunities to recognize trends and identify challenges about the current implementation of middle level programs and practices related to academics. The last large-scale survey was completed over 10 years ago (McEwin & Greene, 2010, 2011) and was focused on principal voices only. When attempting to assess the level of implementation of various school improvement efforts or to gain a deeper understanding of the challenges faced daily in the classroom, the voices of teachers and principals are powerful voices that must be heard. Among other results, educators called for more meaningful collaboration among elementary and middle schools to address gaps in content and skills, and a better focus on educating the whole child in their schools. Implications and future directions are discussed.

The problem of this research is how to use assessment for school learning and what are students' creative thinking skills? Research objectives: First, to uncover, analyze, identify, describe the learning assessment used by teachers and students of Nusantara Indah Sintang Senior High School. Second, to express, analyze and describe students' creative thinking skills. This study uses a descriptive qualitative research method of survey type. The population and sample are the teachers and students of Nusantara Indah Sintang Senior High School. Data collection tools in the form of questionnaires and interviews. Data analysis can be descriptive qualitative. The results of the study revealed that the use of assessment for learning in school was suboptimal or infrequent, both from teacher and student questionnaire data, as well as confirmed teacher and student interview data. Likewise, the creative thinking abilities of students who are also not optimally cheated or do it rarely can be seen from the teacher's questionnaire data - an average of 56.13 percent, while the students' questionnaire data, with an average of 1.66, do not agree with the category. In addition, it is confirmed by the interview data that the four indicators of creative thinking were not optimally enabled at school.

The majority of students from elementary to tertiary levels have misunderstandings and challenges acquiring various statistical concepts and skills. However, the existing statistics assessment frameworks challenge practice in a classroom setting. The purpose of this research is to develop and validate a statistical thinking assessment tool involving form one (Grade 7) students' statistical thinking. The SOLO model was applied to develop five testlet tasks. Each testlet task involved four components. This study employed the survey methodology to assess the statistical thinking of 356 form one students. Content validity was determined using the Content Validity Index (CVI). The construct validity was determined using Rasch analysis. The results demonstrated that the instrument for assessing the statistical thinking of the form one students was valid and trustworthy. This finding of the study also revealed new evidence that the instrument allowed the teachers to identify the students' progress effectively based on the hierarchical manner of item levels in the testlet format. The instrument was useful in identifying students' statistical thinking levels. The students' ability to respond appropriately to a task at a particular level reveals their degree of cognitive development. Testlet task was also easy to diagnose the strengths and weaknesses in learning statistics topics.

We explored three approaches to resolving or re-scoring constructed-response items in mixed-format assessments: rater agreement, person fit, and targeted double scoring (TDS). We used a simulation study to consider how the three approaches impact the psychometric properties of student achievement estimates, with an emphasis on person fit. We found that although TDS is not specifically designed to identify discrepant or misfitting score profiles, this approach can improve person fit for students who are also within a critical range of achievement. Our findings also suggest that TDS alone is not sufficient to provide a comprehensive picture of measurement quality because it does not directly consider person fit. We suggest that researchers and practitioners consider combining TDS with other approaches to score resolution. We consider the implications of our findings for research and practice.

Teachers can develop science literacy among pupils through Inquiry-Based Science Education. IBSE helps to evolve the inquiry skills necessary for understanding the world of science and nature. To be able to improve the level of inquiry skills systematically, we need to know the current level among pupils. For this reason, we designed a multiple-choice test that measures the level of inquiry skills among 13 to 15 years old pupils. The design of the test consisted of six steps: selection of inquiry skills, construction and administration of open-ended test, creation of a database of distractors, construction of multiple-choice test, pilot testing and application of optimised test in pedagogical research. The designed test can be considered a valid and reliable research instrument and the individual test items show a relatively high discriminating power, optimal level of difficulty and a high number of functional distractors. By analysing the most common incorrect answers, we were able to identify various difficulties pupils have, regarding the application of inquiry skills in solving items. These results will help us to design appropriate strategies for improving pupils' inquiry skills in science education.

Large-scale educational assessments are sometimes considered low-stakes, increasing the possibility of confounding true performance level with low motivation. These concerns are amplified in remote testing conditions. To remove the effects of low effort levels in responses observed in remote low-stakes testing, several motivation filtering methods can be used to purify the data. We estimated scores from assessment data collected remotely in Spring 2021 six ways, applying examinee-based filtering methods (filtering examinees based on total time) and response-based filtering methods (filtering responses using the effort-moderated IRT model), varying the thresholds selected to separate effortful and non-effortful responses. We compared the 2021 scores (estimated six ways) to those obtained in previous cohorts tested in person. The results support the use of motivation filtering regardless of which method is used. Practical implications regarding method and threshold selection are discussed.

Studies have shown how school professionals deal with large quantities of student test data and accountability policies in different ways. However, the subtle ways in which school leaders and teachers use their time, their power and their talk when dealing with external policy demands in daily practice is understudied. Using data from interviews with formal school leaders and video observations of leadership teams and teaching teams, this paper investigates how school leaders and teachers deal with external testing policy demands in a fairly new policy context of national testing in Norway. The study aims to explore micro-political aspects of the issue, by applying a theoretical framework on leadership, micro-policy actors and ideas about time. The paper contributes to the field of education policy and reform by introducing the role that time plays in micro-politics in general and how time issues contribute to the shaping of stakeholders' understanding of and relationship to external policy demands. Findings surrounding middle leaders as crucial in the micro-policy construction of time and space, suggesting a need for research aimed at understanding school leaders' and teachers' sharing experiences of time and the care for the collective self.

We draw upon Appadurai's 'scapes' and Latour's Actor Network Theory (ANT) to interrogate historical and spatial flows in relation to specific testing technologies. We reveal how testing systems, conceptualised as actor-networks, rearticulate colonial legacies of inequality which are intensified by new and emerging technologies. ANT helps trace social and relational interactions occurring in various national and global data spaces, and helps make sense of incessant transformations in education datascapes. Mapping actor-networks, and translations, help name and navigate neoliberal forces acting through colonial legacies via emerging educational datascapes. Our examination shows how educational technologies associated with standardised testing in northern and southern contexts -- specifically, England, Singapore, Bangladesh and Australia -- exacerbate structural inequalities. We consider how such technologies are actants and reflective of co-existing historical and temporal influences, and global cultural and spatial flows, allowing us to map the multiple ways in which educational technologies are manifested in an emerging education datascape.

Technical difficulties and other unforeseen events occasionally lead to incomplete data on educational tests, which necessitates the reporting of imputed scores to some examinees. While there exist several approaches for reporting imputed scores, there is a lack of any guidance on the reporting of the uncertainty of imputed scores. In this paper, several approaches are suggested for quantifying the uncertainty of imputed scores using measures that are similar in spirit to estimates of reliability and standard error of measurement. A simulation study is performed to examine the properties of the approaches. The approaches are then applied to data from a state test on which some examinees' scores had to be imputed following computer problems. Several recommendations are made for practice.

Takers of educational tests often receive proficiency levels instead of or in addition to scaled scores. For example, proficiency levels are reported for the Advanced Placement (AP®) and U.S. Medical Licensing examinations. Technical difficulties and other unforeseen events occasionally lead to missing item scores and hence to incomplete data on these tests. The reporting of proficiency levels to the examinees with incomplete data requires estimation of the performance of the examinees on the missing part and essentially involves imputation of missing data. In this article, six approaches from the literature on missing data analysis are brought to bear on the problem of reporting of proficiency levels to the examinees with incomplete data. Data from several large-scale educational tests are used to compare the performances of the six approaches to the approach that is operationally used for reporting proficiency levels for these tests. A multiple imputation approach based on chained equations is shown to lead to the most accurate reporting of proficiency levels for data that were missing at random or completely at random, while the model-based approach of Holman and Glas performed the best for data that are missing not at random. Several recommendations are made on the reporting of proficiency levels to the examinees with incomplete data.

Assessment is one of the five functions of the educational psychologist's (EP's) role, yet there is a dearth of research exploring its distinctive contribution to school-based practice, and a lack of definition about what it is. In this study, the assessment practices of EPs were compared with those of other educational professionals who had achieved certification for competence in educational testing. Data were analysed using descriptive and inferential statistics, with patterns emerging which indicated that, while there is some overlap, assessment practices of EPs and other educational professionals often have different foci. Specifically, evidence suggests that EPs offer a broader, more holistic perspective with strong emphasis on the social and emotional wellbeing of the child. Additionally, other practitioners typically have a more school-orientated focus, for example, testing for exam access arrangements, reflecting their professional role. Implications are discussed, particularly in relation to the distinctive contribution to EP assessment.

Equating error is usually small relative to the magnitude of measurement error, but it could be one of the major sources of error contributing to mean scores of large groups in educational measurement, such as the year-to-year state mean score fluctuations. Though testing programs may routinely calculate the standard error of equating (SEE), the commonly used summary statistics of SEE are not direct quantifications of the impact of equating error on group means. This article proposed summary statistics that directly quantify the impact of equating error on group means or group mean differences and provided empirical and analytical methods to estimate these statistics. Examples based on empirical data were used to illustrate practical applications of these statistics.

Despite agreement about the central importance of validity for educational and psychological testing, consensus regarding the definition of validity remains elusive. Differences in the definition of validity are examined and reveals that a potential cause of disagreement stems from differences in word use and meanings given to key terms commonly employed when discussing validity. A proposal for the meaning and use of specific terms is offered and a framework that divides issues associated with validity is presented. Specifically, the framework divides what Messick termed "an integrated evaluative judgment" into three components: Instrument Validity, Verification of Interpretation and Decision, and Utility of Actions.

There are concerns that soft skills assessment has been conceptualized within the Western context and may not reflect the indigenous African worldview. Without relevant soft skills assessment contextualized in the African cultural cosmology, there is a limitation in assessing African conceptions of abilities. The purpose of this study was to identify relevant soft skills for secondary/high school students and develop a scale relevant for assessing soft skills in Botswana. An exploratory sequential mixed methods design was used to explore the perceptions of 23 education stakeholders on relevant soft skills for secondary students through in-depth interviews. The qualitative findings were used to develop a 63-item Soft Skills Assessment Scale which was administered to a sample of 306 senior secondary school students selected from three educational regions in Botswana. Exploratory factor analysis was conducted to assess the latent factor structure of the scale. Through principal component analysis, four factors were extracted with underlying 38 items. However, a confirmatory factor analysis confirmed a four-factor model (Perseverance, Civic virtue, Teamwork, and Communication) based on a final 14-item scale with Cronbach's alphas above.60 and Cronbach's alpha of.82 for the entire scale. Convergent and discriminant validities of the scale were within an acceptable range. The key contribution of this study was the development of a psychometrically valid and reliable Soft Skills Assessment Scale (SSAS) in the context of Botswana. Plain Language Summary: Developing a new tool to measure soft skills in Africa There are concerns that soft skills assessment has been conceptualized within the Western context and may not reflect the indigenous African worldview. To address this gap, we set out to develop a Soft Skills Assessment Scale (SSAS) among senior secondary school students in Botswana. In this study, we use mixed-methods approach in which we collected qualitative data through interviews and did literature review to develop the SSAS while we conducted a survey to determine the degree to which SSAS measures what it claims to measure (i.e., soft skills). The findings showed that SSAS is a reliable and valid tool to use in the secondary school setting in Botswana. [ABSTRACT FROM AUTHOR]

First Nations and Eurocentric approaches in the field of psychology and mental health are rooted in different worldviews which creates incongruencies as to what may be deemed as effective standards of practice. This autoethnography describes the experience of a school psychology intern navigating through practice particularly as it pertains to the principle of dual relationships. With a first-person perspective the journey of exploration into this matter brings together the research found in academia and the guidelines formally set as well as the beliefs and values of local community members which were brought forth through a series of interviews and conversations. There are three main themes which arose that are highlighted and they are the embodiment of a spiritual orientation, substantiation of trust and integrity, and obligation to community. These themes explicate the differences in and implications for practice when working with First Nations people, particularly in remote spiritually connected communities. The practice of psychology in and with First Nation people needs to be rooted in their beliefs, values, and customs to be effective, to reflect the experience of Indigenous ways of helping, and to decolonize approaches in the field of psychology. [ABSTRACT FROM AUTHOR]

In this study, we present a package for R that is intended as a professional tool for the management and analysis of data from educational tests and useful both in high-stakes assessment programs and survey research. Focused on psychometric models based on the sum score as the scoring rule and having sufficient statistics for their parameters, dexter fully exploits the many theoretical and practical advantages of this choice: lack of unnecessary assumptions, stable and fast estimation, and powerful and sensible diagnostic techniques. It includes an easy to use data management system tailored to the structure of test data and compatible with the current paradigm of tidy data. Companion packages currently include a graphical user interface and support for multi-stage testing.

In this study, we present a package for R that is intended as a professional tool for the management and analysis of data from educational tests and useful both in high-stakes assessment programs and survey research. Focused on psychometric models based on the sum score as the scoring rule and having sufficient statistics for their parameters, dexter fully exploits the many theoretical and practical advantages of this choice: lack of unnecessary assumptions, stable and fast estimation, and powerful and sensible diagnostic techniques. It includes an easy to use data management system tailored to the structure of test data and compatible with the current paradigm of tidy data. Companion packages currently include a graphical user interface and support for multi-stage testing. [ABSTRACT FROM AUTHOR]

The new Croatian Informatics curriculum, which introduces computational thinking concepts into learning outcomes has been put into practice. A computational thinking assessment model reflecting the learning outcomes of the Croatian curriculum was created using an evidence-centered design approach. The possibility of assessing the computational thinking concepts, abstraction, decomposition, and algorithmic thinking, in an actual classroom situation and examples of such assessment is increasingly coming to the forefront of computer science educational research. Precisely for that purpose, the research was conducted. Research data are collected through the test and questionnaire of 407 pupils (10 middle schools, age 12), analysed by exploratory factor analysis and non-parametric tests. Results showed that the presented model was suitable to assess the understanding of the concepts of abstraction and algorithmic thinking, independently of the previous experience with programming languages and pupil's gender, while assessment of decomposition needs more work and improvement, some recommendations are provided. Also, it received positive feedback from pupils and teachers what implicated that such an assessment model could help teachers in building a real-time measurement instrument.

Given a squared Euclidean norm penalty, we examine some less well-known properties of shrinkage estimates. In particular, we highlight that it is possible for some components of the shrinkage estimator to be placed further away from the prior mean than the original estimate. An analysis of this effect is provided within three different modeling settings—encompassing linear, logistic, and ordinal regression models. Additional simulations show that the outlined effect is not a mathematical artefact, but likely to occur in practice. As a byproduct, they also highlight the possibilities of sign reversals ("overshoots") for shrinkage estimates. We point out practical consequences and challenges, which might arise from the observed effects with special emphasis on psychometrics. [ABSTRACT FROM AUTHOR]

The new Croatian Informatics curriculum, which introduces computational thinking concepts into learning outcomes has been put into practice. A computational thinking assessment model reflecting the learning outcomes of the Croatian curriculum was created using an evidence-centered design approach. The possibility of assessing the computational thinking concepts, abstraction, decomposition, and algorithmic thinking, in an actual classroom situation and examples of such assessment is increasingly coming to the forefront of computer science educational research. Precisely for that purpose, the research was conducted. Research data are collected through the test and questionnaire of 407 pupils (10 middle schools, age 12), analysed by exploratory factor analysis and non-parametric tests. Results showed that the presented model was suitable to assess the understanding of the concepts of abstraction and algorithmic thinking, independently of the previous experience with programming languages and pupil's gender, while assessment of decomposition needs more work and improvement, some recommendations are provided. Also, it received positive feedback from pupils and teachers what implicated that such an assessment model could help teachers in building a real-time measurement instrument. [ABSTRACT FROM AUTHOR]

Copyright of Pixel-Bit, Revista de Medios y Educacion is the property of Pixel-Bit, Revista de Medios y Educacion and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The COVID‐19 pandemic negatively affected the quality of data from educational testing programs. These data were previously used for many important purposes ranging from placing students in instructional programs to school accountability. In this article, we draw from the research design literature to point out the limitations inherent in "disrupted" educational testing data, suggest questions and criteria to be considered in evaluating the use of such data for decision making, and indicate how such data may be valid or invalid for specific purposes. Six criteria are proposed for evaluating the degree to which educational testing data are valid for specific decisions. These criteria suggest data from COVID‐disrupted school years are not likely to be valid for accountability purposes, but may be valuable for making decisions at the individual student level. Thus, we encourage researchers and policy makers to focus on how decisions derived from such disrupted data affect children. [ABSTRACT FROM AUTHOR]

Appendix 2, The Malfunction of US Education Policy: Elite Misinformation, Disinformation, and Selfishness (2023), Rowman & Littlefield Publishers