Search

The broad autism phenotype is the phenotypic expression of the primary characteristics of autism. However, currently available tests do not agree with the two-domain operationalization of broad autism phenotype or autism, and their internal structure has shown instability across applications. This study presents the Broad Autism Phenotype--International Test, the first such test designed to target a two-domain operationalization in Spain and the United Kingdom. A broad pilot study was conducted in three community samples, two from Spain (ES1, N = 970; ES2, N = 460) and one from the United Kingdom (N = 530). The Broad Autism Phenotype--International Test consists of 20 items distributed into the SOCIAL-BAP and RIRE-BAP dimensions, representing the seven autistic subdomains. Common factor approaches were used to find a simple and invariant factor structure between countries. The Broad Autism Phenotype--International Test presents full metric invariance between two Spanish samples, partial invariance between countries, and adequate omega reliability estimation. Evidence-based results on the relationships of the SOCIAL-BAP and RIRE-BAP to other variables (the Social Responsiveness Scale-2 and the Broad Autism Phenotype Questionnaire) support the interpretation of its scores for the intended uses. The evidence provided in this article shows that the Broad Autism Phenotype--International Test functions adequately compared to other available broad autism phenotype tests, such as the Broad Autism Phenotype Questionnaire.

Researchers and practitioners need robust measurement tools for evaluating knowledge of child development to better support parents and their children during pregnancy and the transition to parenthood. We addressed this need by evaluating the psychometric properties of the Domains of Development Instrument (DoDI) for measuring knowledge of developmental milestones from birth to 3 years. We evaluated four types of validity evidence for the DoDI: test content, response processes, internal structure, and relations to other variables. We convened an expert panel to evaluate test content and conducted cognitive interviews with mothers to evaluate response processes. We also collected responses from a sample of 418 English-speaking pregnant women to evaluate internal structure and relations to other variables. We observed content validity and response process validity, as well as the predicted internal structure, internal consistency, test--retest reliability, and convergent validity. We conclude with recommendations for future research with the DoDI.

Enhancing students' assessment literacy is essential in enabling all students to manage their learning successfully. Understanding of the assessment standards required and how to meet them impacts students' learning outcomes within higher education (HE). However, there are many different conceptions of what assessment literacy comprises, making it difficult to provide guidance on the most effective approaches to enhancing student and academic understanding of this complex and multifaceted concept. With this concern in mind, we investigated the psychometric properties of an assessment tool and its suitability for use within the context of implementing and evaluating pedagogical interventions aimed at enhancing students' and academics' assessment literacy skills. The validity of the tool was confirmed using exploratory and confirmatory factor analyses. Furthermore, the identified relationship between assessment literacy and feedback elements of the tool confirms the importance of an integrated approach to assessment. Drawing on extensive piloting of the tool across disciplines we highlight the importance of pedagogical co-construction approaches that promote shared understandings of assessment literacy between students and academics. Suggestions for further enhancement of this measure are proposed with the aim of supporting academics' and students' shared understandings and development of assessment literacy.

Background: Objective Structured Clinical Examinations (OSCEs) are a simulation-based assessment tool used extensively in medical education for evaluating clinical competence. OSCEs are widely regarded as more valid, reliable, and valuable compared to traditional assessment measures, and are now emerging within professional psychology training programs. While there is a lack of findings related to the quality of OSCEs in published psychology literature, psychometric properties can be inferred by investigating implementation. Accordingly, the current review assessed implementation of OSCEs within psychology programs against a set of Quality Assurance Guidelines (QAGs). Methods: A systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) recommendations. Electronic databases including ProQuest Psychology, PsycArticles, Psychology and Behavioural Sciences Collection, PsycInfo and key indexing databases such as Scopus, ProQuest, and Web of Science were used to identify relevant articles. Twelve full-text articles met all inclusion criteria and were included in the review. Results: There was considerable heterogeneity in the quality of studies and reporting of OSCE data. Implementation of OSCEs against QAGs revealed overall adherence to be "Fair." Conclusion: The current review consolidated what is known on psychometric quality of OSCEs within psychology programs. A further need for quantitative evidence on psychometric soundness of OSCEs within psychology training is highlighted. Furthermore, it is recommended that future training programs implement and report OSCEs in accordance with standardized guidelines.

Students in higher education (HE) are required to complete a variety of writing tasks for coursework and examinations. However, for some students writing presents a major challenge. In the UK, the availability of tools for specialist assessors to help identify difficulties with the quality of written composition is limited. The aim of this study was to develop a practical new tool, the Writing Quality Scale (WQS) for assessing writing quality in HE, that was both easy to use and did not require specialist or subject knowledge. The reliability and validity of the tool and its ability to identify students in HE who may need support were evaluated by examining scripts from 120 students (60 male) aged 17 to 25 years. The WQS was found to have good inter-rater reliability and was sensitive enough to pick up age differences and differentiate between groups of students with and without dyslexia. The WQS will be a useful tool for specialist assessors in HE to help in the identification of those with poor writing quality and to understand more about the nature of their difficulties.

International English Language Testing System (IELTS) scores have been widely used as the language entry requirements for international students in many English-medium universities. However, research on the predictive validity of IELTS on subsequent academic performance has been inconclusive. Additionally, despite the proliferation of IELTS preparation courses, very few attempts have been made to explore the extent to which these courses support course takers' development of knowledge and skills needed for their subsequent university study. This paper investigated (a) the predictive validity of IELTS results on the subsequent academic performance of Vietnamese international students at UK universities, and (b) the extent to which an IELTS preparation course in Vietnam supported its course takers in their subsequent academic study in the UK. Questionnaires with 80 Vietnamese international students from 31 universities across the UK showed that there was a positive significant correlation between these students' IELTS scores and their academic results. Follow-up interviews with two undergraduates and two postgraduates who had previously studied in the same IELTS preparation course, but were studying different academic disciplines at different institutions in the UK, and analysis of the materials used in their IELTS preparation course and university courses revealed that the course positively supported these students' subsequent academic study, but certain skills and knowledge needed in their academic study were not effectively covered in the IELTS preparation course.

Research is a concrete action in academia which has uplifted societies' prosperity. Although researchers have given particular attention to student perceptions about what research is in a higher education context, little attention has been given to secondary school students' perceptions about this issue. To fill this gap, Yeoman et al. (2016) qualitatively developed an instrument measuring secondary school students' perceptions of what research is. The present study quantitatively validates this scale using the dataset originally used to qualitatively validate it. The factor structure of the 'what research is' scale and measurement invariance across gender, school type, and key stage was examined. The sample is composed of 2634 secondary school students in seven schools located in East Anglia, UK. The data from this original sample showed a relatively acceptable fit to the four-factor structure after omitting some items. The result also highlighted that whilst there was evidence on configural and metric level invariance (i.e. the factor structures and the factor loadings of the scale are equivalent across gender, school type, and key stage), scalar level invariance was not met (i.e. the item intercepts of the scale are not equivalent across gender, school type, and key stage). Recommendations for future studies and future directions for research are discussed.

Was the approach proposed for calculating exam grades in summer 2020 fair? Were the grades eventually awarded (after policy changes) fair? What is a fair arrangement for 2021? These questions have been at the heart of debate in the UK in the light of COVID-19. After schools were closed in the spring of 2020 and the decision was made not to proceed with summer exams, it was judged unfair to deny students the grades they needed to progress to the next stage in their lives. The task was to find a fair way to award grades in the absence of exams. The approaches developed in all four parts of the UK--and the Republic of Ireland--were thought by the regulators and Ministers to be the fairest possible, but in the event the grades initially awarded were widely decried as "unfair" and instead all the UK countries switched to awarding Centre Assessed Grades (CAGs). The result was not only significant grade inflation (compared to previous years) but unequal treatment of different subjects and groups of candidates. In this article, we shall briefly recall the conceptual map of "fairness" that we have offered elsewhere and outline received views of assessment fairness before 2020. We shall then discuss five challenges to those received views raised by the COVID-19 experience, particularly in the UK.

Academic motivation is important to students' mental health and performance. One established measure is the Academic Motivation Scale (AMS), comprising 28 items. AMS assesses intrinsic motivation, extrinsic motivation, and amotivation, which are further categorised into seven subscales. One weakness of AMS is its length. In this study, we constructed and validated a short version of the 14-item Academic Motivation Scale (SAMS). Data from two university student samples were analysed to construct and validate the factorial structure. SAMS yielded adequate internal consistency, and very strong correlations with the original version of AMS in both samples. Confirmatory factor analysis on SAMS replicated the seven-factor model identified in the original AMS. SAMS can be a reliable and valid alternative to the original AMS.

The Reading the Mind in the Eyes Test (RMET) is a purported theory of mind measure and one that reliably differentiates autistic and non-autistic individuals. However, concerns have been raised about the validity of the measure, with some researchers suggesting that the multiple-choice format of the RMET makes it susceptible to the undue influence of compensatory strategies and verbal ability. We compared the performance of autistic (N = 70) and non-autistic (N = 71) adults on the 10-item multiple-choice RMET to that of a free-report version of the RMET. Both the autistic and non-autistic groups performed much better on the multiple-choice than the free-report RMET, suggesting that the multiple-choice format enables the use of additional strategies. Although verbal IQ was correlated with both multiple-choice and free-report RMET performance, controlling for verbal IQ did not undermine the ability of either version to discriminate autistic and non-autistic participants. Both RMET formats also demonstrated convergent validity with a well-validated adult measure of theory of mind. The multiple-choice RMET is, however, much simpler to administer and score.

The sociolinguistic context of language use contributes to individuals' linguistic knowledge. However, how it does so has yet to be fully conceptualised or quantitatively investigated within the language sciences. To meet this goal, a psychometrically sound evidence-based measure that captures important aspects of contextually influenced linguistic experience is necessary. In this paper we describe the development and validation of the Contextual and Individual Linguistic Diversity Questionnaire (CILD-Q). Exploratory factor analysis was conducted with data from 353 participants (62.9% South African, 37.1% UK, Mage = 29.3, SD = 10.09). A three-factor solution best describes the structure of the CILD-Q: Multilingualism in Context (contextual use and societal practice of multiple languages within a community), Multilingualism in Practice (direct and indirect linguistic exchanges and conversational interaction), Linguistic Diversity Promotion (societal and governmental endorsement of linguistic variation). Item scores corresponding to these three factors showed sufficient reliability ([alpha]'s > 0.80). The CILD-Q provides a novel and holistic manner to measure sociolinguistic diversity and can be used when measuring individuals' language experience within and across populations from differing sociolinguistic contexts.

This paper makes available to the broader educational community the instruments which have been originally designed and validated within the European project "CLIL for all: Attention to diversity in bilingual education" ("ADiBE") to determine how diversity is being catered to across a broad array of CLIL contexts in European Secondary Education (Austria, Finland, Germany, Italy, Spain, and the United Kingdom). They include three sets of questionnaires, interviews, and observation protocols and are qualitative and quantitative instruments whose design has been based on the latest research and which have undergone a carefully controlled double-fold pilot process for their validation (external ratings approach and pilot phase with a representative sample of 264 subjects). The questions included in the three sets of instruments are initially characterized, together with their format and main categories. The paper then details the steps undertaken for their research-based design and the double-fold pilot process followed for their validation. The questionnaires and interview and observation protocols are then presented in a format which is directly applicable in any CLIL classroom in order to determine the accessibility of bilingual programs for all types of achievers and to identify the chief difficulties and best practices in promoting inclusion in bilingual education.

In setting a new direction for the field by highlighting the importance of measure development, this article offers an original approach to modeling financial literacy, in which theories of situated learning meet self-efficacy: an approach that we claim fits well with the aims of program evaluation. It presents results from the validation of a new set of measures, intended for use with 16- to 19-year-olds, of financial literacy self-efficacy pertaining to contexts such as the classroom or the everyday activity of personal banking. Self-efficacy implies a domain in which confidence is measured specific to that context--in this case financial literacy. The data were collected in the United Kingdom from high school and college students enrolled in an optional certificate course in personal financial management. The measures were validated on a subset (n = 171) of a larger sample and was an off-shoot project of a larger 3-year evaluation study of the financial literacy certificate course (n = 2,000), which provided additional mixed-methods data used in validation. Correlation analysis supports the contention--incorporated within the framework presented--that self-efficacy is context-specific and so measures of self-efficacy must adequately reflect the contexts in which the associated literacies reside.

This study examined the psychometric properties of the Ego Resiliency Scale-Revised (ER89-R). Though support exists for a multidimensional conceptualisation using classical test theory approaches (i.e., a higher-order model comprising Openness to Life Experiences and Optimal Regulation factors), this measure has not been subjected to Rasch analysis. Accordingly, this paper evaluated the higher-order model via confirmatory factor analysis (CFA) before assessing Openness to Life Experiences and Optimal Regulation components using Rasch analysis. CFA, using a general population sample (N = 2009), supported the higher-order factor structure. Openness to Life Experiences and Optimal Regulation scales met Rasch model assumptions. Specifically, good item/person fit and item/person reliability, and evidence of unidimensionality. Moreover, most items displayed gender invariance. Overall, findings supported the higher-order conceptualisation of the ER89-R, and indicated that the Openness to Life Experiences and Optimal Regulation scales are relatively useful measures of ego resiliency components in a general population sample.

Computational Thinking (CT), entailing both domain-general and domain-specific skills, is a competency fundamental to computing education and beyond. However, as a cross-domain competency, appropriate assessment design and method remain equivocal. Indeed, the majority of the existing assessments have a predominant focus on measuring programming proficiency and neglecting other contexts in which CT can also be manifested. To broaden the promotion and practice of CT, it is necessary to integrate diverse problem types and item formats using a competency-based assessment method to measure CT. Taking a psychometric approach, this article evaluates a novel computer-based assessment of CT competency, Computational Thinking Challenge. The assessment was administered to 119 British upper secondary school students (M = 16.11; SD = 1.19) with a range of prior programming experiences. Results from several reliability analyses, a convergent validity analysis, and a Rasch analysis, provided evidence to support the quality of the assessment. Taken together, the study demonstrated the feasibility to expand from traditional assessment methods to integrating multiple contexts, problem types, and item formats in measuring CT competency in a comprehensive manner.

Vocabulary size measures serve important functions, not only with respect to placing learners at appropriate levels on language courses but also with a view to examining the progress of learners. One of the widely reported formats suitable for these purposes is the Yes/No vocabulary test. The primary aim of this study was to introduce and provide preliminary validity evidence for a lemma-based vocabulary size test (LVST), which has been designed to measure L2 learners' receptive vocabulary size of English. The test was administered to 219 participants. Various validity measures were operationalized to examine the performance of the test. Results show high reliability indices for the test in terms of parallel forms and internal consistency. The results also indicate that a single dimension underlies the test construct, and that the test is capable of distinguishing learners of varying proficiency levels. Concerning the item difficulty, results show approximately a linear order of difficulty across frequency levels. The test also correlated significantly with a measure of general English proficiency. The results, collectively, suggest that the LVST can usefully measure the intended construct of receptive vocabulary size, and greatly extends the range of vocabulary size measurement provided by other Yes/No tests.

The project examined the predictive validity of the "TOEFL iBT"® test with a focus on the relationship between TOEFL iBT scores and students' subsequent academic success in postgraduate studies in one leading university in the United Kingdom, paying specific attention to the role of linguistic preparedness as perceived by students and tutors. We employed a mixed-methods approach to enrich traditionally quantitatively oriented studies with a qualitative perspective. For the sample of 504 students who entered the university for postgraduate studies in the years 2011-2013 on the basis of a TOEFL iBT score, we analyzed the relation between TOEFL iBT scores and final academic award by correlation and regression analyses, taking into consideration discipline, nationality, and additional language support. For the qualitative strand, students entering the university in 2013 on the basis of a TOEFL iBT score were invited to complete questionnaires and interviews, as were their English for academic purposes and academic tutors. A total 48 students and 58 tutors participated, with 25 students and 36 tutors being interviewed at 3 points over the course of the year. Our findings show that students entering the university on the basis of TOEFL iBT scores feel well prepared and generally regard the test as an effective means of preparation for their academic studies in a U.K. setting. They cope well with linguistic demands, and a vast majority graduate successfully. Our findings support the appropriateness of the university's entrance policy with regard to setting minimum test score requirements, thus underpinning the predictive validity of TOEFL iBT in a U.K. setting.

Research on learning environments at the higher-education level has been quite sparse compared with studies at other educational levels. Because statistics is perceived as a difficult subject across disciplines, it suffers from low passing rates in many universities. This study involved validating questionnaires for assessing the psychosocial environment and student attitudes associated with learning business statistics in computing laboratory workshops. The Business Statistics Computer Learning Environment Inventory (BSCLEI) and Attitude to Business Analytics instrument were validated with 275 students enrolled across various business degree programs in the United Kingdom over two academic years. Various data analyses (including exploratory and confirmatory factor analyses) supported the validity of these two questionnaires, thereby paving the way for their future use in research and practical applications relevant to learning environments in higher-education statistics workshop classrooms.

Objective Structured Clinical Examinations (OSCEs) have become ubiquitous as a form of assessment in medical education but involve substantial resource demands and considerable local variation. A detailed understanding of the processes by which OSCEs are designed and administered could improve feasibility and sustainability. This exploration of OSCE design is informed by Practice Theory, which suggests assessment design processes are dynamic, social and situated activities. The overall purpose is to provide insights that inform on-the-ground OSCE administration. Fifteen interviews were conducted with OSCE academics and administrators from three medical schools in Australia, the United Kingdom and Canada. Drawing from post-qualitative inquiry, Schatzki's Practice Theory was used both as a sensibility and as an analytic framework. OSCE design was characterised by planning activities, administration activities, negotiation activities and bureaucratic activities; it involves significant and resource-intensive effort in negotiation and coordination. There was considerable local variation but at the same time activities were remarkably consonant across national boundaries. There was a tension between general understandings such as reliability and validity that underpin the OSCE and the improvisational practices associated with design and administration. Our findings highlighted the role of blueprints as a key coordinating artefact but with too many rules and procedures prompting cycles of bureaucracy and complexity. Emphasising coordination rather than standardisation might ease workloads, support adaptation to local environments and prevent an overly reductive approach to this assessment format.

Attachment difficulties are associated with a range of adverse outcomes in mental health, and people with intellectual disabilities (IDs) may be at greater risk of experiencing difficulties in their attachment relationships. This review critically evaluated recent research measuring the prevalence of attachment difficulties in people with ID. Eight studies met the inclusion criteria, and a higher prevalence of insecure and disorganized attachment classifications, and symptoms of attachment disorder, was found across a number of subgroups of people with diagnoses of ID. However, the validity and reliability of measures of attachment have not been empirically established in this population, and control groups were not always appropriate. These findings indicate the need to (1) develop reliable and standardized assessments of attachment for people with ID and (2) evaluate the efficacy of attachment-based interventions in relation to reducing psychological distress, mental health problems and expression of behaviours experienced by others as challenging.

The Me and My School Questionnaire (M&MS) is a brief self-report measure of elementary school students' (ages 8-12) social, emotional, and behavioral challenges. As there is a need for brief self-report screening measures for students in elementary, or primary, school, this study examined the factor structure and measurement invariance of the M&MS for elementary school students in the United States (U.S.; N = 784) and the U.K. (N = 538). Results replicated its two-factor structure (emotional difficulties and behavioral difficulties) with both samples. Convergent and discriminant validity, test-retest reliability, and measurement invariance for girls and boys were examined in the U.S. sample. Partial measurement invariance was established when comparing factor structures of the U.S. and U.K. samples. Implications for mental health monitoring, and for comparative international research are discussed. [For the corresponding grantee submission, see ED603450.]

The evaluation of sign language proficiency needs to be based on measures with well-established psychometric proprieties. To date, no valid and reliable test is available to assess Polish Sign Language ("Polski Jezyk Migowy," PJM) skills in deaf children. Hence, our aim with this study was to adapt the British Sign Language Receptive Skills Test (the first standardized test to determine sign language proficiency in children) into PJM, a less researched sign language. In this paper, we present the first steps in the adaptation process and highlight linguistic and cultural similarities and differences between the British Sign Language Receptive Skills Test and the PJM adaptation. We collected data from 20 deaf children who were native signers (age range: 6 to 12) and 30 deaf children who were late learners of PJM (age range: 6 to 13). Preliminary analyses showed that the PJM Receptive Skills Test has acceptable psychometric characteristics (item analysis, validity, reliability, and sensitivity to age). Our long-term goal with this work was to include younger children (age range: 3 to 6) and to standardize the PJM Receptive Skills Tests, so that it can be used in educational settings and in scientific research.

The BioMedical Admissions Test (BMAT) has been used to select students for healthcare courses for 15 years. Recently, the candidature has included an increasing number of test takers who did not complete their schooling in the UK. In line with responsibilities to promote widening participation, a revision of the Section 2 Scientific Knowledge and Applications specification was conducted, to support the preparation of candidates from under-represented groups and international candidates. The process was guided by validity frameworks established in language testing research, with a particular focus on consequential validity and washback. Candidate surveys are reported to illustrate the focus on test preparation. Issues identified in the review demonstrate the impact of international test-takers' linguistic backgrounds and differences in teaching approaches, and possible interactions between these are discussed. It is recommended that admissions test developers make use of holistic frameworks of validity from other contexts to systematically interrogate a wide range of validity evidence relating to their tests.

Social and emotional development in early childhood has been shown to predict long-term outcomes, including school readiness and academic achievement. Early years practitioners need robust assessment tools to adequately identify early signs of social, emotional, behavioural and mental health (SEMH) difficulties. The current study aimed to develop and validate the Early Years Boxall Profile (EYBP). A total of 510 children were assessed by preschool practitioners using the EYBP and the Strengths and Difficulties Questionnaire. We found that the EYBP showed limited accuracy in screening children with SEMH difficulties. However, the EYBP had good internal consistency and excellent convergent and concurrent validity. Results suggest that the tool may be helpful to identify early signs of SEMH needs. Future research is needed to explore the structural validity of the assessment tool and to improve its diagnostic abilities.

This research developed a scale to assess the following life skills in higher education students: teamwork, goal setting, time management, emotional skills, interpersonal communication, social skills, leadership, and problem solving and decision making. Study 1 adapted an existing scale for the purposes of this research and provided evidence for the factorial validity of the new scale with 445 students. Study 2 included 423 students and supported the predictive validity of the scale in relation to students' psychological well-being, academic self-efficacy/performance, and health-related quality of life. Study 3 provided evidence for the test-retest reliability of the scale with 49 students. All three studies supported the internal consistency reliability of the scale. Combined, these studies suggest that the scale is a valid and reliable measure that researchers, policymakers and educators could use to assess and potentially enhance higher education students' life skills. Future research directions and uses of the scale are discussed.

In this article, we present details of a new Academic Boredom Survey Instrument (ABSI) incorporating different measures of academic boredom's trait, state and other characteristic attributes for the exploratory study of student engagement in Higher Education (HE). Developed from a review of international research literature and our own empirical work in the field, validation of the ABSI proceeded in detail from a sample of 408 undergraduates enrolled on 16 arts, humanities and science degree programmes at two universities and two further education colleges in the UK. In terms of the ABSI's embedded trait and state questionnaires alone, Exploratory and Confirmatory Factor Analysis resulted in the establishment of three scales, with associated subscales, for general use (boredom proneness and class- and study-related boredom). Together with other characteristic attributes (e.g. sites and triggers, boredom frequency, feelings, coping strategies and revision and assignment boredom), additional data obtained from a modified version of the Shortened Experiences of Teaching and Learning Questionnaire (SETLQ) and course grades from student records, correlation and cluster analysis contributed further in terms of determining the robustness and value of the ABSI as an exploratory tool, as well as highlighting the predictive and diagnostic potential afforded when using complementary research instruments in combination. Offering availability for interdisciplinary use and critical comment across the UK HE sector as a whole, the ABSI has particular relevance in terms of designing and delivering courses, the professional development of staff, student profiling and the provision of student support.

There is an appropriate increasing focus on the need to ensure the voices of people with intellectual disability are captured as part of assessing individuals' quality of life; however, there remains a lack of a consensus on ways to achieve this. This article describes the development of a self-report measure of quality of life for people with intellectual disability, the 'Mini-MANS-LD', based on the concepts of Maslow's hierarchy of needs. Following use with 33 individuals with intellectual disability, the Mini-MANS-LD was found to have acceptable psychometric properties, including moderate congruent validity and acceptable internal consistency. Administrators' feedback suggested good acceptability and feasibility, and the measure was relatively quick to administer, easy to use and acceptable to service users. Despite a small sample size, this initial study suggests that the Mini-MANS-LD may present a conceptually relevant, feasible and acceptable self-report measure of quality of life for people with intellectual disability.

To assist higher education institutions in meeting the challenge of limited student engagement in the implementation of Learning Analytics services, the Questionnaire for Student Expectations of Learning Analytics (SELAQ) was developed. This instrument contains 12 items, which are explained by a purported two-factor structure of "Ethical and Privacy Expectations" and "Service Feature Expectations." As it stands, however, the SELAQ has only been validated with students from UK university, which is problematic on account of the interest in Learning Analytics extending beyond this context. Thus, the aim of the current work was to assess whether the translated SELAQ can be validated in three contexts (an Estonian, a Spanish, and a Dutch University). The findings show that the model provided acceptable fits in both the Spanish and Dutch samples, but was not supported in the Estonian student sample. In addition, an assessment of local fit is undertaken for each sample, which provides important points that need to be considered in future work. Finally, a general comparison of expectations across contexts is undertaken, which are discussed in relation to the General Data Protection Regulation (2018).

The measurement of perfectionistic cognitions has recently caused disagreement among researchers. Flett, Hewitt, Blankstein, and Gray proposed that perfectionistic cognitions are unidimensional. However, after re-examining the factor structure of the instrument used to measure perfectionistic automatic thoughts (Perfectionism Cognitions Inventory [PCI]), Stoeber, Kobori, and Tanno argued that perfectionistic cognitions are multidimensional. Researchers are now faced with a dilemma: Should they adopt a multidimensional approach derived from the analysis of the underpinning structure of the instrument or should theory take precedence and the instrument be revised? In considering these two alternatives, in this instance, we advocate the latter strategy. In accord, in the current study, we assess the factor structure of the PCI with the intention of creating a unidimensional version of the instrument. In doing so, we provide evidence to support the use of a new shorter version of the PCI. Unlike the original PCI, the Perfectionism Cognitions Inventory--10 (PCI-10) has a unidimensional structure that replicates across independent samples. The PCI-10 and the original PCI are also highly correlated. Based on this evidence, we propose that the PCI-10 provides a short, psychometrically sound, instrument to measure perfectionistic cognitions in the unidimensional manner it was intended.

In large-scale educational assessments, it is generally required that tests are composed of items that function invariantly across the groups to be compared. Despite efforts to ensure invariance in the item construction phase, for a range of reasons (including the security of items) it is often necessary to account for differential item functioning (DIF) of items "post hoc." This typically requires a choice among retaining an item as it is despite its DIF, deleting the item, or resolving (splitting) an item by creating a distinct item for each group. These options involve a trade-off between model fit and the invariance of item parameters, and each option could be valid depending on whether or not the source of DIF is relevant or irrelevant to the variable being assessed. We argue that making a choice requires a careful analysis of statistical DIF and its substantive source. We illustrate our argument by analyzing PISA 2006 science data of three countries (UK, France and Jordan) using the Rasch model, which was the model used for the analyses of all PISA 2006 data. We identify items with real DIF across countries and examine the implications for model fit, invariance, and the validity of cross-country comparisons when these items are either eliminated, resolved or retained.

The purpose of this Delphi-study was to establish the face and content validity of 10 movement skills, each with four evaluation criteria, to create the Children Focused Injury Risk Screening Tool (ChildFIRST) for 8-12-year-old children. We asked an international expert panel (n = 22) to validate a series of movement skills and evaluation criteria. This Delphi-process consisted of three rounds. In the first two rounds, the experts scored the movement skills and evaluation criteria using 5-point Likert scales. Consensus on validating an item was achieved when 75% or more of the experts scored "Agree" or "Strongly Agree." In the third-round, the experts ranked and established the final list with the validated movement skills and evaluation criteria. This study provided preliminary validity evidence for 10 movement skills, each with four evaluation criteria, to create the ChildFIRST. The ChildFIRST is designed to be used to evaluate movement competence and risk of musculoskeletal injury.

The Boxall Profile is one of the most frequently used tools to measure the social, emotional, behavioural and/or mental health difficulties (SEMH) of children and young people in schools across the UK. However, limited evidence exists regarding the norms and psychometric properties of the tool. The current study used data from 487 primary school-aged children to carry out a re-standardisation and explore a range of psychometric properties of the Boxall Profile. Updated norms were established for 4- to 11-year-old children using a Receiver Operating Characteristic approach and the Strengths and Difficulties Questionnaire as gold standard measure for SEMH difficulties. Psychometric analyses indicated good internal consistency, concurrent validity and convergent validity but also highlighted the need for further research into the structural validity of the tool. To our knowledge, this is the first study providing substantial evidence for the reliability and validity of the Boxall Profile since its publication in 1984.

Low-level classroom disruption (LLCD) is characterised by pupils swinging on chairs, whispering or fidgeting in class. This paper provides initial data on the development and validation of the teacher-rated Low-Level Classroom Disruption Scale (LLCD-S), with two samples of primary school pupils. Exploratory factor analysis in Study 1 (N= 120) revealed one factor accounting for 61% of the variance; supported by confirmatory factor analysis in Study 2 (N= 274), with one factor accounting for 63% of the variance. Both studies reported high Cronbach's alpha values of.82 and.93 respectively. The evidence supports LLCD being a unidimensional construct, measured by the eight item LLCD-S. Weak convergence validity was found between the LLCD-S and the Strengths and Difficulties Questionnaire's (SDQ) externalising behaviours: conduct problems and hyperactivity. This preliminary evidence indicates that LLCD-S is a valid and reliable measure of low-level classroom disruption. Further research is needed to test the utility of the LLCD-S across different levels of education, cultures and as a pupil-reported measure.

It is now well accepted that working in research teams that span universities, jurisdictions and countries can be rewarding and economically prudent. To this end, investigators collaborate in the pursuit of knowledge to address human and societal problems and translate results into local and global contexts. This implies that investigators need to develop study instruments that are fit for purpose and strategically manage issues arising from geographical, linguistic and cultural diversity. A proven method is cognitive interviewing to pre-test the study materials to ensure clarity and relevance in the study population. This paper describes the steps taken to increase the methodological reliability of study instruments through the use of cognitive interviewing and argues this technique should be a standard step in instrument development.

The ability to reason scientifically and communicate research appropriately is central to psychological literacy. Scientific research has little value unless scientists are able to convey results and their consequences clearly to others. In this study, we outline a method of assessing the development of psychological literacy in undergraduate students. Data from three cohorts undertaking assessed interviews as part of a final year research project unit are examined. This assessment evaluated students' ability to explain the purposes and findings of their research to a lay audience, to articulate the conceptual basis and methodological background to their approach, and to reflect on their own development during the research process. Analysis of marks and feedback from the assessment suggests it provides a reliable means of evaluating skills that contribute to psychological literacy in a manner that is both educationally acceptable to students and a valid indicator of their general level of course performance.

Anecdotal evidence suggests that autistic people experience an elevated risk of homelessness, but systematic empirical research on this topic is lacking. As a step towards filling this gap in knowledge, we conducted a preliminary investigation of the prevalence of "Diagnostic and Statistical Manual of Mental Disorders" (5th ed.) autism symptoms in a group of long-term homeless people. The entire caseload (N = 106) of a UK homeless outreach team was screened (excluding individuals born outside of the United Kingdom or Republic of Ireland) using an in-depth, semi-structured interview with keyworkers, based on "Diagnostic and Statistical Manual of Mental Disorders" (5th ed.) diagnostic criteria. This showed adequate inter-rater reliability, as well as evidence of criterion and construct validity. Of the sample, 13 people (12.3%, 95% confidence interval (7.0, 20.4)) screened positive, meeting "Diagnostic and Statistical Manual of Mental Disorders" (5th ed.) autism criteria by keyworker report. A further nine people (8.5%, 95% confidence interval (4.5, 15.3)) were 'marginal', having autistic traits that were not quite sufficient to meet "Diagnostic and Statistical Manual of Mental Disorders" (5th ed.) criteria. Those with elevated autistic traits, compared to those without, tended to be more socially isolated and less likely to use substances. This study has provided initial evidence that autistic traits are over-represented among homeless people and that autistic homeless people may show a distinct pattern of characteristics and needs. Further investigation is required to build upon these provisional findings.

There are currently many mathematics anxiety rating scales designed typically for adult and older children populations, yet there remains a lack of assessment tools for younger children (< 7 years of age) despite a recent focus on this age range. Following previous testing and validation, the 26-item iteration of the Children's Mathematics Anxiety Scale UK (CMAS-UK) for ages 4-7 years was further validated with 163 children (4-7 years) across two schools in the UK to test the validity and reliability of the items through subsequent exploratory and confirmatory factor analysis. The predictive validity of the scale was also tested by comparing scale scores against mathematics performance on a mathematics task to determine the relationship between scale and mathematics task scores. Exploratory factor analysis and associated parallel analysis indicated a 19-item scale solution with appropriate item loadings (> 0.45) and high internal consistency (a = 0.88). A single factor model of Online Mathematics Anxiety was related to the experience of an entire mathematics lesson, from first entering the classroom to completing a task. A significant negative correlation was observed between the CMAS-UK and mathematics performance scores, suggesting that children who score high for mathematics anxiety tend to score to perform less well on a mathematics task. Subsequent confirmatory factor analysis was conducted to test a range of module structures; the shortened 19-item CMAS-UK was found to have similar model indices as the 26-item model, resulting in the maintenance of the revised scale. To conclude, the 19-item CMAS-UK provides a reliable assessment of children's mathematics anxiety and has been shown to predict mathematics performance. This research points towards the origins of mathematics anxiety occurring when number is first encountered and supports the utility of the CMAS-UK. Subsequent research in the area should consider and appropriately define an affective component that may underlie mathematics anxiety at older ages. Mathematics anxiety relates to more complex procedures that elude the experiences of younger children and may instead be the result of number-based experiences in the early years of education.

The UK National Student Survey (NSS) is a sound and widely used instrument for assessing students' academic experiences. We aimed to translate the NSS for Portuguese students and to validate the instrument in a sample of medical undergraduates. The research team translated and adapted the NSS for Portuguese students (NSS-P). The survey was administered on an online platform to 1,256 final-year students at eight Portuguese medical schools. A total of 329 medical students (69.9% female) replied to the NSS-P, a response rate of 26.2%. Confirmatory factor analysis showed that the original six-factor structure had an adequate fit to the data. Adequate internal consistency was observed for all the subscales. Medium to large correlations were found among all the subscale scores and between the subscale scores and the students' overall satisfaction. Multiple regression showed that the scores on the Teaching, Organization and Management and Personal Development subscales significantly predicted the students' overall satisfaction. Approximately 64% of the students reported being satisfied with the quality of their courses. Significant differences among the medical schools in their NSS-P scores were found. The NSS-P is a valid and reliable measure for assessing medical students' perceptions of academic quality.

Background: The development of a set of questions is a central element of examination development, with the validity of an examination resting to a large extent on the quality of the questions that it comprises. This paper reports on the methods and findings of a project that explores how educational examination question writers engage in the practice of question setting. Although the practice of examination question writing is ubiquitous, limited research has been carried out on how writers actually draft individual examination questions when writing complete examination papers. Purpose: The main aim of this study was to understand more deeply the process of examination question writing. Sample: The study involved 12 experienced educational examination question writers. Design and methods: The first data collection phase involved observing writers whilst they wrote the first draft of a question paper (using a 'think aloud' process). Following the observation, the writers took part in an hour-long interview. This led to the construction of a model of question writing. Phase 2 of the data collection process involved validating the proposed question writing model with six additional question writers by means of telephone interviews. Analysis and findings: A model was developed that outlined the key stages of writing that all of the writers moved through in the completion of their question writing task. The question writing model situates the process of writing individual questions in relation to the broader cognitive tasks required in writing a complete paper. It also situates the writing process within the social dimensions of question writing. Conclusions: Iteration in the writing process appears to have an important function. These iterations represent a form of decision-making that is common to the examination question writing culture and appears to be logical and rule bound. Whilst the recognition of iterative behaviour reveals a highly personalised element of question writing, it also reveals a common expectation that the ideas across a subject area are highly inter-related, and that an examination paper should reflect this.

Language test developers have a responsibility to ensure that their tests are accessible to test takers of various backgrounds and characteristics and also that they have the opportunity to perform to the best of their ability. This principle is widely recognised by educational and language testing associations in guidelines for the production and delivery of ethical tests. This paper reports on the process of formulating good policy and practice within the British Council, an organisation with a clear agenda for promoting equality, diversity and inclusion in education contexts. In accordance with these goals, the British Council's Assessment Research Group has developed a framework to help integrate its multifaceted work on inclusivity: increasing levels of knowledge and expertise by documenting case studies; developing ethical guidelines; and communicating policy to internal and external stakeholders. In developing the framework, there has been an ongoing transition from a responsive model to a more proactive one, in which accessibility issues are addressed at all stages of the test development cycle, from design, production and delivery through to scoring and validation. This framework is demonstrated through the Aptis test. While much progress has been made, theoretical implications and practical challenges still remain to be addressed.

Objective: The evaluation of individual and organizational quality of life through the Guide to Quality-of-Life Indicators and its promotion in English-speaking areas. This guide was initially designed for people living with Autism Spectrum Disorder in specialist autism centers in Spanish and Italian-speaking areas where its effectiveness has been validated. Method: The Delphi technique is applied to the design of a qualitative investigation, with expert validation for the adaptation to English-speaking areas. Results: The results have facilitated the adaptation and validation of the tool, which has 68 indicators grouped into six areas, for its application in both the planning and the evaluation processes of organizations and support services in English-speaking areas. Conclusion: The adaptation of the questionnaire on QoL indicators for people with ASD is a reliable service-provision tool intended to assess integrated programs and to improve them throughout their life-cycles; this aim is approached from the objective perspective and through a subjective evaluation. The Guide will require further updating and review procedures as new needs and contexts emerge.

Research on professional assessment practice in higher education does not always take account of lecturers' perceptions and experiences, or their disciplinary context. This questionnaire study was designed to explore lecturers' views about three inter-related elements of professional assessment practice: assessment design, marking and feedback. It consisted of two questionnaires: the Assessment Design Inventory (ADI), previously published, and the Assessment, Marking and Feedback Inventory (AMFI) which was newly developed for this study. The two main purposes of the research were to: i) establish the validity of these two questionnaires as research tools, ii) explore the existence of professional practice in assessment and analyse any perceived hindrances as well as possible effects of participants' subject discipline. 356 lecturers from two UK universities, with a range of teaching experience and representing soft and hard disciplines, took part. Results indicated that the ADI and the AMFI were robust instruments. Findings showed evidence of assessment professionalism, but also some areas where it was lacking, together with a perceived need for formal training. Participants recognised that there were certain hindrances in putting their pedagogical beliefs into practice. The implications of this study are discussed in terms of advancing the sector's understanding of assessment professionalism.

Objective: This study reports on the operationalization and testing of the newly developed Relationship Quality (RQ) scale, designed to assess an individual's perception of his or her RQ in their current partnership. Methods: Data were generated through extended sampling from an original U.K.-based research project, "Enduring Love? Couple relationships in the 21st century." This mixed methods study was designed to investigate how couples experience, understand, and sustain their long-term relationships. This article utilizes the cross-sectional, community sample (N = 8,132) from this combined data set, drawn primarily from the United Kingdom, United States, and Australia. A two-part approach to scale development was employed. An initial 15-item pool was subjected to exploratory factor analysis leading into confirmatory factor analysis using structural equation modeling. Results: The final 9-item scale evidenced convergent construct validity and known-groups validity along with strong reliability. Conclusion: Implications for future research and professional practice are discussed.

Background: There is paucity in availability of valid and reliable measures of psychopathology that can be routinely applied with an intellectual disability (ID) population in clinical practice. The psychometric properties of the Clinical Outcome Routine Evaluation-Learning Disabilities 30-Item version (CORE-LD30) are examined. Method: The CORE-LD30 was administered to 271 sequential referrals to three National Health Service (NHS) ID services providing psychological support. A principal components analysis with oblique rotation was conducted with examination of convergent validity for extracted domains. Results: Three rotated factors were extracted with good levels of internal consistency reported for the overall measure (a = 0.92) and each of the domains, conceptually labelled Problems/Symptoms (a = 0.90), Risk to Self (a = 0.76) and Risk to Others (a = 0.71). Convergent validity is reported for two domains [Problems/Symptoms with the Glasgow Depression Scale for people with a learning disability (LD) and Risk to Others with the Health of the Nation Outcome Scale-LD], and support for the CORE-LD30 as a 'core' measure indicated. Conclusions: The CORE-LD30 is recommended as a useful broad ranging measure of psychopathology for use with an ID population. Domains may prove to be useful for research and clinical purposes. Further research is recommended to examine the ability to monitor clinical change associated with specific levels of presentation and different clinical presentations/cohorts.

Many countries use national-level surveys to capture student opinions about their university experiences. It is necessary to interpret survey results in an appropriate context to inform decision-making at many levels. To provide context to national survey outcomes, we describe patterns in the ratings of science and engineering subjects from the UK's National Student Survey (NSS). New, robust statistical models describe relationships between the Overall Satisfaction' rating and the preceding 21 core survey questions. Subjects exhibited consistent differences and ratings of "Teaching", "Organisation" and "Support" were thematic predictors of "Overall Satisfaction" and the best single predictor was "The course was well designed and running smoothly". General levels of satisfaction with feedback were low, but questions about feedback were ultimately the weakest predictors of "Overall Satisfaction". The UK's universities affiliated groupings revealed that more traditional "1994" and "Russell" groups over-performed in a model using the core 21 survey questions to predict "Overall Satisfaction", in contrast to the under-performing newer universities in the Million+ and Alliance groups. Findings contribute to the debate about "level playing fields" for the interpretation of survey outcomes worldwide in terms of differences between subjects, institutional types and the questionnaire items.