95,615 results on '"Reliability"'
Search Results
2. Perceived Social Development Scale for Physical Education Lessons (PSDSPEL) -- Parent Version
- Author
-
Yasin Karaca and Bijen Filiz
- Abstract
In this study, a valid and reliable scale was developed to measure parents' perceptions of their children's social development in physical education classes. In the first part of the research, the relevant literature reviewed and student-parent interviews were held. As a result of the data obtained from the interviews, a 30-item draft scale was created. The draft scale was applied to 30 parents of students, and as a result of the feedback received, 5 items were removed from the pool. The remaining draft scale items were submitted to expert opinion and the number of items was reduced to 22. The 22-item scale was applied to 240 volunteer parents for Exploratory Factor Analysis (EFA) and to 216 different volunteer parents for Confirmatory Factor Analysis (CFA). As a result of the analyses, the final scale consisting of 15 items and four sub-dimensions was obtained. The Cronbach's Alpha values of the sub-dimensions were found to be between 0.74-0.79. In the final part, the 15-item scale was applied to 540 volunteer parents. Accordingly, while it was observed that parents' PSDSPEL scores were higher in the sub-dimension of showing positive behavior, it was determined that male parents' perception levels were higher than female parents. As a result of the study, it was found that the validity, reliability and internal consistency coefficients of the scale were at acceptable levels.
- Published
- 2024
3. Alternative Grit Models: Explorations into the Psychometric Properties of Grit-S and Academic Performance
- Author
-
Miguel Eduardo Uribe-Moreno, Iván Felipe Medina-Arboleda, Alfredo Guzmán-Rincón, and Suelen Emilia Castiblanco-Moreno
- Abstract
Grit, the passion for achieving long-term goals, has been conceived as a two-dimensional construct (Consistency of interest and Perseverance of effort). The construct is well known for its easy measurement and its relationship with performance, including academic performance. However, there have been different criticisms, such as the overlap of grit with other personality characteristics, the variability in the psychometric structure of the Grit-O and Grit-S tests, and the fact that some work reports a weak influence of grit on academic performance. Within this framework, this study contrasts different psychometric structures of the Grit-S scale and its relationship with the academic performance of higher education students. So, with this purpose, a dichotomous model of high and low grit, a K-medias clustering model, and three structural equation models have been tested. The results indicate that (a) there is a statistically significant relationship between grit--mainly determined by the consistency dimension--and academic performance, although it decreases when controlling for contextual variables, and (b) Consistency mediates the relationship between Perseverance and academic performance. The instability of grit to predict performance is discussed, and the thesis of a two-sub-dimensional structure is supported.
- Published
- 2024
4. Generative AI and Educators: Partnering in Using Open Digital Content for Transforming Education
- Author
-
Geesje van den Berg
- Abstract
Recent advancements in generative artificial intelligence (GAI) have sparked debates and research on its transformative potential in education. This study explored how a specific group of educators partner with GAI tools, particularly ChatGPT, to complement and enhance their teaching. Within an interpretative paradigm, the study used an exploratory case study design to investigate how 13 educators enrolled as students in a structured master's programme at an open distance learning university in South Africa use GAI tools in their teaching. Their posts on a discussion forum were used to collect data. Findings reveal that they actively employ GAI to streamline their teaching practices, specifically to personalized tutors, simplifying English texts and translations, assessments, lesson planning, and critical thinking tasks. The study also highlights the need to understand the limitations and boundaries of GAI, including concerns about accuracy, biases and reliability. Importantly, the absence of institutional guidelines on GAI raises questions about the necessity for clear guidelines and policies to ensure responsible and ethical integration in educational contexts. The paper concludes by emphasizing the transformative potential of GAI and the fact that it will increasingly act as a partner in teaching and influence modern teaching practices. Ultimately, GAI's foothold in education is undeniable, challenging educators and policymakers to navigate its implications while maximizing its potential to support effective teaching.
- Published
- 2024
5. Measuring Social and Emotional Learning Skills of Preschool Children in Croatia: Initial Validation of the SSIS SEL Brief Scales
- Author
-
Sanja Tatalovic Vorkapic, Christopher J. Anthony, Stephen N. Elliott, Ilaria Grazzani, and Valeria Cavioni
- Abstract
Although there is increased interest in social and emotional competence and mental health in Croatia, there are currently limited measurement options available for early childhood settings. Thus, the SSIS SEL Brief Scales (SSIS SELb), an efficient measure of social and emotional learning competencies developed in the United States, was translated to Croatian and used by 49 early childhood educators to rate a sample of 685 children (average age 4.3 years) from several counties in Croatia. Regarding measurement invariance, the final model cohered substantially with a CASEL-inspired framework. Overall reliability was also high especially for the SEL Composite ([alpha] = 0.94), with notably lower reliability for subscales. Regarding cross-group concurrent validity, concurrent coefficients were largely similar across age and gender, but there were regional differences in validity. Likewise, validity correlations were in line with expectations, with moderate relationships observed between the SSIS SEL Composite and Child and Youth Resilience Measure scores. In sum, the high level of reliability provided a foundation for applied and research usage of the Croatian SSIS SELb, although further validation research will continue to be necessary before widescale implementation. Limitations to the study are discussed and also point to needed additional research before utilizing the Croatian translated SSIS SELb for applied decisions with young children.
- Published
- 2024
6. Bridging the SEL CASEL Framework with European Educational Policies and Assessment Approaches
- Author
-
Valeria Cavioni, Luisa Broli, and Ilaria Grazzani
- Abstract
The importance of enhancing social and emotional skills in educational settings has gained prominence, with many countries and organizations embracing the Social and Emotional Learning (SEL) framework to equip individuals with the tools needed for shaping a self-identity, emotional regulation, goal achievement, empathy, nurturing relationships, and responsible decision-making and overall well-being. In this paper, we aim to connect the globally acknowledged Collaborative for Academic, Social, and Emotional Learning SEL framework with international policies that underscore the importance of social and emotional skills in the school context. To accomplish this goal, we first provide a brief overview of the key components of the SEL framework. Subsequently, we explore two significant educational policies within the European context. The first policy is the World Health Organization Health Promoting Schools initiative. We present its objectives, a WHO-affiliated program example, the promoted and assessed competencies of students, and its results, connecting its framework with the CASEL SEL approach. The second focus is the Organisation for Economic Co-operation and Development Study on Social and Emotional Skills, describing its developmental process and the assessment framework. Finally, we describe the alignment of SEL with these European educational policies and illustrate their role in advancing and improving the evaluation of SEL initiatives in educational environments.
- Published
- 2024
7. Towards a Consensus: Harmonizing Definitions and Consistency in Terminology Use of Conceptions of Teaching in Higher Education. A Systematic Literature Review
- Author
-
Velibor Mladenovici, Mariana Crasovan, and Marian D. Ilie
- Abstract
Teaching conceptions in higher education, or so-called academics' conceptions of teaching (ACTs), are essential in informing teaching behaviors and influencing students' learning. Consequently, several attempts have been made since the 1990s to understand what ACTs represent and how they can be developed towards student-centered teaching. However, the expected results did not occur as planned because ACTs were frequently misinterpreted mainly because of the similarities with teaching beliefs and other comparable concepts like cognitions about teaching or perspectives of teaching. Hence, many fundamental issues still need to be solved (e.g., ACTs definition, terminology use, measurement, etc.). The present systematic literature review aimed to clarify the conceptual discrepancies in the ACTs' definitions and terminology and propose a consensus regarding the most appropriate working definition and terminological use. We analyzed 1123 studies using systematic online searching in the Web of Science Database and citation searching. After the eligibility process, we came across 78 eligible articles. The results showed that most of the studies used the terminology and definitions of "conceptions of teaching" (Pratt, 1992) to the detriment of the "beliefs of teaching" or another related term. Even though the concepts "conceptions of teaching" and "beliefs of teaching" come from different theoretical perspectives--where "conceptions" originate from a phenomenological approach, characterized by qualitative methodologies, and "beliefs" come from a cognitive approach, represented by quantitative studies--most studies used them interchangeably. However, the authors only extremely rarely (N = 2) appeared aware of this interchangeability and explicitly mentioned it. While the "conceptions of teaching" emerged as the most utilized term, Pratt's (1992) definition was the most often employed definition of ACTs. We advocate for consistency in teaching conceptions, research definitions, and terminological use, paramount for diminishing the risk of misinterpretation, comparing, and synthesizing findings, as well as straightforward communication among educationalists, researchers, and policymakers.
- Published
- 2024
8. Revision and Multiple Validity Verification of Self-Presentation Skills Assessment for Middle School Students
- Author
-
Weiwei Tong, Prasong Saihong, and Kanyarat Sonsupap
- Abstract
The main objective of this study is to revise and validate the assessment of self-presentation skills of middle school students. The assessment is based on existing self-assessment scales and adaptively modified for a more accurate assessment of middle school students' self-presentation skills. Considering the characteristics of middle school students and teachers' assessment, we have reconstructed and adjusted the dimensions of the original assessment. This paper includes background research, scale design and innovation, scale testing, and scale reliability and multiple validity verification. In order to further verify the effectiveness and reliability of the tool, this study added the application of content validity and factor analysis on the basis of previous studies. The results of this study are intended to compensate for the limitations of current teacher assessment for middle school students' self-presentation. At the same time, this study also provides a basis for the development of middle school students' self-presentation skills integrated training programs.
- Published
- 2024
9. Operationalizing a Weighted Performance Scoring Model for Sustainable e-Learning in Medical Education: Insights from Expert Judgement
- Author
-
Deborah Oluwadele, Yashik Singh, and Timothy Adeliyi
- Abstract
Validation is needed for any newly developed model or framework because it requires several real-life applications. The investment made into e-learning in medical education is daunting, as is the expectation for a positive return on investment. The medical education domain requires data-wise implementation of e-learning as the debate continues about the fitness of e-learning in medical education. The domain seldom employs frameworks or models to evaluate students' performance in e-learning contexts. However, when utilized, the Kirkpatrick evaluation model is a common choice. This model has faced significant criticism for its failure to incorporate constructs that assess technology and its influence on learning. This paper aims to assess the efficiency of a model developed to determine the effectiveness of e-learning in medical education, specifically targeting student performance. The model was validated through Delphi-based Expert Judgement Techniques (EJT), and Cronbach's alpha was used to determine the reliability of the proposed model. Simple Correspondence Analysis (SCA) was used to measure if stability is reached among experts. Fourteen experts, professors, senior lecturers, and researchers with an average of 12 years of experience in designing and evaluating students' performance in e-learning in medical education participated in the evaluation of the model based on two rounds of questionnaires developed to operationalize the constructs of the model. During the first round, the model had 64% agreement from all experts; however, 100% agreement was achieved after the second round, with all statements achieving an average of 52% strong agreement and 48% agreement from all 14 experts; the evaluation dimension had the most substantial agreements, next to the design dimension. The results suggest that the model is valid and may be applied as Key Performance Metrics when designing and evaluating e-learning courses in medical education.
- Published
- 2024
10. Devising an Early Childhood Engineering Habits of Mind Scale
- Author
-
Ahmet Erol and Mustafa Erol
- Abstract
Engineering education aims to equip children with the skills to solve and apply complex problems. Problem-solving processes in engineering require high-level thinking and mind habits. Habit is a term used to describe various aspects of intelligence. Engineering habits of mind are the values, attitudes, and thinking skills associated with engineering. This research aimed to develop a scale to assess the engineering habits of mind of children ages 5-8. The study involved 417 children in two provinces in the southwest and northwest of Türkiye. We performed Explanatory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) for the scale's construct validity. The scale consisted of 35 items and six factors and explained 59.2% of the total variance. We called the factors "system thinking," "creativity," "optimism," "collaboration," "communication," and "attention to ethical considerations." According to the CFA result, the construct we obtain is reasonable (X[superscript 2]/sd= 1.97, RMSEA= 0.068, CFI= 0.93, TLI= 0.91, SRMR= 0.062). The item total correlations range from 0.43 to 0.66. We determined the Cronbach Alpha coefficient of the scale to be 0.94 and the test re-test reliability to be 0.87. According to the results, it can be said that the scale obtained can be used validly and reliably to determine the engineering habits of mind children according to teacher reports. The study contributes to the ever-increasing engineering habits of mind literature.
- Published
- 2024
11. Exploring the Impact of the Integrated Think-Pair Share and Active Learning Management on Non-Credentialed Teacher Learning Assessment Competency
- Author
-
Nattapon Yotha, Wasinee Rungruang, and Wuthikrai Pommarang
- Abstract
This study investigates the effectiveness of an integrated approach combining think-pair share and active learning management in enhancing non-credentialed teachers' assessment competency and compares their learning achievement in educational assessment across pre, post, and delayed examination phases. Utilizing a one-group experimental design, 29 participants were selected through cluster sampling. Instruments included an integrated think-pair-share and active learning management system, along with assessments evaluating participants' assessment knowledge, skills, and attributes. Data analysis involved mean scores, standard deviation, effectiveness index (E[subscript 1]/E[subscript 2]), One - way repeated measure ANOVA, and pairwise comparisons. Results indicate that the integrated learning management plan effectively facilitated teaching assessment competency to non-credentialed teachers at the graduate level, leading to the attainment of expected learning levels within the class. Additionally, it significantly enhanced participants' competency compared to their baseline levels, with sustained effectiveness demonstrated over time.
- Published
- 2024
12. Exploring the Feasibility and Efficacy of ChatGPT3 for Personalized Feedback in Teaching
- Author
-
Irum Naz and Rodney Robertson
- Abstract
This study explores the feasibility of using AI technology, specifically ChatGPT-3, to provide reliable, meaningful, and personalized feedback. Specifically, the study explores the benefits and limitations of using AI-based feedback in language learning; the pedagogical frameworks that underpin the effective use of AI-based feedback; the reliability of ChatGPT-3's feedback; and the potential implications of AI integration in language instruction. A review of existing literature identifies key themes and findings related to AI-based teaching practices. The study found that social cognitive theory (SCT) supports the potential use of AI chatbots in the learning process as AI can provide students with instant guidance and support that fosters personalized, independent learning experiences. Similarly, Krashen's second language acquisition theory (SLA) was found to support the hypothesis that AI use can enhance student learning by creating meaningful interaction in the target language wherein learners engage in genuine communication rather than focusing solely on linguistic form. To determine the reliability of AI-generated feedback, an analysis was performed on student writing. First, two rubrics were created by ChatGPT-3; AI then graded the papers, and the results were compared with human graded results using the same rubrics. The study concludes that e-Learning arning certainly has great potential; besides providing timely, personalized learning support, AI feedback can increase student motivation and foster learning independence. Not surprisingly, though, several caveats exist. It was found that ChatGPT-3 is prone to error and hallucination in providing student feedback, especially when presented with longer texts. To avoid this, rubrics must be carefully constructed, and teacher oversight is still very much required. This study will help educators transition to the new era of AI-assisted e-Learning by helping them make informed decisions about how to provide useful AI feedback that is underpinned by sound pedagogical principles.
- Published
- 2024
13. Leveraging AI for Enhanced Support: Satisfaction Levels of Users Utilizing Virtual Assistant in Open Education
- Author
-
Sefa Emre Öncü and I?rfan Süral
- Abstract
This study explores users' perceptions of the Virtual Assistant application facilitated by the Anadolu University Open Education System. It assesses the application's effectiveness and user satisfaction by analyzing opinions regarding its importance and the experience it delivers. The research delineates its population as learners engaging with the Virtual Assistant within the Anadolu University Open Education Framework. Employing a sequential explanatory design within mixed-method research methodologies, the investigation commenced with an online Virtual Assistant Satisfaction Survey involving 374 participants as the sample-subsequent focus group discussions with 18 users aimed to delve deeper into satisfaction levels vis-à-vis user expectations. Findings indicate a moderate level of satisfaction among users with the Virtual Assistant application, reflecting a congruence with anticipated outcomes. Moreover, the analysis across various dimensions revealed a comprehensive satisfaction in alignment with user expectations, particularly highlighting approval for the application's adherence to the Open Education website's design language. Conversely, dissatisfaction was noted concerning the specificity and utility of the information provided. The study identifies a disparity in satisfaction levels, with high marks in design aspects and lower evaluations in usability. Recommendations are extended towards enhancing the Virtual Assistant and similar tools through user-centric development and encouraging further research predicated on the insights garnered from this study.
- Published
- 2024
14. Validity and Reliability of Concept Inventory Test in Human Physiology
- Author
-
Gio Jay B. Aligway, Jo C. Delos Angeles, Angeli V. Collano, Eljoy P. Barroca, Anna Clarissa D. Aves, Juneflor F. Catubay, Jennifer T. Edjec, Ma. Diana A. Butaya, and Sylvester T. Cortes
- Abstract
Biology education plays a vital role in nurturing the understanding of learners about the intricacy of life. Various efforts have emerged to strengthen learning biological concepts but there were still studies that showed that learners have low mastery in some aspects. To determine how well students understood various biological topics, including human physiology, Concept inventory tests (CIT) were used. The concept inventory test may be able to spot students' misconceptions and ultimately lead to improved comprehension. The crafted CIT developed with the aid of a table of specifications based on Bloom's taxonomy of cognitive domain was assessed according to its validity and reliability. In validation, content validity and item analysis were considered while reliability test was employed through Cronbach's alpha. Distractor analysis was also performed to determine possible source of misconception per item. The CIT was administered to 120 senior high school STEM students (50.8% from the private schools, 37.5% in regular public schools and 11.7% from public schools with special programs in science). The results displayed high content validity with a mean of 4.83 for content validity and an average Aiken's validity coefficient of 0.98. It also highlighted that the test is moderately difficult with the test difficulty of 0.58, as well as, discriminatory with a discriminating level of 0.46. After item classification, 63 items were retained (39 accepted, 24 for revisions) and Chronbach's alpha ([alpha]=0.74) indicated good internal consistency. The concept inventory test propounds to be a good classroom test science with minor items to be revised.
- Published
- 2024
15. Confirmatory Factor Analysis Affecting Aptitude in the Specialized Nursing Competency of Nursing Students
- Author
-
Varude Boonprayong, Pinanta Chatwattana, and Pallop Piriyasurawong
- Abstract
The objective of the research is to explore the consistency of specialized nursing competency models of nursing students. The participants in this research are 165 Year 4 nursing students at Boromarajonani College of Nursing, Chakriraj, Thailand. The data was collected using a questionnaire consisting of 45 questions, answered with the use of a 5-level scale. The questionnaire was evaluated by three experts, and the questionnaire's accuracy ratings ranged from 0.70 to 1.00. The internal reliability of the questionnaire was measured using Cronbach's alpha coefficient, resulting in values of 0.763 for educational excellence, 0.757 for socio-economic factors, 0.806 for lifestyle, and 0.814 for personal factors. The study's findings showed that competency. 1) Education throughout nursing student training. 2) The socio-economic context. 3) Habits pertinent to everyday existence. 4) Attitude towards the nursing profession. 5) GPAx represents the grade point average. 6) OSCE refers to the objective structured clinical examination. The confirmatory factor analysis of specialized nursing aptitude is consistent with the empirical data, Chi-Square = 14.226; df = 8; Relative Chi-Square = 1.778; p-value = 0.076; GFI = 0.974; NFI = 0.943; TLI = 0.951; CFI = 0.974; RMSEA = 0.069; RMR = 0.113. Each element has a standard element weight value of between 0.010 and 0.817, with a standardized weight value ranging from 0.010 to 0.817.
- Published
- 2024
16. Validating and Shortening the Achievement Emotions Questionnaire (AEQ) for Measuring Positive Emotions in Foreign Language Learning and Classroom Settings
- Author
-
Majid Sadoughi and S. Yahya Hejazi
- Abstract
Motivated by the "emotional turn" in applied linguistics and the recent introduction of positive psychology in L2 studies, the present study aimed to validate and shorten the Achievement Emotions Questionnaire (AEQ) for measuring positive emotions in foreign language learning and classroom settings. To this end, data were collected from 442 Iranian English as a Foreign Language (EFL) students from several major state universities using the adapted sub-scales of positive classroom- and learning-related emotions of the AEQ. To prepare the short version, five items with the highest factor loadings in each of the six sub-scales were retained, which resulted in a total of 30 items. A six-factor model of this short version called Foreign Language Positive Emotions Questionnaire (FL-PEQ) was corroborated, which indicated that this measure could adequately differentiate between enjoyment, hope, and pride in foreign language "learning" and "classroom." Additionally, internal validity, convergent validity, and discriminant validity of the FL-PEQ were confirmed.
- Published
- 2024
17. The Development and Initial Tests for the Psychometric Properties of the Equal Opportunity in Education Scale
- Author
-
Yavuz Ercan Gül and Rüstü Yesil
- Abstract
Although equality of opportunity in education is a current and important issue, most of the studies on this subject are based on theoretical studies rather than empirical studies. The main reason for this can be said to be the lack of a valid and reliable measurement tool in this field. For this reason, this study aims to develop a valid and reliable measurement tool for determining pre-service teachers' perceptions of equality of opportunity in education. For this purpose, data were collected from four different samples using a simple random sampling technique. First, a literature review was conducted, and qualitative interviews were conducted with the first sample on equal opportunities in qualitative education. After the item pool was created, the first application was made in order to determine the factor structure of the measurement tool. Data were collected from a separate sample to test the resulting structure and perform confirmatory factor analysis. Then, in the context of reliability studies of the measurement tool, calculation of the internal consistency coefficient, extracted mean variance, composite reliability and test-retest processes were used. As a result, a valid and reliable scale for determining the perception of equality of opportunity in education consisting of three factors and 23 items has been brought to the education literature.
- Published
- 2024
18. Examining Coping Strategies and Academic Resilience among Indonesian Students: The Role of Gender and Domicile
- Author
-
Zuraida Lubis, Muhammad B. Dalimunthe, Wildansyah Lubis, Reza Aditia, and Raudah Z. Dalimunthe
- Abstract
This study investigates the influence of coping strategies (problem-solving, emotion-focused, purpose-oriented) on academic resilience, considering gender and domicile differences. By analyzing data from 398 Indonesian students using MGA PLS-SEM, the study confirms satisfactory validity and reliability of the measurement models. The results reveal significant relationships between coping strategies and academic resilience. The study contributes to understanding the interplay between coping strategies and academic resilience across diverse groups, informing the development of targeted interventions and support systems for student well-being and success.
- Published
- 2024
19. Translating Bloom's Taxonomy Action Verb List into Arabic for Teacher Preparation Programs: Challenges/Problems and Solutions
- Author
-
Ziad ElJishi, Faten S. M. Abdel-Hame, Ahmad Khuddro, and Salman Yusuf Zay
- Abstract
This paper presents a unified Arabic version of the English action verbs mentioned in Bloom's taxonomy (1956) that has been checked for validity and reliability by a panel of education experts. Methodologically, the Bloom's list of action verbs was first translated into Arabic by an expert professor of translation. Then, using a mixed-method approach the Arabic list was validated by three bilingual education experts working at universities in Bahrain, Qatar, and the United Arab Emirates. As part of the process, the paper established a single Bloom's taxonomy list of action verbs, based on the data provided by two reputable educational institutions that were cross-checked with a well-referenced textbook. A contextualized Arabic translation of the list was then produced and was subjected to rigorous checks by a panel of reviewers who were the education experts. The paper presents the challenges faced in translating the list into Arabic and the solutions we have adopted as well as the validity and reliability checks. The implication of this paper is in offering teacher preparation programs across the Arab world an Arabic Bloom's taxonomy list that can be used by in-service and pre-service teachers. Moreover, it contributes to teacher literacy in the correct use of Bloom's action verbs in the Arabic language.
- Published
- 2024
20. Validity and Reliability of Child-Friendly School Policy Evaluation Instruments in Primary Schools: Confirmatory Factor Analysis
- Author
-
Riana Nurhayati, Suranto Aw, Siti Irene Astuti Dwiningrum, Mami Hajaroh, and Herwin Herwin
- Abstract
Evaluation of child-friendly school (CFS) policies is essential to determine the achievements of school efforts in reducing violence cases. This research aims to proving the reliability and validity of CFS policy evaluation instruments in elementary schools with different locations. This investigation uses the Context Input Process Product (CIPP) evaluation model to confirm the factor structure or dimensions of several observed variables (items) to evaluate the reliability and validity of the CFS policy evaluation tool in elementary schools. The validity and reliability of previously created instruments are evaluated quantitatively, but this study utilises different subjects and study sites. There were 320 respondents, with 145 school principals and 175 teachers taken randomly. Confirmatory factors analysis (CFA) results show goodness of fit (GOF), and the model is acceptable. The CFS evaluation instrument can be accepted after eliminating several question items and modifying them. All items of the teal instrument meet the goodness of fit criteria in terms of chi-square and root mean square error of approximation (RMSEA). The instrument for evaluating CFS policies in primary schools using the CIPP model has met a valid and reliable psychometric property test so that it can be applied.
- Published
- 2024
21. Service Quality and Student Satisfaction in Higher Education
- Author
-
Gordana Stankovska, Fatime Ziberi, and Dimitar Dimitrovski
- Abstract
Education is a significant institution given the shift to a knowledge economy. Today students have numerous expectations that range from choosing what to learn, how to learn and how much to learn based on their individual academic needs. Student satisfaction is an important facet for higher education institutions and specifically, it is highly related to service quality. Thus, the main goal of this study was to investigate the relationship between student perceived service quality and the global satisfaction level among university students. The ServQual instrument was administered to a sample of 250 university students. The results indicated that the higher gap of the service quality is related to the responsiveness dimension, followed by the reliability, assurance, empathy and tangibles dimensions. At the same time, there was a significant positive relationship between student satisfaction and responsiveness, reliability, assurance and empathy, but negative correlation between student satisfaction and tangibles. The findings generally indicate that a majority of students are satisfied with the facilities provided by the university. Such findings should help the university make a better strategic plan as to enhance student satisfaction in particular and its overall performance in general. So, the service quality in higher education has a significant influence on student satisfaction and in view of this, higher education institutions need to be aware of the service quality dimensions that influence the satisfaction of their students and therefore that it is important to note that these dimensions should be determined by the students, because the students are the primary recipients of the services provided by the institution. [For the complete Volume 22 proceedings, see ED656158.]
- Published
- 2024
22. Assessing the Psychometric Properties of the Intrinsic Motivation Inventory in Blended Learning Environments
- Author
-
Chantelle Bosch
- Abstract
This study investigates the validity and reliability of the Intrinsic Motivation Inventory (IMI) in the context of blended learning. In the digital age, the fusion of online components with traditional classroom instruction has become integral to modern pedagogy, giving rise to blended learning--a flexible approach accommodating diverse learning needs. In such multifaceted environments, intrinsic motivation emerges as pivotal for sustaining student engagement, bridging the gap between in-person and online learning components. Utilizing a quasi-experimental quantitative research design, the research focuses on first-year university students enrolled in a blended learning economics course. Thorough statistical analysis affirms the IMI's effectiveness in accurately measuring intrinsic motivation levels within the diverse framework of blended learning. The findings not only validate the IMI as a tool for assessing intrinsic motivation but also underscore its importance for educators. Understanding students' intrinsic motivation is essential for tailoring instructional strategies, identifying at-risk students, and proactively preventing potential dropouts in blended learning environments. The validation of the IMI enhances the accuracy of assessments, interventions, and program evaluations, ultimately contributing to the improvement of education quality for students engaged in the integrated format of traditional and online instruction.
- Published
- 2024
23. Self-Efficacy and Self-Regulated Learning's Role in Self-Directed Learning's Impact on Academic Writing Problem-Solving for EFL Undergraduates
- Author
-
Mujiono
- Abstract
The study examined the role of self-efficacy and self-regulated learning on the effect of self-directed learning on academic writing problem-solving for EFL undergraduates. The study applied a cross sectional design to evaluate 213 students from the Departments of English Education and English Literature of a private university in Malang, Indonesia. Participants aged 18 to 24 years were randomly selected to ensure representative samples. This study assessed undergraduate students' writing problem-solving using custom instruments inventory of problem-solving in academic writing. Selfdirected learning, self-efficacy, and self-regulated learning are evaluated on different scales. Methods such as confirmatory factor analysis and structural equation models are used to ensure the instrument's validity and reliability. The study found that selfdirected learning had a significant impact on self-efficacy and selfregulated learning and on writing problem-solving. The study also found an important relationship between self-directed learning and the ability of EFL undergraduates to solve academic writing problems through self-efficacy and self-regulated learning. This research contributes to ongoing discussions on the relevance of selfdirected learning for personal and professional development. It encourages re-evaluating educational and organizational strategies to maximize self-directed learning's benefits and minimize potential challenges.
- Published
- 2024
24. Comparing the Feedback of University Supervisor and Cooperating Teachers for Preservice Science Teachers within the Scope of Pedagogical Content Knowledge
- Author
-
Isik Saliha Karal Eyüboglu
- Abstract
To comprehend the perspectives of school and university mentors in evaluating practicum experiences based on educational reform, this study aimed to investigate the feedback, within the framework of pedagogical content knowledge, provided by two cooperating teachers and a university supervisor to preservice teachers. Observation and document were used for data collection. Data collected within the context of the Teaching Practice course were analyzed deductively, considering the components of pedagogical content knowledge. Findings showed that although feedback given by mentors was positive or corrective, specific changes suggested by mentors about the preservice teachers' teaching practices were rather corrective. Although mentors' feedback seems to be concentrated in the fields of 'instructional strategies and representation' and 'pedagogy,' the majority of feedback provided in other PCK components points out those mentors focused on student learning within the knowledge of 'student understanding'. Feedback indicated that mentors especially attached importance to students' understanding of the subject and active participation in the learning, which are the goals of a science teaching program. The other aims of teaching based on inquiry and students' self-responsibilities in learning seemed to be in the background. Although mentors' feedback pointed out some differences in the categories of curriculum and assessment, they were consistent in general.
- Published
- 2024
25. Adaptation of Flexible Thinking in Learning Scale to Turkish Culture
- Author
-
Niyazi Aktas, Suleyman Agah Demirgul, and Iskender Volkan Sancar
- Abstract
This article aims to adapt the Flexible Thinking in Learning (FTL) scale developed by Barak and Levenberg in 2016 to Turkish culture The FTL scale, whose original language is English, consists of three sub-dimensions: "Acceptance of Learning Technologies", "OpenMindedness in Learning" and "Adaptation to New Learning Situations". During the process, language and field experts conducted mutual translations between English and Turkish to assess the language equivalence of the scale. The original scale was supplied to English teachers to measure the degree of similarity between the original language (English version) and the translated language (Turkish version). After a fifteen-day interval, the same teachers were given the adapted version of the scale. The degree of similarity between the scale's original language and its Turkish translation was calculated. Once the scale's harmony was deemed satisfactory in both languages, a pilot study was conducted to receive feedback and suggestions on the comprehensibility, applicability, and general structure of the scale. After necessary improvements were made, the scale was finalized. The Flexible Thinking in Learning (FTL) scale, which was adapted to Turkish, was applied to 516 volunteer teachers. As per the findings of the analysis, the adapted scale has three sub-dimensions, just like the original scale. The measured structure of the adapted scale was found to be consistent with the original scale's factor structure.
- Published
- 2024
26. Investigating EFL Students' Perspectives of the Influence of Podcasts on Enhancing Listening Proficiency
- Author
-
Fatimah Ghazi Mohamm and Hanadi Abdulrahman Khadawardi
- Abstract
Listening is widely regarded as the predominant language proficiency utilized in virtually all forms of communication. However, its intricacies often engender feelings of complexity and, at times, provoke anxiety and frustration among both foreign and second-language learners. The enhancement of successful communication fundamentally hinges upon the precise comprehension of spoken messages. In this quantitative investigation, the present study delves into the perceptions of English as a Foreign Language (EFL) students concerning the utilization of podcasts as a tool to cultivate and bolster their listening proficiency. The study cohort comprised female university students enrolled in a preparatory year program. The examination of attitudes toward podcasts was conducted via a survey questionnaire. The findings unveiled that most participants derived enjoyment from utilizing podcasts, which in turn catalyzed their enthusiasm for English language acquisition. Additionally, they conceded that podcasts held promise in augmenting their linguistic abilities, with a primary focus on listening comprehension. These outcomes posit that podcasting serves as a medium with significant implications for students' learning trajectories, particularly regarding the acquisition of listening proficiencies.
- Published
- 2024
27. Development and Application of an Instrument for Assessing Upper-Secondary School Biology Teachers' Pedagogical Content Knowledge of Scientific Thinking
- Author
-
Shan Lin and Jian Wang
- Abstract
Scientific thinking constitutes a vital component of scientific competencies, crucial for citizens to adapt to the evolving societal landscape. To cultivate students' scientific thinking, teachers should possess an adequate professional knowledge foundation, which encompasses pedagogical content knowledge (PCK). Assessing teachers' PCK of scientific thinking facilitates the development of effective curricula tailored to their continuous professional development. Despite its significance, empirical studies on biology teachers' PCK of scientific thinking are notably lacking. Hence, this research aimed to create a reliable and valid tool to evaluate upper-secondary school biology teachers' PCK of scientific thinking. The results showed that the instrument exhibits high reliability and good validity, affirming its efficiency for investigative purposes. A collective of 292 in-service biology teachers from upper-secondary schools participated in this investigation through the completion of an online survey. The results indicated that, overall, as well as for each component, upper-secondary school biology teachers' performance on PCK of scientific thinking fell within the lower to middle range. Specifically, the performance levels of four components: knowledge of students (KSU), knowledge of instructional strategies (KIS), knowledge of curriculum (KC), and knowledge of assessment (KA) declined sequentially.
- Published
- 2024
28. Development of 'VAScoR': A Rubric to Qualify and Score Responses to the Views of Nature of Science ('VNOS') Questionnaire
- Author
-
Fouad Abd-El-Khalick, Ryan Summers, Jeanne L. Brunner, Jeremy Belarmino, and John Myers
- Abstract
We report on the development of a rubric to reliably qualify and score responses to the "Views of Nature of Science Questionnaire" (VNOS): The "VNOS Analysis and Scoring Rubric" (VAScoR). The "VAScoR" is designed to (a) provide systematic guidance for the qualitative analysis, and score assignment to nuanced categories, of "VNOS" responses, (b) explicitly scaffold qualitative inferencing and standardize score assignment to substantially lessen the burden of, and variance in, analyzing and scoring the "VNOS," and (c) improve the viability and meaningfulness of cross-study comparisons drawing on "VNOS" data. The rubric adopted the "VNOS's" consensus NOS framework and further delineated core and related elements across 10 target NOS aspects. The "VAScoR's" reliability was examined in two studies that drew on "VNOS" questionnaires completed by 185 preservice secondary science teachers (58% female; 126 undergraduate and 59 graduate students) enrolled over several years in a combined undergraduate and graduate licensure program in a large U.S. Midwestern university. In Study I, "VAScoR" analyses of 86 "VNOS" questionnaires undertaken by a single author were used to examine the rubric's intra-rater reliability, which resulted in a robust Cronbach's alpha value of 0.81. In Study II, analyses by four authors of a randomly generated, overlapping set of 18 questionnaires were used to examine inter-rater reliability, which was supported with substantial consensus among raters as indicated by a Cohen's kappa of 0.71. Further evidence for the "VAScoR's" inter-rater reliability was indicated by moderate to strong consistency among four raters with an overall Pearson's correlation coefficient of 0.82, and coefficient values ranging from 0.77 to 0.89 for six possible rater pairings.
- Published
- 2024
- Full Text
- View/download PDF
29. Measurement of Basic Psychological Needs for Physical Activity Participation for College Students with Intellectual Disabilities: A Validation Study
- Author
-
Myung Ha Sur, Deborah R. Shapiro, and Hongli Li
- Abstract
Background: Measurement instruments to understand self-determined motivation towards physical activity among college students with intellectual disabilities are needed to develop programs to support physical and psychological health and well-being. The purpose of the current study was to validate a modified questionnaire measuring basic psychological needs towards physical activity among college students with intellectual disabilities. Methods: A total of 108 college students with intellectual disabilities completed the modified questionnaire. Validity and reliability of the questionnaire was examined. Results: Confirmatory factor analysis demonstrated a six-factor model had good model fit. Cronbach's alpha values showed acceptable reliability evidence of the instrument as a whole, although some alpha values in subdomains of the instrument were below acceptable values. Conclusion: The modified questionnaire was found to have acceptable validity evidence. Further studies are needed with refinement of answer options and the addition of more questions to increase reliability.
- Published
- 2024
- Full Text
- View/download PDF
30. Coherence-Based Automatic Short Answer Scoring Using Sentence Embedding
- Author
-
Dadi Ramesh and Suresh Kumar Sanampudi
- Abstract
Automatic essay scoring (AES) is an essential educational application in natural language processing. This automated process will alleviate the burden by increasing the reliability and consistency of the assessment. With the advances in text embedding libraries and neural network models, AES systems achieved good results in terms of accuracy. However, the actual goals still need to be attained, like embedding essays into vectors with cohesion and coherence, and providing student feedback is still challenging. In this paper, we proposed coherence-based embedding of an essay into vectors using sentence-Bidirectional Encoder Representation for Transformers. We trained these vectors on Long short-term memory and bidirectional long short-term memory to capture sentence connectivity with other sentences' semantics. We used two datasets: standard ASAP Kaggle and a domain-specific dataset with almost 2500 responses from 650 students. Our model performed well on both datasets, with an average quadratic weighted kappa score of 0.76. Furthermore, we achieved good results compared to other prescribed models, and we also tested our model on adversarial responses of both datasets and observed decent outcomes.
- Published
- 2024
- Full Text
- View/download PDF
31. Does Creativity Matter in Personality Traits and Emotional Intelligence?
- Author
-
Ugur Akpur
- Abstract
Creativity has long been consistently assessed as a fundamental aspect of educational psychology and has been treated with much devotion by scholars. In this study, intricate relationship patterns between creativity, emotional intelligence, and personality traits were examined. The Marmara Creative Thinking Dispositions Scale, Trait Emotional Intelligence Questionnaire-Short Form (TEIQue-SF), and The Ten-Item Personality Inventory (TIPI) were administered to a cohort of 297 university students attending a university in Istanbul. The findings unveiled important insights into how these components interact. Employing correlational analysis to examine the relationships among the variables, it was found that while agreeableness did not exhibit a significant correlation with creativity, the remaining traits within the Big Five personality dimensions -- openness to experience (r = 0.63), emotional stability (r = 0.29), conscientiousness (r = 0.38), and extraversion (r = 0.49) -- displayed positively significant associations with creativity (p < 0.01), suggesting that individuals characterized by greater openness to experience, emotional stability, conscientiousness, and extraversion tend to exhibit higher levels of creativity in academic settings. That the link between emotional intelligence and creativity highlights a noteworthy association implies that individuals possessing higher levels of emotional intelligence are more likely to excel in creative endeavors. Regression analysis was employed to determine the specific contributions of each personality characteristic to creativity. Interestingly, when examining the predictive value of these factors, conscientiousness did not emerge as a statistically significant predictor of creativity. In contrast, both openness to experiences and extraversion exhibited strong predictive power, significantly influencing creativity. Similarly, emotional intelligence continued to be a significant predictor of creativity, reinforcing its role as a catalyst for innovative thinking. The results imply that individuals with higher levels of openness to experience, emotional stability, conscientiousness, and extraversion tend to be more creative in academic settings, highlighting the significance of emotional intelligence in promoting creative thinking and points out its potential to support creative endeavors.
- Published
- 2024
- Full Text
- View/download PDF
32. The Teacher's Role in Supporting Young Children's Level of Play Engagement
- Author
-
Singer, Elly, Nederend, Merel, Penninx, Lotte, Tajik, Mehrnaz, and Boom, Jan
- Abstract
This paper discusses the results of a study of the relationships between teacher behaviour and the level of play engagement in two- and three-year-old children in Dutch childcare centres. We found that the continuous proximity of the teacher had the greatest impact on the level of play engagement, while the teacher's walking around and only brief contacts with the children had a negative impact. In line with earlier studies, two-sided and reciprocal interactions between teacher and children also yielded positive results for play engagement. Both our quantitative and qualitative analyses showed a strong co-variation of variables. When the teacher paid only brief visits, and peers also walked in and out, there was a greater likelihood of one-sided interactions, When the teacher was always nearby, we observed the opposite. Dutch teachers spend most of their time walking around. Their pedagogy seems to be based on a model of individual care and control and insensitiveness of group dynamic processes.
- Published
- 2024
- Full Text
- View/download PDF
33. Measuring Original Thinking in Elementary School: Development and Validation of a Computational Psychometric Approach
- Author
-
Selcuk Acar, Denis Dumas, Peter Organisciak, and Kelly Berthiaume
- Abstract
Creativity is highly valued in both education and the workforce, but assessing and developing creativity can be difficult without psychometrically robust and affordable tools. The open-ended nature of creativity assessments has made them difficult to score, expensive, often imprecise, and therefore impractical for school- or district-wide use. To address this challenge, we developed and validated the Measure of Original Thinking for Elementary School (MOTES) in five phases, including the development of the item pool and test instructions, expert validation, cognitive pilots, and validation of the automated scoring and latent test structure. MOTES consists of three game-like computerized activities (uses, examples, and sentences subscales), with eight items in each for a total of 24 items. Using large language modeling techniques, MOTES is scored for originality by our open-access artificial intelligence platform with a high level of agreement with independent subjective human ratings across all three subscales at the response level (rs = .79, .91, and .85 for uses, examples, and sentences, respectively). Confirmatory factor analyses showed a good fit with three factors corresponding to each game, subsumed under a higher-order originality factor. Internal consistency reliability was strong for both the subscales (H = 0.82, 0.85, and 0.88 for uses, examples, and sentences, respectively) and the higher-order originality factor (H = 0.89). MOTES scores showed moderate positive correlations with external creative performance indicators as well as academic achievement. The implications of these findings are discussed in relation to the challenges of assessing creativity in schools and research.
- Published
- 2024
- Full Text
- View/download PDF
34. Identifying Informative Predictor Variables with Random Forests
- Author
-
Yannick Rothacher and Carolin Strobl
- Abstract
Random forests are a nonparametric machine learning method, which is currently gaining popularity in the behavioral sciences. Despite random forests' potential advantages over more conventional statistical methods, a remaining question is how reliably informative predictor variables can be identified by means of random forests. The present study aims at giving a comprehensible introduction to the topic of variable selection with random forests and providing an overview of the currently proposed selection methods. Using simulation studies, the variable selection methods are examined regarding their statistical properties, and comparisons between their performances and the performance of a conventional linear model are drawn. Advantages and disadvantages of the examined methods are discussed, and practical recommendations for the use of random forests for variable selection are given.
- Published
- 2024
- Full Text
- View/download PDF
35. Exploring an Effective Automated Grading Model with Reliability Detection for Large-Scale Online Peer Assessment
- Author
-
Zirou Lin, Hanbing Yan, and Li Zhao
- Abstract
Background: Peer assessment has played an important role in large-scale online learning, as it helps promote the effectiveness of learners' online learning. However, with the emergence of numerical grades and textual feedback generated by peers, it is necessary to detect the reliability of the large amount of peer assessment data, and then develop an effective automated grading model to analyse the data and predict learners' learning results. Objectives: The present study aimed to propose an automated grading model with reliability detection. Methods: A total of 109,327 instances of peer assessment from a large-scale teacher online learning program were tested in the experiments. The reliability detection approach included three steps: recurrent convolutional neural networks (RCNN) was used to detect grade consistency, bidirectional encoder representations from transformers (BERT) was used to detect text originality, and long short-term memory (LSTM) was used to detect grade-text consistency. Furthermore, the automated grading was designed with the BERT-RCNN model. Results and Conclusions: The effectiveness of the automated grading model with reliability detection was shown. For reliability detection, RCNN performed best in detecting grade consistency with an accuracy rate of 0.889, BERT performed best in detecting text originality with an improvement of 4.47% compared to the benchmark model, and LSTM performed best with an accuracy rate of 0.883. Moreover, the automated grading model with reliability detection achieved good performance, with an accuracy rate of 0.89. Compared to the absence of reliability detection, it increased by 12.1%. Implications: The results strongly suggest that the automated grading model with reliability detection for large-scale peer assessment is effective, with the following implications: (1) The introduction of reliability detection is necessary to help filter out low reliability data in peer assessment, thus promoting effective automated grading results. (2) This solution could assist assessors in adjusting the exclusion threshold of peer assessment reliability, providing a controllable automated grading tool to reducing manual workload with high quality. (3) This solution could shift educational institutions from labour-intensive grading procedures to a more efficient educational assessment pattern, allowing for more investment in supporting instructors and learners to improve the quality of peer feedback.
- Published
- 2024
- Full Text
- View/download PDF
36. Developing and Using a Scalable Assessment to Measure Preservice Elementary Teachers' Content Knowledge for Teaching about Matter
- Author
-
Katherine E. Castellano and Jamie N. Mikeska
- Abstract
There is strong agreement in science teacher education of the importance of teachers' content knowledge for teaching (CKT), which includes their subject matter knowledge and their pedagogical content knowledge. However, there are limited instruments that can be easily administered and scored on a large scale to assess and study elementary science teachers' CKT. Such measures would support strategic monitoring of large groups of science teachers' CKT and the investigation of comparative questions about science teachers' CKT longitudinally across the professional continuum or across teacher education or professional development sites. To address this gap, this study focused on designing an automatically scorable summative assessment that can be used to measure preservice elementary teachers' (PSETs') CKT in one high-leverage science content area: matter and its interactions. We conducted a field test of this CKT instrument with 822 PSETs from across the United States and used the response data to examine how this instrument functions as a potential tool for measuring PSETs' CKT in this science content area. Results suggest this instrument is reliable and can be used on large scale to support valid inferences about PSETs' CKT in this content area. In addition, the dimensionality analysis showed that all items measure a single construct of CKT about matter and its interactions, as participants did not show any differential performance by content topic or work of teaching science instructional tool categories. Implications for progressing the field's understanding of the nature of CKT and approaches to developing summative instruments to assess science teachers' CKT are discussed.
- Published
- 2024
- Full Text
- View/download PDF
37. The Factor Structure of the Arabic Version of the Metacognitive Awareness Inventory Short Version: Insights from Network Analysis
- Author
-
Albandri Sultan Alotaibi
- Abstract
Metacognition awareness is a fundamental skill for the 21st century. Accurately measuring metacognitive awareness would be highly relevant regardless of age, background, or cognitive abilities. The current study aimed to evaluate the psychometric properties of the 19-item Metacognitive Awareness Inventory-Arabic version (MAI-A) in the general population of Saudi Arabia. The current study employed Exploratory Factor Analysis (EFA), Confirmatory Factor Analysis (CFA), Cronbach's alpha and McDonald's omega (reliability), and average variance extracted and composite reliability (validity to evaluate the psychometric properties of MAI-A among a sample of the Arabian population. Measurement invariance across male and female samples had been conducted. Finally, the Exploratory Graph Analysis (EGA) was used to estimate the dimensional structure of the MAI. In the first step, quantitative face validity was presented to remove the one on the items because of poor indexes. So, the evaluated version was 18 items MAI. Also, the first-order and second-order CFA confirmed the 2-factor model. So, the 18-item MAI presented suitable internal consistency. Second-order average variance extracted validity showed suitable validity of the MAI-A. According to [delta]CFI and [delta]RMSEA, there was no gender invariance between males and females in the MAI-A structure. Finally, the EGA estimated a 3-dimensional structure of the MAI, which was different from the factor structure in the CFA. The MAI-A is a practical and cost-effective tool for evaluating metacognitive awareness in Arab populations. However, future studies should be conducted due to differences between traditional methods (CFA)I and novel methods (EGA) in extracting factors.
- Published
- 2024
- Full Text
- View/download PDF
38. Instability in Early Childhood Education Arrangements from Birth to 30 Months of Age: Associations with Children's Mental Health
- Author
-
Samantha Burns, Olesya Falenchuk, Evelyn McMullen, and Michal Perlman
- Abstract
Instability in early childhood education and care (ECEC) arrangements may have detrimental consequences on children's mental health. This study examined ECEC trajectories in the first 30 months of life for 373 children from low-income families in Toronto. We provide information about patterns of instability and reasons for instability. We also tested whether instability in ECEC was related to children's mental health using the Strengths and Difficulties Questionnaire. Five main profiles of ECEC stability were identified: (1) 8% in Home-Only, (2) 56.3% in Home-Center, (3) 8.0% in Home-Center-Home, (4) 5.9% in In-and-Out, and (5) 20.9% in Home-Multiple-Centers. Frequently mentioned reasons for change were a new baby in the family, availability of grandparents, quality of ECEC and family relocation. Families in these different profiles had similar demographic characteristics. Of the five profiles, children in the Home-Center-Home profile had a significantly higher likelihood of experiencing mental health problems compared to the Home-Center and Home-Multiple-Centers groups. These findings have important implications for future research, policy and practice related to ECEC availability.
- Published
- 2024
- Full Text
- View/download PDF
39. Agreement between Parental Reports of Part C Early Intervention Service Utilization and Part C Early Intervention Service Records
- Author
-
Michelle L. Stransky, Jocelyn Kuhn, and Emily Feinberg
- Abstract
Background: Part C of the Individuals with Disabilities Education Act mandates that eligible children under age three receive free access to early intervention (EI) through state-based programs. Despite a robust "Child Find" system, enrollment remains lower than expected. Community and medical professionals often rely on parent report of service utilization to gauge whether children are receiving developmental services. Understanding the reliability of parent report of EI can guide strategies to increase EI participation. Objective: To assess the extent to which parental reports of Part C EI services agree with state-based service records. Method: We address this research objective among 162 children and their parents from marginalized communities in Boston, MA and New Haven, CT who were identified as having a higher likelihood of autism and participated in a randomized controlled trial to facilitate access to autism evaluation and treatment. Parents reported whether their child was currently receiving EI services at four time points and released their child's EI service records from state agencies. Results: Overall agreement between parent and service records was 70% and ranged from fair to moderate and service-specific agreement ranged from 46% to 81.6%. Sensitivity, specificity, and positive/negative predictive values ranged widely for both overall and service-specific EI use. We found no differences in agreement by sociodemographic characteristics, but agreement was higher when children received EI services at home. Conclusions: Our findings support community and medical professionals' elicitation of children's EI service use from parents, especially for participation in EI overall and for physical and occupational therapy.
- Published
- 2024
- Full Text
- View/download PDF
40. Delicate Links: Ephemerality in Web-Based Evidence in Electronic Theses and Dissertations
- Author
-
Sarah Potvin, Tina Budzise-Weaver, and Kathy Christie Anders
- Abstract
This study suggests the need for best practices, specialized tools and standards, and targeted outreach related to Web-based content cited in theses. It analyzes citation practices in a corpus of master's theses in performance studies published at Texas A&M University from 2012 to 2020. Finding that only a slim majority of Web-based material cited in the theses remains fully functional within a decade of citation, this study confirms that "Web at large" content poses the greatest risk of irretrievable loss. Additionally, it considers actions by student authors that make theses vulnerable to evidentiary loss or change. A deeper understanding of the fragility of Web-based content and the potential for mitigation can inform needed interventions by librarians and other partners in graduate research.
- Published
- 2024
- Full Text
- View/download PDF
41. Validating the Communication and Symbolic Behavior Scales-Developmental Profile Infant-Toddler Checklist (CSBS-DP ITC) beyond Infancy in the CDKL5 Deficiency Disorder
- Author
-
Jacinta Saldaris, Helen Leonard, Kingsley Wong, Peter Jacoby, Mary Spence, Eric D. Marsh, Tim A. Benke, Scott Demarest, and Jenny Downs
- Abstract
CDKL5 deficiency disorder (CDD) results in early-onset epilepsy and lifelong cognitive and motor impairments. With no validated measure for communication in CDD, this study evaluated the psychometric properties of the Communication and Symbolic Behavior Scales-Developmental Profile Infant Toddler Checklist (CSBS-DP ITC). Caregivers (n = 150; affected individuals aged 1-29 years) completed the CSBS-DP ITC. Distribution of scores indicated a floor effect. There was poor divergent validity for the three-factor model but goodness of fit and convergent validity data were satisfactory for the one-factor model. Individuals with poorer overall functional abilities scored lower on the CSBS-DP ITC. Test-retest reliability was excellent. The floor effect could explain the very high reliability, suggesting problems as a sensitive outcome measure in clinical trials for CDD.
- Published
- 2024
- Full Text
- View/download PDF
42. Item Understanding of Common Quality of Life Measures for Use with Autistic Adults
- Author
-
Kelly B. Beck, Lauren A. Terhorst, Carol M. Greco, Jamie L. Kulzer, Elizabeth R. Skidmore, and Michael P. McCue
- Abstract
Quality of life (QOL) and life satisfaction are important research priorities for autistic adults. As such, we saw a need to evaluate individual items of commonly used subjective QOL scales to understand how they are interpreted and perceived by autistic adults. This study used cognitive interviews and repeated sampling to evaluate the accessibility, test-retest reliability and internal consistency of several common QOL measures in a sample of young autistic adults (n = 20; aged 19-32). Cognitive interviews suggested that the Satisfaction with Life Scale was well understood and demonstrated excellent internal consistency and test-retest reliability. While the WHOQoL-BREF and WHOQoL Disability Modules had adequate reliability, cognitive interviews suggested that additional instructions and examples would further enhance their accessibility for use with autistic adults.
- Published
- 2024
- Full Text
- View/download PDF
43. How Barriers Become Invisible: Children Are Less Sensitive to Constraints That Are Stable over Time
- Author
-
Jamie Amemiya, Gail D. Heyman, and Caren M. Walker
- Abstract
When making inferences about the mental lives of others (e.g., others' preferences), it is critical to consider the extent to which the choices we observe are constrained. Prior research on the development of this tendency indicates a contradictory pattern: Children show remarkable sensitivity to constraints in traditional experimental paradigms, yet often fail to consider real-world constraints and privilege inherent causes instead. We propose that one explanation for this discrepancy may be that real-world constraints are often stable over time and lose their salience. The present research tested whether children (N = 133, 5- to 12-year-old mostly US children; 55% female, 45% male) become "less" sensitive to an actor's constraints after first observing two constrained actors (Stable condition) versus after first observing two actors in contexts with greater choice (Not Stable condition). We crossed the "stability" of the constraint with the "type" of constraint: either the constraint was deterministic such that there was only one option available (No Other Option constraint) or, in line with many real-world constraints, the constraint was probabilistic such that there "was" another option, but it was difficult to access (Hard to Access constraint). Results indicated that children in the Stable condition became less sensitive to the probabilistic Hard to Access constraint across trials. Notably, we also found that children's sensitivity to constraints was enhanced in the Not Stable condition regardless of whether the constraint was probabilistic or deterministic. We discuss implications for children's sensitivity to real-world constraints.
- Published
- 2024
- Full Text
- View/download PDF
44. Dependability and Usability of the DASS-21 as a Progress-Monitoring Tool for Youth
- Author
-
Aberdine R. Dwight, Amy M. Briesch, Jessica A. Hoffman, and Christopher Rutt
- Abstract
Despite widespread evidence-based treatments for addressing internalizing concerns, many youth do not demonstrate reliable or clinically meaningful improvement. Regular progress monitoring, consisting of measurement and feedback, offers the opportunity to improve outcomes in real time. The 21-item Depression Anxiety and Stress Scales (DASS-21; Lovibond & Lovibond, 1995) has potential as a progress-monitoring tool for internalizing concerns in youth; however, limited psychometric data are available to support this use. The purpose of this study was to investigate the dependability of data obtained from the DASS-21 when completed by adolescents in a clinical setting. This study also aimed to understand the feasibility and utility of using the DASS-21 as a progress-monitoring tool from youth and clinicians' perspectives. Generalizability and dependability analyses were conducted to determine the number of ratings needed to obtain a dependable estimate of youth functioning within 1 week. Whereas two daily ratings were needed to dependably estimate total distress, results for the depression, anxiety, and stress subscales indicated that two to five data points would be needed over the course of the week. Finally, results demonstrate the usability of the DASS-21 from both youth and clinician perspectives when used in a progress-monitoring context.
- Published
- 2024
- Full Text
- View/download PDF
45. Revision and Validation of the Parental Autonomy Support Scale among Chinese College Students
- Author
-
Yuanshu Fu, Yang Wang, Min Hao, Zhenling Ou, Ziru Zeng, Wanting Ren, and Xinwen Xu
- Abstract
To revise the Parental Autonomy Support Scale (PASS) and to validate its psychometric properties in Chinese college students, this study recruited 738 Chinese college students, with 72 of them retested four weeks later. The Parental Psychological Control Scale (PPCS) and the Self-Esteem Scale (RSES) were used to evaluate the validity of the PASS. Results showed that the revised PASS contained a total of nine items measuring three dimensions: autonomous decision-making, transpositional thinking, and willingness to respect. Results showed that the three-factor ESEM model fitted the data well (TLI = 0.981, CFI = 0.994, RMSEA = 0.046). The PASS showed a significant negative correlation between the total score of the PASS and the total score of the PPCS (r = -0.344, p < 0.001), and a significant positive correlation of the PASS with the total score of the RSES (r = 0.297, p < 0.001). The composite reliability coefficient was 0.913 for the total score. The test-retest reliability coefficients were 0.827 for the total score and ranged from 0.626 to 0.744 for the three dimensions. The revised PASS is shown to have sound psychometric properties and thus can be used as a tool to measure the level of parental autonomy support among Chinese college students.
- Published
- 2024
- Full Text
- View/download PDF
46. Adaptation and Validation of the Academic Motivation Scale for Higher Education across Four Eastern European Countries
- Author
-
Ilona Kocvarová, Jan Kalenda, Jitka Vaculíková, Zuzana Neupauer, Ruženka Šimonji Cernak, and Anna Wloch
- Abstract
The article focuses on adaptation and validation of the Academic Motivation Scale questionnaire (AMS-28) in higher education in four Eastern European countries: Czechia, Slovakia, Serbia, and Poland. The research was conducted with a total of 1711 respondents. We examined the construct validity of AMS-28 including measurement invariance and reliability according to national, gender and age groups. Our analysis confirmed its original seven-factor structure as well as its reliability. The tool is measurement invariant across all compared groups (gender, age, countries) except Poland, whose results are specific. The results identify new places on the world map where AMS-28 is functional. The tool appears functional in time, space and various language mutations. Despite satisfactory results, there is still room for future examination of the AMS-28 among different countries. In practice, the tool has a wide range of application possibilities.
- Published
- 2024
- Full Text
- View/download PDF
47. Can Large Language Models Replace Humans in Systematic Reviews? Evaluating GPT-4's Efficacy in Screening and Extracting Data from Peer-Reviewed and Grey Literature in Multiple Languages
- Author
-
Qusai Khraisha, Sophie Put, Johanna Kappenberg, Azza Warraitch, and Kristin Hadfield
- Abstract
Systematic reviews are vital for guiding practice, research and policy, although they are often slow and labour-intensive. Large language models (LLMs) could speed up and automate systematic reviews, but their performance in such tasks has yet to be comprehensively evaluated against humans, and no study has tested Generative Pre-Trained Transformer (GPT)-4, the biggest LLM so far. This pre-registered study uses a "human-out-of-the-loop" approach to evaluate GPT-4's capability in title/abstract screening, full-text review and data extraction across various literature types and languages. Although GPT-4 had accuracy on par with human performance in some tasks, results were skewed by chance agreement and dataset imbalance. Adjusting for these caused performance scores to drop across all stages: for data extraction, performance was moderate, and for screening, it ranged from none in highly balanced literature datasets (~1:1) to moderate in those datasets where the ratio of inclusion to exclusion in studies was imbalanced (~1:3). When screening full-text literature using highly reliable prompts, GPT-4's performance was more robust, reaching "human-like" levels. Although our findings indicate that, currently, substantial caution should be exercised if LLMs are being used to conduct systematic reviews, they also offer preliminary evidence that, for certain review tasks delivered under specific conditions, LLMs can rival human performance.
- Published
- 2024
- Full Text
- View/download PDF
48. Different Methods for Assessing Pre-Service Teachers' Instruction: Why Measures Matter. EdWorkingPaper No. 23-862
- Author
-
Annenberg Institute for School Reform at Brown University, Arielle Boguslav, and Julie Cohen
- Abstract
Teacher preparation programs are increasingly expected to use data on pre-service teacher (PST) skills to drive program improvement and provide targeted supports. Observational ratings are especially vital, but also prone to measurement issues. Scores may be influenced by factors unrelated to PSTs' instructional skills, including rater standards and mentor teachers' skills. Yet we know little about how these measurement challenges play out in the PST context. Here we investigate the reliability and sensitivity of two observational measures. We find measures collected during student teaching are especially prone to measurement issues; only 3-4% of variation in scores reflects consistent differences between PSTs, while 9-17% of variation can be attributed to the mentors with whom they work. When high scores stem not from strong instructional skills, but instead from external circumstances, we cannot use them to make consequential decisions about PSTs' individual needs or readiness for independent teaching.
- Published
- 2023
49. Simulated Learners in Educational Technology: A Systematic Literature Review and a Turing-Like Test
- Author
-
Tanja Käser and Giora Alexandron
- Abstract
Simulation is a powerful approach that plays a significant role in science and technology. Computational models that simulate learner interactions and data hold great promise for educational technology as well. Amongst others, simulated learners can be used for teacher training, for generating and evaluating hypotheses on human learning, for developing adaptive learning algorithms, for building virtual worlds in which students can practice collaboration skills with simulated pals, and for testing learning environments. This paper provides the first systematic literature review on simulated learners in the broad area of artificial intelligence in education and related fields, focusing on the decade 2010-19. We analyze the trends regarding the use of simulated learners in educational technology within this decade, the purposes for which simulated learners are being used, and how the validity of the simulated learners is assessed. We find that simulated learner models tend to represent only narrow aspects of student learning. And, surprisingly, we also find that almost half of the studies using simulated learners do not provide "any" evidence that their modeling addresses the most fundamental question in simulation design -- is the model valid? This poses a threat to the reliability of results that are based on these models. Based on our findings, we propose that future research should focus on developing more complete simulated learner models. To validate these models, we suggest a standard and universal criterion, which is based on the lasting idea of Turing's Test. We discuss the properties of this test and its potential to move the field of simulated learners forward.
- Published
- 2024
- Full Text
- View/download PDF
50. Transforming Assessment: The Impacts and Implications of Large Language Models and Generative AI
- Author
-
Jiangang Hao, Alina A. von Davier, Victoria Yaneva, Susan Lottridge, Matthias von Davier, and Deborah J. Harris
- Abstract
The remarkable strides in artificial intelligence (AI), exemplified by ChatGPT, have unveiled a wealth of opportunities and challenges in assessment. Applying cutting-edge large language models (LLMs) and generative AI to assessment holds great promise in boosting efficiency, mitigating bias, and facilitating customized evaluations. Conversely, these innovations raise significant concerns regarding validity, reliability, transparency, fairness, equity, and test security, necessitating careful thinking when applying them in assessments. In this article, we discuss the impacts and implications of LLMs and generative AI on critical dimensions of assessment with example use cases and call for a community effort to equip assessment professionals with the needed AI literacy to harness the potential effectively.
- Published
- 2024
- Full Text
- View/download PDF
Catalog
Discovery Service for Jio Institute Digital Library
For full access to our library's resources, please sign in.