Search

This paper develops a two-way research method - both top-down and bottom-up - to define a "standard but tailored" assessment framework for smart cities, based on shared smart city concepts, but designed to respond to different needs of each city. The method aims not to design a measurement standard, but to define a process able to create the best smart city measurement system. It is based on a standard framework but tailored on each city - with its own features, problems, values, and ideas about the quality of life. This method overcomes the limitations of using standard framework, as it links smart city assessment tools to local policies but giving to the tool the authoritativeness deriving from the scientific literature and the robustness acquired from international nest practices. An empirical implementation supports the theoretical background and allows to validate the method, as it has been successfully implemented in Genoa, a medium-sized city in Italy looking for its own performance measurement system that allows comparison with other smart cities at the same time. The bidirectional method is designed by the authors of this paper, and it is a novelty in the international literature about research methodology for business and management studies. [ABSTRACT FROM AUTHOR]

Currently, Living Labs are evaluated in multiple different ways which is hampering comparison. This paper proposes a harmonized method combining various elements (6 chapters, 15 criteria and 34 KPIs), mainly based on the standardized evaluation framework developed by Vervoort et al. (2022) for evaluating the diverse types of Living Labs. Such a harmonized approach will help to simplify evaluation processes and will allow to compare the stability and maturity of LLs in a uniform manner. The premise is that creating a harmonized assessment method will increase the sustainability of Living Labs by providing them a method that allows comparison of the maturity of multiple types of Living Labs without losing sight of the particularities of individual Living Labs within their specific contexts. [ABSTRACT FROM AUTHOR]

This paper provides for a systematic review of the literature related to social impacts, indicators, and metrics, associated with the circular economy. A total of 47 papers were included within the final review, which included 34 methods for the assessment of social circularity. Primarily, the paper sought to understand how social impacts are identified when circular economy ventures are assessed, as well as identifying what the key methodologies are when actors seek to evaluate these social impacts. The study identified a prevalence of social impact indicators utilized within wider methods of circular evaluation, that sought to evaluate social sustainability within minimal indices, alongside more easily quantifiable phenomena. Job creation and health and safety figures were most frequently utilized within the limited indices. [ABSTRACT FROM AUTHOR]

This paper develops a two-way research method -- both top-down and bottom-up -- to define a "standard but tailored" assessment framework for smart cities, based on shared smart city concepts, but designed to respond to different needs of each city. The method aims not to design a measurement standard, but to define a process able to create the best smart city measurement system. It is based on a standard framework but tailored on each city -- with its own features, problems, values, and ideas about the quality of life. This method overcomes the limitations of using standard framework, as it links smart city assessment tools to local policies but giving to the tool the authoritativeness deriving from the scientific literature and the robustness acquired from international nest practices. An empirical implementation supports the theoretical background and allows to validate the method, as it has been successfully implemented in Genoa, a medium-sized city in Italy looking for its own performance measurement system that allows comparison with other smart cities at the same time. The bidirectional method is designed by the authors of this paper, and it is a novelty in the international literature about research methodology for business and management studies. [ABSTRACT FROM AUTHOR]

The American Council for Construction Education (ACCE) requires all accredited construction programs to demonstrate a direct assessment of twenty Student Learning Outcomes (SLOs). Before graduation, many construction programs require students to take the American Institute of Constructors (AIC) Associate Constructor (AC) exam, where SLOs will be assessed at the national level. There has been some discussion in the literature regarding an improvement to program-level SLO assessment to increase the overall AC exam score. This paper aims to investigate whether construction programs should improve their program-level SLOs assessments to meet the AC exam criteria and whether such improvements will help students pass the AC exam. Data were collected for two groups of students from Spring 2018 to Spring 2020. The first group of students was assessed using the original program-level assessment. The second group of students was assessed using improved program-level assessment developed based on the AC exam guidelines. Descriptive and statistical analysis were conducted and the results indicated that these actions and the improvement to the assessment plan can effectively impact and improve the students' AC exam results. This research provides current construction educators with a new perspective on using and validating the AC exam as an effective assessment tool. It also elucidates correlations between program SLOs and the bodies of knowledge in the AC exam. [ABSTRACT FROM AUTHOR]

As a key aspect of higher education programmes, assessment is both unavoidable and complex. Students must be assessment literate to achieve high grades. This paper will discuss a key finding from a Ph.D. study that set out to explore how written assignment briefs affect students' understanding of the requirements of assessment, and students' potential to perform effectively. Before the COVID-19 pandemic, the written word was the dominant mode of communicating assessment and for distance education students, it was potentially the sole form of communication. For online programmes, reliance on the written assignment brief is significant. While the impact of COVID-19 on the future of higher education is unclear, higher education institutions may continue to offer an online option, or retain some online elements; thus signalling the continued prevalence of written briefs. With a dearth of research focusing on the design stage of the written assignment brief, this Ph.D. set out to determine how the assignment brief influences effective communication. Consisting of three phases, this mixed-methods study combined quantitative and qualitative data gathered across three semesters (spring 2016, autumn 2016, and spring 2017). Following the University's assignment regulations, Hughes' Assessment Task Design (2009) framework, and Gilbert and Maguire's Assignment Brief Guidelines (2014), the researcher worked with four academics (one from each of the four faculties in the University) to design their assignment briefs. The third phase involved conducting focus groups with students and interviews with academics to discuss their experiences of the effectiveness of these briefs. This paper will examine the impact of one of the key findings: the need for comprehensive yet concise assignment briefs, and its relevance to planning an assessment strategy. This paper will recommend a potential solution to balance the tension of providing detailed assignment briefs while maintaining conciseness. The findings will be valuable to academics and instructional designers involved in designing assessment documentation or planning an assessment strategy. [ABSTRACT FROM AUTHOR]

Automatic difficulty calibration (ADC) is the application of computational techniques to estimate the difficulty levels of assessment questions before administering them. Compared to traditional difficulty calibration approaches, ADC eliminates the need for pretesting, and minimises the time and efforts typically involved in manual calibration. In recent years, Ontology-based Automatic Question Generation (OAQG) has emerged as a powerful tool to generate assessment questions effortlessly and in massive numbers with minimal human intervention. Despite these benefits, not being able to control the characteristics of generated questions hinders their suitability to be used in pedagogical settings. However, much of the research up to now has tended to focus on the quantity of the questions rather than their quality. Indeed, most current automatic question generators produce simple questions which consist of a few facts, and simply test the recall of knowledge. Furthermore, the majority of existing frameworks are mostly technical and are not supported with a strong theoretical underpinning. In this paper, we propose a novel framework to assess the complexity of ontology-based, automatically generated questions. We discuss various aspects that are involved in determining the complexity of assessment questions, and attempt to quantify important characteristics of question complexity through the use of novel ontological metrics. To further support the plausibility of our computational framework, we shed light on its consistency with theories from education and cognitive psychology. This will provide us with a solid theoretical foundation which ensures that questions are generated according to principled methods that are grounded in theories of learning and cognition. The proposed approach is agnostic to different domains and independent of the question format, therefore, is highly general and applicable to a variety of contexts. [ABSTRACT FROM AUTHOR]

Increasingly ports, like other infrastructure asset owners are faced with investment decisions regarding the use of aging and deteriorating wharves and seawalls against a backdrop of increased performance and health and safety requirements. The decision to build new vs extending the life of an asset requires in-depth knowledge of the condition, strength and expected levels-of-service and remaining life of the structure. Often the age of the structure means that as-built information is lacking and to determine an asset risk profile, condition and structural capacity assessments are required. Undertaking inspections within operational restrictions, confined spaces and contending with adverse weather means that smarter technologies are needed to ensure quality of data as well as safety of people. There are also instances where extreme events such as the Kaikoura earthquake in New Zealand require quick engineering assessments to identify damage and reinstatement options. This paper describes case studies of wharf condition and performance assessment for CentrePort (Wellington) and Port Otago. As trusted advisors to these organisations, we utilise experienced structural engineers, supported by technologies for data capture and processing to build a complete picture of the critical elements of the wharves in terms of condition, strength and remaining service life. This includes mobile applications for data capture and condition rating, 3D modelling leading into structural analysis and ultimately determining long-term asset management actions. A structures health approach can be used to capture all relevant structural and non-structural risks, in order to prioritise value-for-money options. [ABSTRACT FROM AUTHOR]

Engagement can be described as students' tendency to work productively, think deeply, enjoy and value their learning, and to support each other in the process of learning (ZDM article). Each of these dimensions can be indexed by a variety of psychologically validated constructs such as interest and enjoyment, self-regulation, effort, emotional valence and object. The purpose of this paper is to describe a method of assessment that takes under 5 minutes to administer as students are learning mathematics concepts or shortly thereafter in the context of classroom observation and analysis of practices that may support productive engagement. A description of an online survey, its development and psychometric properties assessed in a study of over 1,000 secondary mathematics students in the US is presented, and implications for research on task-level engagement in mathematics classrooms is discussed. [ABSTRACT FROM AUTHOR]

This paper examines the theoretical groundings for the application of gamified elements in the mathematics classroom. The analysis focused on observing gamified elements that are already commonly used within mathematics classrooms and identifying other gamified elements that could be applicable to mathematics teaching to increase student engagement. Key gamified elements such as content unlocking, teams, and boss battles can be integrated into teaching can increase student engagement and create a unique educational experience within classrooms. [ABSTRACT FROM AUTHOR]

Tests tend to be the default assessment in fundamental engineering courses. Tests can be useful for learning such as retaining information, but not for all learning outcomes like conceptual change. In addition, tests can decrease student motivation to learn. Therefore, complementing tests with alternative assessments and being intentional in using and designing tests should be considered to address the problem test being the go-to form of assessments in fundamental courses. Instructor resistance to adopting teaching practices informed by research poses as another barrier toward addressing the problem. Thus, understanding engineering instructors' beliefs about why they use tests in their courses is an important first step toward addressing the problem. This research study begins addressing the gap by analyzing one case out of a larger multi-case study to provide hermeneutic insights for future analysis and studies on the beliefs of engineering instructors about using tests. Grounded in Situated Expectancy Value Theory (SEVT), this research paper analyzed one thermodynamics instructor, Charlie, who was a mechanical engineering associate professor in an R1 public, land-grant institution, and had taught for more than ten years. Analysis methods were inspired by the case study research methodology. Findings show slight conflicting beliefs within Charlie as they demonstrated strong beliefs in test benefits in helping students learn, but apologetic about using tests because of factors like inertia and peer pressure that compelled them to continue using tests in the course. Charlie was also apologetic about using tests due to their lack of ability to predict future student success. Charlie also expressed a lack of experience and knowledge in designing assessments in their courses. Future work will focus on understanding test usage beliefs at a broader scale and informing research to design alternative assessments that can be adapted to complement tests. [ABSTRACT FROM AUTHOR]

Over the past decades, educational pedagogies have been shaped by many theorists and frameworks. For the purpose of this paper, the researchers define educational pedagogy through a lens that is influenced by Paulo Freire. For instance, educational pedagogy can be defined as seeking a balance within the tension between freedom and authority, while navigating the challenging structures that shape educational institutions (Freire, 2005, 2020). These structures include the impact of lived experiences and realities, the role of curricula, and the changing interactions within society that influence the pedagogical frameworks in education (Freire, 2020). This paper begins to consider a new framework for assessment and evaluation in Ontario, Canada's K-12 schools, where educational pedagogy, empathy, metacognition, and critical consciousness may be posited as potential components moving forward. This paper deeply examines these three key elements of effective and authentic assessment: empathy, critical consciousness and metacognition. Each of these terms is defined and situated in a 21C digital learning context, and the authors suggest that new directions in assessment and evaluation must consider these three significant as fundamental to any redesign of assessment practices. Consideration may be given to the perspectives and experiences of educational leaders, educators, students, and their communities, and how these perspectives can shape the values and how we meet the challenges in education. Pedagogy in practice can often reflect the strategies and beliefs that influence educators' values and choices, and the resulting impact on learning. Part of these pedagogical considerations may include the development of empathy, critical consciousness and metacognition throughout the assessment and evaluation process. Using a Canadian context, the authors argue that post-pandemic learners require a different set of tools to learn, and post-pandemic teachers need to reconfigure the ways in which that learning is measured, assessed and evaluated. [ABSTRACT FROM AUTHOR]

A movement to include problem-solving and computer science in k-12 education has sparked significant interest in introducing computational thinking (CT). CT education is mainly defined as teaching and learning problem-solving skills. CT is considered a 21-century skill, and like other essential skills aiming to educate students as efficient members of the technology-dependent society, CT learning and assessment are associated with the use of technology-enhanced learning methods and environments. Although most researchers categorize CT skills into three groups, including CT concepts, practices, and perspectives, there is no consensus view regarding CT assessment methods to evaluate these three CT skill categories. Addressing this gap, we explored key topics in the computational thinking assessment (CTA) literature using a data-driven approach for topic modeling. We analyzed 395 articles in CTA literature and identified 11 research topics of CTA. We also performed a network analysis to explore the key links between CTA's identified topics. Based on the results from topic modeling, we presented CTA methods and categorized the assessment tools based on their assessment strategy and the types of CT skills they aim to evaluate. Also, the paper analyzes the identified assessment methods based on the purpose of assessment and the different types of insights they provide for the evaluation of CT skills. The paper discusses the advantages of new forms of CTA through technology compared to traditional assessment methods and provides recommendations for further studies. [ABSTRACT FROM AUTHOR]

The paper reports on the developments of a repository of quantitative assessments used in mathematics education contexts. This repository centralizes assessments and the associated validity evidence. The repository is public and freely available and has potential to inform future quantitative mathematics education scholarship. [ABSTRACT FROM AUTHOR]

Tests of statistical significance often play a decisive role in establishing the empirical warrant of evidence-based research in education. The results from pattern-based assessment items, as introduced in this paper, are categorical and multimodal and do not immediately support the use of measures of central tendency as typically related to interpretations of measures of statistical significance. Responses from the duplicate implementation of selected pattern-based items (PBIs) in successive grades (3-8) as part of the statewide Interim Assessment Program in Texas are used to illustrate how non-parametric methods can be used to establish statistically significant comparisons of student results. Not all the repeat-item results improved across years. [ABSTRACT FROM AUTHOR]

This full paper concerns an exploratory study that investigates students' reasoning about torsion. Mechanical reasoning is critical to engineering applications and yet students still struggle to accurately predict, analyze, and model mechanical systems using formal symbolic notations (i.e., formalizations). To understand the nature of students' reasoning, we analyzed students' discourse to explore two competing hypotheses: (H1) The Formalisms First (FF) hypothesis that students report their mechanical reasoning predominantly using mathematical formalisms that take on a disembodied, allocentric (observer) point-of-view; or (H2) the Grounded and Embodied Cognition (GEC) hypothesis that students predominantly use independent speech which includes analogy and imagery to simulate the physical structure and function of an object(s) using an embodied, egocentric (first-person) point-of-view in addition to an allocentric point-of-view. Qualitative results from discourse analysis of two student dyads showed that students' mechanical reasoning revealed through their speech included both analogy and imagery, as predicted by H2. Students generated analogies and imagery that described dynamic behaviors, such as how torque caused displacement, stored and released energy, and fractured. Usage of analogies and imagery supports that students' mechanical reasoning often drew upon simulations of torsion-related sensorimotor experiences. Students' egocentric and allocentric imagery invoked sensorial experiences in their speech, with allocentric viewpoints being more common, as predicted by H1 and H2. Student discourse included many references to formalisms, also consistent with the H1. Data from students' verbal discourse on mechanical reasoning suggests they employ both GEC and FF viewpoints of torsion, which has implications for designing effective learning experiences and for assessing students' knowledge. [ABSTRACT FROM AUTHOR]

The Internet of things (IoT) has revolutionized many aspects of the world, including industrial systems, automobiles, home automation, and surveillance, to name a few. IoT has offered a multitude of conveniences for our daily lives, such as being able to control our thermostats remotely, view our home surveillance cameras while away, or have a smart television that can surf the web. However, the widespread adoption of IoT devices combined with their vast device vulnerabilities results in significant security risks reinforcing the need for more robust default security controls and public awareness. As such, this paper aims to discover and document security vulnerabilities in the Asus Zenbo Junior IoT robot, along with providing a few best practices when securing smart home devices. This work presents an experiment using several security vulnerability assessment tools such as Nmap and OpenVAS scans to assess cybersecurity vulnerability currently present on Zenbo based on the 4P forensic investigative framework. The result of the experiment shows multiple open ports were discovered, along with miscellaneous information that an attacker could use to their advantage to attack the Zenbo robot. Based on the result, this work presents various security precautions that can help users protect against cyber-attack. [ABSTRACT FROM AUTHOR]

The public availability of sophisticated AI content generators based on large -scale language models achieved enormous media attention in recent months. AI content generators are expected to have a major impact on future information sy stems developments as well as private and business life. Particularly in the field of education, the question arises as to how these innovations in AI technology will impact teaching, learning, and examination. With a specific focus on examinations in the information systems di scipline, we show in this research-in-progress paper, based on an experimental field study, how a today’s AI content generator performs in solving a fundamental information systems exam. Based on our preliminary results, we found that the quality varies ac ross different exam question ty pes. With this research project, we offer start points and initial guidance on how educators can deal with the availability of AI content generators. [ABSTRACT FROM AUTHOR]

Student experiences in first-year mathematics (FYM) courses are crucial to their success in college. In recent years, there has been a push to improve FYM courses by centering and engaging students. The purpose of this study is to examine how students perceive their experiences with active learning and standards-based grading in FYM courses. In this paper, we present our analysis from end-of-semester Mathographies written by students who were enrolled in FYM courses in Fall 2021. We discuss three preliminary findings: community, affect, and assessment. We compare these findings to a framework relating to students' experiences with active learning and provide questions for future exploration. [ABSTRACT FROM AUTHOR]

This paper offers a socio-cultural perspective on the mechanism by which research-based assessment tools designed to evaluate mathematics teaching, are used as pedagogical tools for teacher learning. This process is exemplified by the Realization Tree Assessment tool (RTA), showing how pre-service teachers refer to the pedagogical messages valued and devalued by the assessment tool, and which conflicts between these messages they attend to. [ABSTRACT FROM AUTHOR]

This paper examines the possibilities of designing a formative assessment that gathers information about novice elementary teachers' skills with modeling content and makes sense of such information. A decomposition of the practice of modeling content was developed and used to design the assessment. Participants included ten first-year teachers who graduated from a range of different teacher education programs. The findings reveal that our formative assessment works to gather information about teachers' capabilities with modeling content and that the associated tools support making sense of the information gathered. [ABSTRACT FROM AUTHOR]

In this paper, we describe the method and results of our investigation of structural validity of the 2021 Knowledge for Teaching Early Elementary Mathematics test, which measures teachers' mathematical knowledge for teaching at the early elementary level. The 2021 Knowledge for Teaching Early Elementary Mathematics test was used in a long-term, randomized controlled trial of the effect of a Cognitively Guided Instruction professional-development program on teachers, teaching, and students. Our sample includes data collected in spring 2021 from 651 grades K-2 elementary educators in Florida. Exploratory factor analysis and analyses based on classical test theory and item response theory were conducted to examine the psychometric properties of the test. Results of factor analysis suggest a one-factor solution. The marginal reliability for response pattern scores was 0.84, and the test difficulty appears to align well with the levels of mathematical knowledge for teaching of the target population. [ABSTRACT FROM AUTHOR]

What is gained when midterm oral exams are implemented in the undergraduate engineering classroom? This research paper examines whether midterm oral exam scores add value above and beyond midterm written exam scores in predicting students' final written exam scores. The purpose of this study is to evaluate the potential utility of oral exams as formative assessments: if oral exam scores provide additional information beyond written exam scores, they may add meaningful value for students and instructors. The current study investigates this question using data from 10 undergraduate engineering classes (N = 925), representing 6 different courses and 5 different instructors. Though course and exam context differed, all classes implemented a low-stakes midterm oral exam, a midterm written exam, and a midterm final exam. We compared two multiple regression models: a smaller model with only the midterm written exam score as a predictor for the final written exam score, and a full model with both the midterm written exam score and the midterm oral exam score as predictors for the final written exam score. We found that the fuller model with oral exam score was a better fit for our data, indicating that including oral exams explains more variance in students' final exam performance than midterm written exams alone. Further analyses tentatively indicate that the granularity of the rubric used to score oral exams matters, with finer-grained rubrics more consistently providing predictive value. This study has implications for developing a theory of oral exams, as it leaves room for the possibility that oral exams tap deeper learning processes than written exams. These results show that oral exams provide actionable information instructors can use to make interventions to foster students' meaningful learning before the end of the term. The quantitative analysis also provides instructors with a simple statistical measure to assess the role of oral exams in student's learning. [ABSTRACT FROM AUTHOR]

This full paper concerns the use of gesture analysis to guide instructional approaches in engineering education. Engineering is rife with abstract mathematics and processes for quantifying physical phenomena. In engineering instruction, formalisms first is a practice that privileges formalisms over grounded and applied ways of knowing that are common in engineering curricula. By way of contrast, progressive formalization is an alternative pedagogical practice that intentionally grounds the meaning of mathematical formalisms in one's sensorimotor experiences in order that the formalisms are meaningful to learners. In the courses of explaining engineering concepts, instructors often make iconic gestures (gestures that represent objects, actions, and relationships) that are based in perception and action as a means for grounding domain knowledge prior to introducing formalisms. In response, students' gestures can be either concordant (i.e., conceptually aligned) or discordant (i.e., conceptually misaligned). The latter, also known as gesture-speech mismatches, are indices for states of transitional knowledge in which learners exhibit a readiness to learn. Thus, the current research observes the spontaneous gestures students make while describing torsional loading and investigates the added benefits of incorporating gesture into formative assessments of engineering education. Results indicate that students do use gestures as integral parts of their explanations in an engineering lab setting and that gestures and co-articulated speech were often matched. Instances of gesture-speech mismatches provides instructors opportunities to assess student knowledge, knowledge-in-transition, and initial learning and correct understandings prior to summative assessments. [ABSTRACT FROM AUTHOR]

This paper presents the findings of the mapping and evaluation of macrophyte vegetation from Tataru and Uzlina lakes, located within Danube Delta, Romania. Throughout time, the Danube Delta unique ecosystem has endured considerable ecological changes due to natural and anthropogenic stressors. Currently, the potential threats are those related to pollution, agriculture, land use, climate change and subsequently, the proliferation and/or the extinction of many macrophyte species. The excessive development of aquatic vegetation has negative consequences, reducing the flow velocity of water in the connecting channels, exacerbates the sedimentation processes in the deltaic lakes which are subject to the clogging phenomenon (i.e., silting). The abundance, distribution and biodiversity of the emergent aquatic species were surveyed in Tataru and Uzlina lakes. The biodiversity of macrophytes is different in the investigated lakes, being dependent on the fluvial input of water and sediment of the Danube River, extent of lakes and water depth. In Tataru Lake, the macrophyte vegetation was represented by a greater variety of species (Nymphaea alba, Nuphar lutea, Trapa natans, Salvinia natans, Lemna minor and Phragmites australis), in comparison to Uzlina Lake (Nymphaea alba, Nuphar lutea, Trapa natans and Phragmites australis). This methodology and results are valuable for the strategic environmental assessment and management of the Danube Delta Biosphere Reserve and can be used in similar settings. [ABSTRACT FROM AUTHOR]

This Complete Research paper investigates the holistic assessment of creativity in design solutions in engineering education. Design is a key element in contemporary engineering education, given the emphasis on its development through the ABET criteria. As such, design projects play a central role in many first-year engineering courses. Creativity is a vital component of design capability which can influence design performance; however, it is difficult to measure through traditional assessment rubrics and holistic assessment approaches may be more suitable to assess creativity of design solutions. One such holistic assessment approach is Adaptive Comparative Judgement (ACJ). In this system, student designs are presented to judges in pairs, and they are asked to select the item of work that they deem to have demonstrated the greatest level of a specific criterion or set of criteria. Each judge is asked to make multiple judgements where the work they are presented with is adaptively paired in order to create a ranked order of all items in the sample. The use of this assessment approach in technology education has demonstrated high levels of reliability among judges (~0.9) irrespective of whether the judges are students or faculty. This research aimed to investigate the use of ACJ to holistically assess the creativity of first-year engineering students design solutions. The research also sought to explore the differences, if any, that would exist between the rank order produced by first-year engineering students and the faculty who regularly teach first-year students. Forty-six first-year engineering students and 23 faculty participated in this research. A separate ACJ session was carried out with each of these groups; however, both groups were asked to assess the same items of work. Participants were instructed to assess the creativity of 101 solutions to a design task, a "Ping Pong problem," where undergraduate engineering students had been asked to design a ping pong ball launcher to meet specific criteria. In both ACJ sessions each item of work was included in at least 11 pairwise comparisons, with the maximum number of comparisons for a single item being 29 in the faculty ACJ session and 50 in the student ACJ session. The data from the ACJ sessions were analyzed to determine the reliability of using ACJ to assess creativity of design solutions in first-year engineering education, and to explore whether the rankings produced from the first-year engineering students ACJ session differed significantly from those of the faculty. The results indicate a reasonably high level of reliability in both sessions as measured by the Scale Separation Reliability (SSR) coefficient, SSRfaculty = 0.65 ± 0.02, SSRstudents = 0.71 ± 0.02. Further a strong correlation was observed between the ACJ ranks produced by the students and faculty both when considered in terms of the relative differences between items of work, r = .533, p < .001, and their absolute rank position, s = .553, p < .001. These findings indicate that ACJ is a promising tool for holistically assessing design solutions in engineering education. Additionally, given the strong correlation between ranks of students and faculty, ACJ could be used to include students in their own assessment to reduce the faculty grading burden or to develop a shared construct of capability which could increase the alignment of teaching and learning. [ABSTRACT FROM AUTHOR]

Along with the rapid advances in technology and innovation today because of globalization of the 21st century and the industrial revolution 4.0, educational learning outcomes need to be maintained so as not to deviate to the point of failure. Mandarin is a compulsory subject in Three Language schools (Indonesian, English, Mandarin). Thus, learning Mandarin at the Three-Language School also involves all forms of innovation and transformation that are currently taking place in the delivery of learning materials and learning assessments. Learning outcomes from learning are students' ability to communicate. This paper intends to analyze the latest technological advances and innovations in the Chinese language teaching process along with student test scores, data collection methods based on student activity in collecting assignments in Google Classroom (GCR) and students' final grades. The respondents of this study were 26 junior high school students at the Pahoa Three Language School, Tangerang. The exam questions include completing the dialogue (60%), answering questions after hearing the contents of the text (30%) and answering questions from reading (10%). The learning process that lasted for 4 months was evaluated 5 times with assignment at GCR. The conclusion of this study is that the intensity of task collection has a close relationship with test results. This research is expected to be a reference for further research by utilizing technology and innovation in learning and assignments, especially in foreign language learning. [ABSTRACT FROM AUTHOR]

The seeds of innovation are changing how libraries use their space and plan for future remodels, renovations or expansions, especially post-COVID. Examples of this range from converting space from materials storage to user space needs along with technology enhancements. Technology driven initiatives that impact space usually start as small ideas or seeds of innovative initiatives, such as makerspaces, but are growing into digital media centers, video and imaging rooms, gaming labs and augmented reality spaces. This presentation shares a case study approach to a master space planning project but goes beyond the typical consultant led project into developing a boarder perspective on the factors related to addressing innovative changes and initiatives for both present needs and future developments. As part of renovation planning, an assembled team, representing a wide range of interests conducted site visits to selected academic libraries to visualize and discuss trends and initiatives impacting space planning. This information was shared with stakeholders in a focus group format to vet out potential seeds of expectation on how the use of library space can contribute to the success of individuals, groups and the larger institution. The drivers for creating goals in the planning effort went beyond the library walls to include community, digital scholarship and convening spaces. Research was conducted regarding the pedagogy that libraries can offer, in order to create environments that promote and support knowledge creation. The library should be considered the learning hub on campus and be integrated into each level of the educational mission to ensure student success and faculty research support. This paper/presentation breaks down how small seeds of ideas, new concepts or futuristic thinking can be incorporated into space design and renovation efforts. This anatomy of a renovation will be very visual and active in its delivery. [ABSTRACT FROM AUTHOR]

An accurate global assessment of gifted children is essential to provide them with educational and counseling programs tailored to their strengths and vulnerabilities. Without neglecting the fundamental role of social-emotional skills on individual well-being and academic success, it is essential to shedding light on the attentional profile of intellectually gifted children. In this regard, findings from empirical studies are not always consistent. The purpose of this exploratory study was to investigate the relationship between the cognitive profile of fifteen gifted children (mean age 9.6 years; SD 2.3), as described through the main WISC-IV indexes (IVE, IRP, IML, IVE, FSIQ), and their attentional profile assessed through AULA, an innovative VR-based ecological assessment tool that evaluates attention and inhibitory control. These first findings highlighted the correlation between intelligence and attention and provided indications for educational interventions. Therefore it is crucial to use innovative and highly ecological assessment tools to engage the IGen gifted learners. In this regard, researchers have recently developed and tested new tools that employ virtual reality (VR) to assess EFs (Climent et al., 2010; Rizzo et al., 2006). VR-based assessment tools allow children to interact dynamically with realistic scenarios and situations that reproduce aspects of their everyday life. The present study aimed to explore the correlation between the cognitive profile of gifted students, as measured by the WISC-IV paper-and-pencil test, and their attentional profile assessed through AULA, an innovative virtual reality-based attention assessment tool. A previous study by Diaz-Orueta et al. (2014) explored the correlation between WISC-IV indices in a sample of averaged cognitive ability learners diagnosed with ADHD, demonstrating the reliability of the VRbased attention assessment tool in discriminating clinical subjects. To our knowledge, no studies have yet been conducted analysing the attentional profile of gifted children using a VR-based attention assessment tool. Therefore, we expect to provide valuable new information on the peculiar attentional functioning of gifted children. [ABSTRACT FROM AUTHOR]

The limited use of the potential of inclusive education may be attributed to an insufficiently high level of its participants’ inclusive culture. The teacher’s inclusive culture as a collection of values, attitudes, beliefs, motives, and notions regarding inclusion and actualizing them in pedagogical practice remains underresearched. There has yet to be developed tools for assessing the level of development of the various components of inclusive culture and exploring the ways in which it develops in practicing teachers. Accordingly, a study was conducted to achieve the following objectives – to develop a set of relevant assessment tools, assess the level of development of the various components of inclusive culture in practicing teachers, and identify ways to develop it. The following methods were employed: summarization of Russian and foreign research on inclusive culture; development of tools for assessing inclusive culture; surveying teachers by questionnaire and testing them; development of practical recommendations based on the assessment results. The research results showed that it is important to not only promote an inclusive culture in society and in schools but nurture an inclusive culture in teachers and other participants in the educational process. It may be advisable to assess with a set of special tools the level of development of the following key components in the structure of teachers’ inclusive culture: motivation, emotional-axiological, cognitive, and conative. [ABSTRACT FROM AUTHOR]

This study reports an impact assessment of the basic Arduino programming training conducted among select Junior High School Students in a National High School in Cagayan De Oro City. The Focus Group Discussion sessions and survey questionnaires via goggle form complemented the content of the impact study table. The researchers aimed to determine the impact of the training towards the perception of the trainees in programming, skills development and learning gain. Furthermore, the training program used was evaluated by both trainers and trainees in terms of content, strategies and activities used. Research results showed the respondents' negative perception on Arduino programming yet manifested positive views on the training for programming skills development and enhancement of learning. The training material was rated high ranging from format to organization and content. [ABSTRACT FROM AUTHOR]

Students with mature number sense make sense of numbers and operations, use reasoning to notice patterns, and flexibly select the most effective and efficient problem-solving strategies (McIntosh et al., 1997; Yang, 2005). Despite being highlighted in national standards and policy documents (CCSS, 2010; NCTM, 2000), the association between students' mature number sense and other important outcomes is not well specified. For example, how does students' mature number sense relate to their grade-level mathematics achievement? We analyzed 153 upper elementary school students' scores on measures of mature number sense, fraction and decimal knowledge, multiplication fluency, and grade-level mathematics achievement. We found mature number sense to be measurably distinct from their fraction and decimal knowledge and uniquely associated with students' grade-level mathematics achievement. [ABSTRACT FROM AUTHOR]

We conducted a collective case study investigating two college algebra students' graph reasoning and selection on an online assessment. Students completed the assessment during individual, semi-structured interviews, as part of a broader validation study. The assessment contained six items; students selected Cartesian graphs to represent relationships between attributes in dynamic situations and explained their reasoning. Each student spontaneously wondered whether to select a piecewise-linear or nonlinear graph. Our qualitative analysis revealed that students' expectations about whether a graph "should" be linear or nonlinear impacted their graph selection. These expectations also influenced how they narrowed down between two graph choices that both represented the same gross covariation in attributes. We conclude with implications for course textbooks to promote students' covariational reasoning. [ABSTRACT FROM AUTHOR]

This study explores how dual-language teachers' decision-making was shaped by language policies, accountability policies, and standardized assessments. The study draws on interviews with third and fourth grade elementary teachers in California and Texas (N=17) to understand how they balanced accountability policies and assessments that worked in opposition to being able to provide mathematical learning opportunities within a dual language context. In particular, this study highlights how teachers contended with tensions when the language of instruction was different from the language of assessment. These findings raise the need for coherence across federal, state, and local policymaking for dual language programs to remain student-centered in the implementation of mathematics curriculum to meet the needs of all learners. [ABSTRACT FROM AUTHOR]

To base teaching on student thinking requires analyzing and interpreting students' thinking, key components of the construct of professional noticing (e.g., Jacobs et al., 2010). Although substantial research has been conducted using this construct, less attention has been paid to teachers' evaluations of student work based on these analyses and interpretations. In this theoretical report, I argue that evaluation of student thinking is a key prerequisite for effective instructional decision making for responsive teaching. I then present a synthesized framework of evaluation criteria and an elaborated process for effective and responsive evaluation of student written work using these criteria. Finally, I analyze excerpts from two interviews with elementary prospective teachers to demonstrate the utility of these products. [ABSTRACT FROM AUTHOR]

SimulaTE is studying teaching simulations as formative assessments of pre-service teachers' (PST) practice of eliciting and interpreting students' mathematical thinking. Preparation and protocols that promote reliability and validity of the simulations as formative assessments will enhance their effectiveness and generalizability. Teacher educators who use the simulations document each PST's performance to generate relevant feedback for the PST. As part of a coordinated set of validity studies, six researchers were prepared on the documentation protocol. Consistency of documentation within the group and with the simulation developers' judgments provided evidence supporting reliability and validity of the documentation protocol. [ABSTRACT FROM AUTHOR]

Assessment is a topic of concern to all stakeholders in our educational system. Pattern Based Questions are an assessment tool which is an alternative to the standardized assessment tool, and they are based on generative learning pedagogy, which shows promise in engaging all learners and usefulness in teaching and learning but validity has not yet been empirically established. Pattern-based questions seek to provide a qualitative expression of student understanding. It is the purpose of this research to empirically explore a correspondence between student response patterns and the students' expressed ways of thinking using a grounded theory approach and clinical interviews. Findings include rich descriptions of participants' ways of thinking about equivalent fractions as written in the PBQ. One response pattern was clearly differentiated from the others. Future research is discussed. [ABSTRACT FROM AUTHOR]

We previously (Gantt et al., 2023; Paoletti et al., 2021) identified items from the publicly released TIMSS 2011 assessments that had potential for students to employ covariational reasoning as a solution strategy. In this report, we explore the extent to which fourth-grade students' performance on such items in mathematics differed among 26 nations. Using multilevel modeling, we conclude that, in general, fourth-grade students were less successful on mathematics items for which covariational reasoning was a viable strategy than on items for which we could not identify a possible covariational reasoning strategy. However, three countries (Finland, Sweden, and the Netherlands) did not follow this pattern. [ABSTRACT FROM AUTHOR]

Depth-of-knowledge (DOK) is a means to communicate the cognitive demand of tasks and is often used to categorize assessment items. Webb's (2002) framework has been applied across content areas. The aim of this two-phase iterative study was to modify Webb's DOK framework for word problems. Through work with school partners, this iterative design-research based study provides supportive evidence for a modified DOK framework reflecting levels of complexity in word problems. The resulting modified DOK framework presents an opportunity for mathematics educators to reflect on various aspects of cognitive complexity. [ABSTRACT FROM AUTHOR]

This study conducted a comprehensive impact assessment of a Communicative Language Teaching (CLT) training program within a specific cohort of eighty-five English language education lecturers from private and public universities in Northern Mindanao. The participants were purposively chosen, ensuring a representative cross-section of private and public university lecturers from across the entirety of Mindanao. The research integrated theories on CLT and Task-Based Language Teaching (TBLT), focusing on meaningful tasks to facilitate language instruction and promote communication and interaction. Baseline information about the training needs of university teachers were gathered using five group discussions, interviews, and online surveys via Google Forms. Moreover, the study examined the effects of the CLT training on participants' perceptions of CLT approaches, interaction patterns, activity utilization, and learning outcomes. Results indicated a noteworthy shift in participants' initial negative perceptions of the communicative approach, with positive views emerging regarding its impact on interaction patterns, activity utilization, and learning enhancement. Evaluation of the training program by trainers and trainees reflected high ratings for format, organization, and content. These findings endorse the effectiveness of CLT in language instruction and its potential to foster learner engagement and proficiency development. This study provides empirical evidence supporting the positive impact of CLT training among university lecturers. It underscores the significance of incorporating communicative approaches in language teaching and recommends further research and professional development opportunities. [ABSTRACT FROM AUTHOR]

Before the COVID-19 pandemic, studying at German universities was traditionally dominated by inclass activities. During the pandemic, technical and organizational foundations were laid not only for teaching but also for exams. With the resumption of in-class events, the question is to what extent online exams will also be used in the future. Here, examiners and educational institutions can learn from the experience during the pandemic to increase the students' intention to participate in online exams in the future. To identify the problems encountered and derive lessons from them, a mixed-method approach was followed. For this purpose, 478 examinees were surveyed over 3 semesters using a quantitative questionnaire, followed by in-depth semi-structured follow-up interviews with 11 students. As a result, 5 recommendations for action could be derived, covering the topics of exam routine and familiarity, problem-solving, cheating, and teaching-examination arrangements. [ABSTRACT FROM AUTHOR]

The lack of understandability of AI-based decisions is increasingly posing trust-related and regulatory problems. This also applies to the educational sector, where AI is a central element of modern automated essay scoring (AES) systems. However, current research on explainable AI primarily focuses on complex technical approaches. These explanations usually show a lack of understandability by the actual users, who often have no knowledge of AI. Based on an experiment with 245 students at a German university, we were able to show that even the basic principles of user interface design can improve understandability and hereby trustworthiness. Thus, the use of visual elements promotes understandability even when only little information is provided. Especially when providing further AI-specific information on the scoring of AES systems, however, it must be considered that in combination with visual elements an information congruency can be observed, leading to a cognitive overload in the worst case. [ABSTRACT FROM AUTHOR]

People with intellectual disabilities often experience inequalities that affect the standard of their everyday lives. Assistive technologies can help alleviate some of these inequalities, yet abandonment rates remain high. This is in part due to a lack of involvement of all stakeholders in their design and evaluation, thus resulting in outputs that do not meet this cohort's complex and heterogeneous needs. The aim of this half-day workshop is to focus on community building in a field that is relatively thin and disjointed, thereby enabling researchers to share experiences on how to design for and with people with intellectual disabilities, provide internal support, and establish new collaborations. Workshop outcomes will help to fill a gap in the available guidelines on how to include people with intellectual disabilities in research, through more accessible protocols as well as personalised and better fit-for-purpose technologies. [ABSTRACT FROM AUTHOR]

Judicious use of energy sources assumes prime importance in attaining energy efficient behaviours and this also helps to achieve environmental protection. This study aims to develop the most conducive energy efficient and environmentfriendly behavioural assessment framework for household energy consumers in India. The multi-grade fuzzy and Importance Performance Analysis (IPA) are applied to develop this assessment framework. The case study has been conducted with residents of XYZ residents' association in India. The results obtained from the case-study Energy Efficiency Behavioural (EEB) Index is 6.6 which belongs to 'Energy Efficient'. The IPA analysis classified the weaker attributes of the case residents' association. The appropriate suggestions are provided to improve the weaker attributes. This framework will enable energy auditors, residents' association, and government to periodically evaluate the EEB. [ABSTRACT FROM AUTHOR]

The main goal of this research is to identify the impact of COVID-19 on online final exam scores among Computer Science students. The correlation matrix we used indicates the interrelationships among learning outcomes and student profile, type of classes and student online behaviour. Six courses were taken under consideration: Practical Algorithms, Discrete Mathematics, Software Engineering, Programming, Team Projects and Artificial Intelligence. A total of 4,988 final exam results were examined. After a deep analysis of the literature on the topic, we expected two scenarios. The first scenario constituted a decline in passing grades due to challenges such as: learning platform failures, poor internet connections or poorer quality of lessons due to teachers' lack of online competence. We hypothesized the second scenario as extraordinary student performance compared to their prior exams, but due to their dishonesty. The results of the study revealed that neither of the scenarios took place. It turned out that the challenges that seemed to be the most difficult ultimately did not matter. The present study finds that there is not a significant difference in the students' final exam performance between their online and traditional courses. Our strategy as described in this article has demonstrated a smooth transition from traditional to online teaching and assessment in terms of the final assessment. [ABSTRACT FROM AUTHOR]

Validity and validation is central to conducting high quality quantitative mathematics education scholarship. This presentation aims to support scholars engaged in quantitative research by providing information about the degrees to which validity evidence related to their instrument use or interpretation, were found in mathematics education scholarship. Findings have potential to steer future quantitatively focused scholarship and support equity aims. [ABSTRACT FROM AUTHOR]

Students exhibiting mature number sense make sense of numbers and operations, use reasoning to notice patterns, and flexibly select the most effective and efficient problem-solving strategies (McIntosh et al., 1997; Reys et al., 1999; Yang, 2005). Despite being highlighted in national standards and policy documents (CCSS, 2010; NCTM, 2000, 2014), students' mature number sense and its nomological network are not yet well specified. For example, how does students' mature number sense relate to their knowledge of fractions and their grade-level mathematics achievement? We analyzed 129 middle school students' scores on measures of mature number sense, fraction and decimal computation, and grade-level mathematics achievement. We found mature number sense to be measurably distinct from their fraction and decimal knowledge and uniquely associated with students' grade-level mathematics achievement. [ABSTRACT FROM AUTHOR]

With the adoption of the Common Core State Standards for Mathematics 12 years ago, the content of geometric transformations was shifted from high school to grade 8. In our research with middle grades teachers, they often discussed their difficulty in teaching geometric rotations. Therefore, we analyzed 444 middle grade students' responses, across four states, to eight rotation questions from the SMART assessment. The results corroborate teachers' challenges with teaching and student learning of rotations. Results indicate that students have a rigid understanding of angle measure that may impact their understanding of geometric rotations. Although angle measure is introduced in grade 4, we hypothesize that teachers need to provide additional opportunities for students to expand their meaning of angle measure. [ABSTRACT FROM AUTHOR]