Search

Abstract Positive effects of study progress on the economic knowledge of bachelor students are evident and were often referred to as proof of the validity of tests in the context of a longitudinal analysis. However, differences between domain-specific (e.g., semester) or discipline-specific (e.g., attended courses) indicators of study progress on students’ solving the economics tasks are relevant for both diagnostic and instructional purposes, but have hardly been researched to date. In an eye tracking study, we therefore calculated the average fixation duration during 900 task-solving processes of 35 economics teacher students solving 25 single-choice economics tasks. A multi-level mediation analysis shows that both indicators (i.e., semester and attended courses) were positively correlated with economics knowledge. At the same time, the impact of semesters and courses attended on their test performance was partially mediated by average fixation duration on the attractor and distractors, respectively. This suggests that visual attention differs during the task-solving process depending on the domain- or discipline-specific indicator of study progress.

Abstract To successfully learn using freely available (and non-curated) Internet resources, university students need to search for, critically evaluate and select online information, and verify sources (defined as Critical Online Reasoning, COR). Recent research indicates substantial deficits in COR skills among higher education students. To support students in learning how to critically use online information for their learning, it is necessary to better understand the strategies and practices that might elicit less critically-reflective judgments about online information and thus account for such deficits. To this end, using eye tracking data, we investigate how the COR behaviors of students who critically-reflectively evaluate the credibility of online information (‘high performers’) differ from those of students who do not critically-reflectively evaluate it (‘low performers’): 19 students were divided into high and low performers according to their performance in the newly developed Critical Online Reasoning Assessment (CORA). The fixation and dwell times of both groups during CORA task processing were compared regarding time spent on the different processing steps and eye movements on the visited web pages. The results show noticeable differences between the two groups, indicating that low performers indeed approached the task rather heuristically than systematically, and that COR skills require targeted and effective training in higher education.

Abstract In the present study, we recorded the eye movements of 20 criterion-based selected trainee teachers in economics while they responded to 25 single choice (SC) items in an economic content knowledge (CK) test and rated their confidence for each response in a digital assessment. By using a multilevel modeling approach with crossed random effects, we confirmed prior findings from eye-tracking research on SC tests, which showed longer dwell time on the correct response options (attractor) and shorter dwell time on the distractors are positively linked to correct options. Furthermore, we identified an additional effect on dwell time on the attractor in a moderator model with participants who highly rated their confidence for correct response options. Thus, we identified a specific role of students’ confidence in their CK on the gaze bias effect. We interpret these results in terms of students’ actual understanding of test contents from assessments of their professional knowledge and draw implications for further research and teacher education.

Quantitative reasoning is considered a crucial prerequisite for acquiring domain-specific expertise in higher education. To ascertain whether students are developing quantitative reasoning, validly assessing its development over the course of their studies is required. However, when measuring quantitative reasoning in an academic study program, it is often confounded with other skills. Following a situated approach, we focus on quantitative reasoning in the domain of business and economics and define domain-specific quantitative reasoning primarily as a skill and capacity that allows for reasoned thinking regarding numbers, arithmetic operations, graph analyses, and patterns in real-world business and economics tasks, leading to problem solving. As many studies demonstrate, well-established instruments for assessing business and economics knowledge like the Test of Understanding College Economics (TUCE) and the Examen General para el Egreso de la Licenciatura (EGEL) contain items that require domain-specific quantitative reasoning skills. In this study, we follow a new approach and assume that assessing business and economics knowledge offers the opportunity to extract domain-specific quantitative reasoning as the skill for handling quantitative data in domain-specific tasks. We present an approach where quantitative reasoning – embedded in existing measurements from TUCE and EGEL tasks – will be empirically extracted. Hereby, we reveal that items tapping domain-specific quantitative reasoning constitute an empirically separable factor within a Confirmatory Factor Analysis and that this factor (domain-specific quantitative reasoning) can be validly and reliably measured using existing knowledge assessments. This novel methodological approach, which is based on obtaining information on students’ quantitative reasoning skills using existing domain-specific tests, offers a practical alternative to broad test batteries for assessing students’ learning outcomes in higher education.

In recent decades, the acquisition of information has evolved substantially and fundamentally affects students’ use of information, so that the Internet has become one of the most important sources of information for learning. However, learning with freely accessible online resources also poses challenges, such as vast amounts of partially unstructured, untrustworthy, or biased information. To successfully learn by using the Internet, students therefore require specific skills for selecting, processing, and evaluating the online information, e.g., to distinguish trustworthy from distorted or biased information and for judging its relevance with regard to the topic and task at hand. Despite the central importance of these skills, their assessment in higher education is still an emerging field. In this paper, we present the newly defined theoretical-conceptual framework Critical Online Reasoning (COR). Based on this framework, a corresponding performance assessment, Critical Online Reasoning Assessment (CORA), was newly developed and underwent first steps of validation in accordance with the Standards for Educational and Psychological Testing. We first provide an overview of the previous validation results and then expand them by including further analyses of the validity aspects “internal test structure” and “relations with other variables”. To investigate the internal test structure, we conducted variance component analyses based on the generalizability theory with a sample of 125 students and investigated the relations with other variables by means of correlation analyses. The results show correlations with external criteria as expected and confirm that the CORA scores reflect the different test performances of the participants and are not significantly biased by modalities of the assessment. With these new analyses, this study substantially contributes to previous research by providing comprehensive evidence for the validity of this new performance assessment that validly assesses the complex multifaceted construct of critical online reasoning among university students and graduates. CORA results provide unique insights into the interplay between features of online information acquisition and processing, learning environments, and the cognitive and metacognitive requirements for critically reasoning from online information in university students and young professionals.

Digital teaching interventions allow for tailor-made university teaching. This is especially relevant for teacher education, where applicable professional teaching knowledge needs to be promoted for later professional success. Digital teaching tools have been shown to be a promising supplement for this purpose. Even though the corresponding demands in teacher education have been increasing in recent years, the need to develop digital learning tools usable in instruction is still urgent. The TWIND project develops digital learning tools for teacher education and evaluates them in a quasi-experimental design. The present work investigates the usability and application of these newly developed tools. Sixty-three trainee teachers worked independently over four weeks with one of two digital learning tools, focusing on either ‘Multilingualism in Classrooms’ or ‘Professional Communication in Classrooms.’ This study includes a pre-post-test of pedagogical knowledge facets as well as student and instructor ratings on the digital tools. The digital learning tools led to a positive change in the respective target facets of pedagogical knowledge. The student and instructor feedback reflected positively on the usability and usefulness of the new digital tools. Based on these findings, the limitations of the study as well as implications for further research and teacher education practice have been outlined.

Research FocusThe promotion of domain-specific knowledge is a central goal of higher education and, in the field of medicine, it is particularly essential to promote global health. Domain-specific knowledge on its own is not exhaustive; confidence regarding the factual truth of this knowledge content is also required. An increase in both knowledge and confidence is considered a necessary prerequisite for making professional decisions in the clinical context. Especially the knowledge of human physiology is fundamental and simultaneously critical to medical decision-making. However, numerous studies have shown difficulties in understanding and misconceptions in this area of knowledge. Therefore, we investigate (i) how preclinical medical students acquire knowledge in physiology over the course of their studies and simultaneously gain confidence in the correctness of this knowledge as well as (ii) the interrelations between these variables, and (iii) how they affect the development of domain-specific knowledge.MethodIn a pre–post study, 169 medical students’ development of physiology knowledge and their confidence related to this knowledge were assessed via paper-pencil questionnaires before and after attending physiology seminars for one semester. Data from a longitudinal sample of n = 97 students were analyzed using mean comparisons, regression analyses, and latent class analyses (LCAs). In addition, four types of item responses were formed based on confidence and correctness in the knowledge test.ResultsWe found a significant and large increase in the students’ physiology knowledge, with task-related confidence being the strongest predictor (apart from learning motivation). Moreover, a significantly higher level of confidence at t2 was confirmed, with the level of prior confidence being a strong predictor (apart from knowledge at t2). Furthermore, based on the students’ development of knowledge and confidence levels between measurement points, three empirically distinct groups were distinguished: knowledge gainers, confidence gainers, and overall gainers. The students whose confidence in incorrect knowledge increased constituted one particularly striking group. Therefore, the training of both knowledge and the ability to critically reflect on one’s knowledge and skills as well as an assessment of their development in education is required, especially in professions such as medicine, where knowledge-based decisions made with confidence are of vital importance.

Studieneingangsvoraussetzungen, wie das Vorwissen, gelten als wesentliche Prädiktoren für den Studienerfolg. Studienanfänger/innen in den Wirtschaftswissenschaften zeichnen sich durch eine große Heterogenität zu Studienbeginn aus. In einer Längsschnittstudie wird betrachtet, inwiefern die heterogenen Studieneingangsvoraussetzungen den Studienerfolg erklären und wie dieser Zusammenhang unter Berücksichtigung der studienprozessbezogenen Einflussfaktoren sich verändert. Mittels Mehrebenenmodellen wird der Einfluss voruniversitärer und studienprozessbezogener Einflussfaktoren auf die Studiennote am Ende des Bachelorstudiums differenziert betrachtet. Implikationen für die Hochschulpraxis und weitere Forschung werden kritisch diskutiert.

Context: The transfer of vocational education and training (VET) has a long tradition and can be based on various conceptual and methodical approaches. Transfer process and success can be influenced by numerous factors at different levels (systemic, institutional and individual). However, the existing research on the challenges and success factors of VET transfer is very heterogenous and fragmented. Method: To provide a comprehensive and structured overview of the current state of the international research on transfer in the context of VET, we conducted a literature review, which is presented here. Using the method of systematic literature review, a total of 231 studies were found and 41 studies were selected based on the defined criteria for full text analysis. Findings: Various specific research foci and analytical approaches used on the included studies were identified and documented. The transfer of a dual VET system or its elements is based on different approaches and perspectives of the recipient country and the transferee. Major challenges for the transfer of dual VET systems include the social reputation of VET in the recipient countries and (language- and culture-related) communication difficulties between the involved parties. For a successful transfer of VET, a deeper knowledge of the contextual conditions in the recipient country are of particular importance, as well as a close, longstanding cooperation between the involved parties. Conclusion: The generated knowledge about key contextual conditions at the individual, company and systemic level in the context of VET transfer can provide a valuable basis for future VET cooperation.

To teach effectively, teachers need subject-specific knowledge, such as content knowledge and pedagogical content knowledge, but also an ability to apply that knowledge to master demanding classroom situations. However, there is no consensus in research whether this ability should be modeled as a subject-specific ability or as a generic ability. This question is important for effective teacher training and especially for out-of-field teaching. In this exploratory study, we investigate the subject-specificity of the ability to apply subject-specific knowledge with German secondary pre-service teachers who are equally trained to teach mathematics and economics. We administered paper-pencil tests for subject-specific knowledge in both subjects to 37 pre-service teachers. In addition, video vignettes of instructional situations were used to elicit their ability to apply that knowledge. N = 6 cases showed apt subject-specific knowledge in both subjects to be analyzed regarding knowledge application. Based on a qualitative analysis of 93 responses to the video vignettes, teachers’ ability to apply that knowledge was examined. Our findings indicate systematic qualitative differences in the pre-service teachers’ responses in mathematics and economics. The results favor a subject-specific conceptualization of teachers’ ability to apply subject-specific knowledge in instructional settings. This implies for teacher training that learning opportunities for promoting teachers’ ability to apply their subject-specific knowledge in instructional settings should be designed specifically for the subject that will be taught. Our study also suggests that out-of-field teachers require training in both knowledge and an ability to apply this knowledge in teaching another subject, as their ability to apply knowledge may not transfer from their field of expertise.

The seven articles in this special issue represent a wide range of international comparative and review studies by international research teams from China, Germany, India, Russia, Switzerland and Mexico. The presented projects are part of the national program "Research on the Internationalisation of Vocational Education and Training", funded by the German Federal Ministry of Education and Research (BMBF). An adapted version of Urie Bronfenbrenner’s ecological systems theory forms the conceptual framework of the special issue. The four system levels (micro, meso, exo and macro) are addressed by one article each. The article on the microsystem level focuses on the intended and implemented curricula in a cross-country comparison of China and Russia. The article on the mesosystem level aims at the development of a quality management model for vocational education and training (VET) institutions in India. At the exolevel, the regional structures of the education and employment systems in Mexico, particularly the cooperation between schools and companies in the hotel industry, are investigated. At the macrosystem level, the social representation of non-academic labour in Mexico is examined in terms of cultural artefacts. Furthermore, three overarching review studies systematise relevant research developments and approaches. The topics of the three review studies are European VET policy, transfer of VET and VET research. The scope ranges from the development of a comparative research tool to a summary analysis of over 5,000 individual publications. Given the broad scope and heterogeneity of the findings, a summative conclusion would hardly be appropriate. Nevertheless, with regard to the model of the ‘triadic conception of purposes in comparative VET research’ that represents a heuristic for describing the purposes of international VET research, we conclude with an emphasis on a need of more criticality. In this context, one finding can be pointed out as an example: One review study found that most studies (here, with reference to VET transfer) refer to the recipient country without a comparative perspective. Thus, there is a clear demand for more comparative research following a critical-reflective approach.

Accessing higher education without having to overcome bureaucratic hurdles is a serious concern for refugees. Although empirical studies on the integration and success of refugees in higher education are scarce, the challenges related to this issue are becoming apparent. The Success and Opportunities for Refugees in Higher Education (SUCCESS) research project has been launched to investigate the effectiveness of new online study programs offered on the Kiron Open Higher Education (Kiron) platform that provides refugees with access to Massive Open Online Courses (MOOCs). SUCCESS measures the prior knowledge and skills of refugee students and investigates to what extent their study opportunities, learning processes, and chances of academic success can be improved effectively through different forms of support provided in Kiron. In this paper, we present the assessment framework and study design of the SUCCESS project as well as data on 1,376 students entering the study program in Kiron in summer 2017. As students’ language skills, intellectual abilities, and prior study-related knowledge play a significant role in their performance in higher education degree programs, we focus on the crucial introductory study phase and valid diagnostics of students’ study preconditions. We analyze refugee students’ socio-biographical and educational data such as gender, country of origin, highest level of education achieved etc. and examine their English language skills, intellectual abilities, and previous study domain related knowledge. We find extreme differences in levels of education and preconditions on starting to study in Kiron. Based on these results, we discuss implications for the effective and successful integration of refugee students in higher education. In this paper, we present the evidence-based model, assessment framework, and study design of the XXXXX project, as well as data on 1,376 students entering a study program through Kiron in summer 2017. Because students’ language skills, intellectual abilities, and prior study-related knowledge play a significant role in their performance in higher education degree programs, we focus on the crucial introductory study phase and valid diagnostics of students’ study preconditions. We analyze refugee students’ socio-biographical and educational data such as gender, country of origin, highest level of education achieved etc. and examine their English language skills, intellectual abilities, and previous study domain related knowledge. We find extreme differences in levels of education and preconditions on starting to study through Kiron. Based on these results, we discuss implications for the effective and successful integration of refugee students in higher education.

Critical evaluation skills when using online information are considered important in many research and education frameworks; critical thinking and information literacy are cited as key twenty-first century skills for students. Higher education may play a special role in promoting students' skills in critically evaluating (online) sources. Today, higher education students are more likely to use the Internet instead of offline sources such as textbooks when studying for exams. However, far from being a value-neutral, curated learning environment, the Internet poses various challenges, including a large amount of incomplete, contradictory, erroneous, and biased information. With low barriers to online publication, the responsibility to access, select, process, and use suitable relevant and trustworthy information rests with the (self-directed) learner. Despite the central importance of critically evaluating online information, its assessment in higher education is still an emerging field. In this paper, we present a newly developed theoretical-conceptual framework for Critical Online Reasoning (COR), situated in relation to prior approaches (“information problem-solving,” “multiple-source comprehension,” “web credibility,” “informal argumentation,” “critical thinking”), along with an evidence-centered assessment framework and its preliminary validation. In 2016, the Stanford History Education Group developed and validated the assessment of Civic Online Reasoning for the United States. At the college level, this assessment holistically measures students' web searches and evaluation of online information using open Internet searches and real websites. Our initial adaptation and validation indicated a need to further develop the construct and assessment framework for evaluating higher education students in Germany across disciplines over their course of studies. Based on our literature review and prior analyses, we classified COR abilities into three uniquely combined facets: (i) online information acquisition, (ii) critical information evaluation, and (iii) reasoning based on evidence, argumentation, and synthesis. We modeled COR ability from a behavior, content, process, and development perspective, specifying scoring rubrics in an evidence-centered design. Preliminary validation results from expert interviews and content analysis indicated that the assessment covers typical online media and challenges for higher education students in Germany and contains cues to tap modeled COR abilities. We close with a discussion of ongoing research and potentials for future development.

To successfully learn using open Internet resources, students must be able to critically search, evaluate and select online information, and verify sources. Defined as critical online reasoning (COR), this construct is operationalized on two levels in our study: (1) the student level using the newly developed Critical Online Reasoning Assessment (CORA), and (2) the online information processing level using event log data, including gaze durations and fixations. The written responses of 32 students for one CORA task were scored by three independent raters. The resulting score was operationalized as “task performance,” whereas the gaze fixations and durations were defined as indicators of “process performance.” Following a person-oriented approach, we conducted a process mining (PM) analysis, as well as a latent class analysis (LCA) to test whether—following the dual-process theory—the undergraduates could be distinguished into two groups based on both their process and task performance. Using PM, the process performance of all 32 students was visualized and compared, indicating two distinct response process patterns. One group of students (11), defined as “strategic information processers,” processed online information more comprehensively, as well as more efficiently, which was also reflected in their higher task scores. In contrast, the distributions of the process performance variables for the other group (21), defined as “avoidance information processers,” indicated a poorer process performance, which was also reflected in their lower task scores. In the LCA, where two student groups were empirically distinguished by combining the process performance indicators and the task score as a joint discriminant criterion, we confirmed these two COR profiles, which were reflected in high vs. low process and task performances. The estimated parameters indicated that high-performing students were significantly more efficient at conducting strategic information processing, as reflected in their higher process performance. These findings are so far based on quantitative analyses using event log data. To enable a more differentiated analysis of students’ visual attention dynamics, more in-depth qualitative research of the identified student profiles in terms of COR will be required.

The digital and information age has fundamentally transformed the way in which students learn and the study material they have at their disposal, especially in higher education. Students need to possess a number of higher-order cognitive and metacognitive skills, including effective information processing and critical reasoning to be able to navigate the Internet and use online sources, even those found outside of academically curated domains and in the depths of the Internet, and to solve (domain-specific) problems. Linking qualitative and quantitative research and connecting the humanities to empirical educational science studies, this article investigates the role of narratives and their impact on university students’ information seeking and their critical online reasoning (COR). This study focuses on the link between students’ online navigation skills, information seeking behavior and critical reasoning with regard to the specific domains: economics and medicine. For the empirical analysis in this article, we draw on a study that assesses the COR skills of undergraduate students of economics and medicine at two German universities. To measure COR skills, we used five tasks from the computer-based assessment “Critical Online Reasoning Assessment” (CORA), which assesses students’ skills in critically evaluating online sources and reasoning using evidence on contentious issues. The conceptual framework of this study is based on an existing methodology – narrative economics and medicine – and discusses its instructional potential and how it can be used to develop a new tool of “wise interventions” to enhance students’ COR in higher education. Based on qualitative content analyses of the students’ written responses, i.e., short essays, three distinct patterns of information seeking behavior among students have been identified. These three patterns – “Unambiguous Fact-Checking,” “Perspective-Taking Without Fact-Checking,” and “Web Credibility-Evaluating” – differ substantially in their potential connection to underlying narratives of information used by students to solve the CORA tasks. This analysis suggests that training university students in narrative analysis can strongly contribute to enhancing their critical online reasoning.

The Internet has become one of the main sources of information for university students’ learning. Since anyone can disseminate content online, however, the Internet is full of irrelevant, biased, or even false information. Thus, students’ ability to use online information in a critical-reflective manner is of crucial importance. In our study, we used a framework for the assessment of students’ critical online reasoning (COR) to measure university students’ ability to critically use information from online sources and to reason on contentious issues based on online information. In addition to analyzing students’ COR by evaluating their open-ended short answers, we also investigated the students’ web search behavior and the quality of the websites they visited and used during this assessment. We analyzed both the number and type of websites as well as the quality of the information these websites provide. Finally, we investigated to what extent students’ web search behavior as well as the quality of the used website contents are related to higher task performance. To investigate this question, we used five computer-based performance tasks and asked 160 students from two German universities to perform a time-restricted open web search to respond to the open-ended questions presented in the tasks. The written responses were evaluated by two independent human raters. To analyze the students’ browsing history, we developed a coding manual and conducted a quantitative content analysis for a subsample of 50 students. The number of visited webpages per participant per task ranged from 1 to 9. Concerning the type of website, the participants relied especially on established news sites and Wikipedia. For instance, we found that the number of visited websites and the critical discussion of sources provided on the websites correlated positively with students’ scores. The identified relationships between students’ web search behavior, their performance in the CORA tasks, and the qualitative website characteristics are presented and critically discussed in terms of limitations of this study and implications for further research.

Enhancing students’ critical thinking (CT) skills is an essential goal of higher education. This article presents a systematic approach to conceptualizing and measuring CT. CT generally comprises the following mental processes: identifying, evaluating, and analyzing a problem; interpreting information; synthesizing evidence; and reporting a conclusion. We further posit that CT also involves dealing with dilemmas involving ambiguity or conflicts among principles and contradictory information. We argue that performance assessment provides the most realistic—and most credible—approach to measuring CT. From this conceptualization and construct definition, we describe one possible framework for building performance assessments of CT with attention to extended performance tasks within the assessment system. The framework is a product of an ongoing, collaborative effort, the International Performance Assessment of Learning (iPAL). The framework comprises four main aspects: (1) The storyline describes a carefully curated version of a complex, real-world situation. (2) The challenge frames the task to be accomplished (3). A portfolio of documents in a range of formats is drawn from multiple sources chosen to have specific characteristics. (4) The scoring rubric comprises a set of scales each linked to a facet of the construct. We discuss a number of use cases, as well as the challenges that arise with the use and valid interpretation of performance assessments. The final section presents elements of the iPAL research program that involve various refinements and extensions of the assessment framework, a number of empirical studies, along with linkages to current work in online reading and information processing.

Critical reasoning (CR) when confronted with contradictory information from multiple sources is a crucial ability in a knowledge-based society and digital world. Using information without critically reflecting on the content and its quality may lead to the acceptance of information based on unwarranted claims. Previous personal beliefs are assumed to play a decisive role when it comes to critically differentiating between assertions and claims and warranted knowledge and facts. The role of generic epistemic beliefs on critical stance and attitude in reflectively dealing with information is well researched. Relatively few studies however, have been conducted on the influence of domain-specific beliefs, i.e., beliefs in relation to specific content encountered in a piece of information or task, on the reasoning process, and on how these beliefs may affect decision-making processes. This study focuses on students’ task- and topic-related beliefs that may influence their reasoning when dealing with multiple and partly contradictory sources of information. To validly assess CR among university students, we used a newly developed computer-based performance assessment in which the students were confronted with an authentic task which contains theoretically defined psychological stimuli for measuring CR. To investigate the particular role of task- and topic-related beliefs on CR, a purposeful sample of 30 university students took part in a performance assessment and then were interviewed immediately afterward. In the semi-structured cognitive interviews, the participants’ task-related beliefs were assessed. Based on qualitative analyses of the interview transcripts, three distinct profiles of decision-making among students have been identified. More specifically, the different types of students’ beliefs and attitudes derived from the cognitive interview data suggest their influence on information processing, reasoning approaches and decision-making. The results indicated that the students’ beliefs had an influence on their selection, critical evaluation and use of information as well as on their reasoning processes and final decisions.

To validly assess teachers’ pedagogical content knowledge (PCK), performance-based tasks with open-response formats are required. Automated scoring is considered an appropriate approach to reduce the resource-intensity of human scoring and to achieve more consistent scoring results than human raters. The focus is on the comparability of human and automated scoring of PCK for economics teachers. The answers of (prospective) teachers (N = 852) to six open-response tasks from a standardized and validated test were scored by two trained human raters and the engine “Educational SCoRIng Toolkit” (ESCRITO). The average agreement between human and computer ratings, κw = 0.66, suggests a convergent validity of the scoring results. The results of the single-sector variance analysis show a significant influence of the answers for each homogeneous subgroup (students = 460, trainees = 230, in-service teachers = 162) on the automated scoring. Findings are discussed in terms of implications for the use of automated scoring in educational assessment and its potentials and limitations.

Domain-specific understanding of digitally represented graphs is necessary for successful learning within and across domains in higher education. Two recent studies conducted a cross-sectional analysis of graph understanding in different contexts (physics and finance), task concepts, and question types among students of physics, psychology, and economics. However, neither changes in graph processing nor changes in test scores over the course of one semester have been sufficiently researched so far. This eye-tracking replication study with a pretest–posttest design examines and contrasts changes in physics and economics students’ understanding of linear physics and finance graphs. It analyzes the relations between changes in students’ gaze behavior regarding relevant graph areas, scores, and self-reported task-related confidence. The results indicate domain-specific, context- and concept-related differences in the development of graph understanding over the first semester, as well as its successful transferability across the different contexts and concepts. Specifically, we discovered a tendency of physics students to develop a task-independent overconfidence in the graph understanding during the first semester.

Abstract Background When beginning higher education studies in business and economics students bring with them diverse knowledge and experience in the field, which could affect their success in the program. Differences in prior economic education pose challenges to the teachers in higher education as they have to decide which knowledge of economics the students have. This is important for preparing the lessons in higher education. In this paper, we investigate how prior economic education influences beginning university students’ knowledge of economics. Methods We administered items from the German adaptations of the test of economic literacy and the test of understanding college economics, both developed by the national council on economic education, to assess the general knowledge of economics and specific knowledge of micro- and macroeconomics of 511 beginning students at two universities in Germany. Results Participants who had completed vocational training or a course in economics as a major subject at secondary school (“Leistungskurs Wirtschaft”) performed significantly better on items relating to general economics and macroeconomics; however, there was no significant difference in performance on items relating to microeconomics between the comparison groups. Attendance of an upper secondary school specializing in economics (“Wirtschaftsgymnasium”) has no significant effect when controlling for these two learning opportunities and other personal characteristics. In addition, we performed regression-analytic modelling to examine the correlation between economic knowledge and personal factors such as gender, grade on university entrance examination, and migration background and found effects depended on the economic content area assessed. Conclusions The study demonstrates that there is a heterogeneity in students’ knowledge of economics when beginning higher education studies in business and economics. The results of this research highlight the importance of designing target-oriented teaching methods that take into consideration the study-relevant (prior) knowledge of beginning students.

Learning to solve graph tasks is one of the key prerequisites of acquiring domain-specific knowledge in most study domains. Analyses of graph understanding often use eye-tracking and focus on analyzing how much time students spend gazing at particular areas of a graph—Areas of Interest (AOIs). To gain a deeper insight into students’ task-solving process, we argue that the gaze shifts between students’ fixations on different AOIs (so-termed transitions) also need to be included in holistic analyses of graph understanding that consider the importance of transitions for the task-solving process. Thus, we introduced Epistemic Network Analysis (ENA) as a novel approach to analyze eye-tracking data of 23 university students who solved eight multiple-choice graph tasks in physics and economics. ENA is a method for quantifying, visualizing, and interpreting network data allowing a weighted analysis of the gaze patterns of both correct and incorrect graph task solvers considering the interrelations between fixations and transitions. After an analysis of the differences in the number of fixations and the number of single transitions between correct and incorrect solvers, we conducted an ENA for each task. We demonstrate that an isolated analysis of fixations and transitions provides only a limited insight into graph solving behavior. In contrast, ENA identifies differences between the gaze patterns of students who solved the graph tasks correctly and incorrectly across the multiple graph tasks. For instance, incorrect solvers shifted their gaze from the graph to the x-axis and from the question to the graph comparatively more often than correct solvers. The results indicate that incorrect solvers often have problems transferring textual information into graphical information and rely more on partly irrelevant parts of a graph. Finally, we discuss how the findings can be used to design experimental studies and for innovative instructional procedures in higher education.

The aim of the 15 collaborative projects conducted during the new funding phase of the German research program Modeling and Measuring Competencies in Higher Education—Validation and Methodological Innovations (KoKoHs) is to make a significant contribution to advancing the field of modeling and valid measurement of competencies acquired in higher education. The KoKoHs research teams assess generic competencies and domain-specific competencies in teacher education, social and economic sciences, and medicine based on findings from and using competency models and assessment instruments developed during the first KoKoHs funding phase. Further, they enhance, validate, and test measurement approaches for use in higher education in Germany. Results and findings are transferred at various levels to national and international research, higher education practice, and education policy.

Hochschulrankings sind in den letzten Jahren auch in Deutschland gerade in den Wirtschaftswissenschaften sehr populär und damit zunehmend zum Gegenstand der Forschung geworden. Allerdings erweisen sich die Erkenntnisse sowohl zur generellen Bedeutung von Rankings bei der Hochschulwahl als auch zur Wirkung der Rankings in Abhängigkeit bestimmter personenbezogener Eigenschaften der Studieninteressierten bislang als begrenzt. An diesem Forschungsdefizit setzt der vorliegende Beitrag an, indem zunächst die Wirkungsweise von Rankings theoretisch betrachtet wird, bevor Befragungsergebnisse von 1.314 Studienanfängerinnen und -anfängern der Wirtschaftswissenschaften präsentiert werden. 19.12.2014 | Roland Happ, Manuel Förster, Olga Zlatkin-Troitschanskaia (Mainz), Michael-Jörg Oesterle & Stefan Doetsch (Stuttgart)

Quality development in higher education (HE) relies on the assessment, evaluation, and optimisation of university teaching. Only a few studies have evaluated the new bachelor (BA) and master (MA) degree courses in Germany based on an objective measurement of key performance indicators, such as students' knowledge acquisition. To address this research deficit, students' content knowledge of business and economic (b&e) was assessed in a longitudinal study. Results of students in the BA/MA study model were compared to those of students in the former Diplom study model while many potential structural and personal influence factors were controlled. This paper presents in detail the findings and discusses their implications for quality assurance and performance assessment in HE. 21.03.2014 | Roland Happ & Olga Zlatkin-Troitschanskaia (Mainz)