Search

The aim of this half-day face-to-face workshop, organized in cooperation of three universities, is twofold. First, it will provide a platform for sharing of experiences, research and challenges related to the link between learning analytics (LA) and learning design (LD). Second, the workshop will enable participants to engage with an innovative, free-to-use LD tool (learning-design.eu), and create advanced learning analytics on LD using the tool. Participants will be invited to work collaboratively on LD of their own courses, reflect on the LA generated by the LD tool and improve the course LD accordingly. This will also contribute to the further development of the concept and tool, based on a pre-established research protocol. Participants will take away recommendations for improvement of their own courses, as well as know-how on how to use an innovative LD tool at their own institutions. Ahead of the workshop, if interested, participants will be invited to apply for a short presentation (5-10 minutes). They will also be asked to consider their courses and particular learning outcome(s) which could be redesigned at the workshop.

This paper presents a study conducted within an international project to investigate the experiences of higher education (HE) teachers with the shift to digital teaching during the COVID-19 pandemic. The study was based on a survey conducted among HE teachers (n = 167) in autumn 2021 in four European countries: Spain, Portugal, Finland and Croatia. Analysis of Variance (ANOVA) was used to investigate the differences in responses with respect to the country and the study field, with more significant differences found among the countries than among the study fields. The results will contribute to a MOOC for teachers’ professional development.

Technology can support teaching and learning processes in many ways. In particular, e-assessment can provide a meaningful upgrade to standard approaches. We will present a model of a student perspective on e-assessment in mathematics in higher education. The focus of our presentation will be on e-assessment supported by a Learning Management System (LMS). Within our mathematics courses paper-based assessments were replaced with computer-based ones. E-assessment questions were individualized, taken from a systematized online question bank, in which questions are organized according to a taxonomy of learning outcomes. E-assessments were conducted in a controlled online environment. We collected 631 students' responses to questionnaires on three occasions, in two different mathematics courses during one academic year. The research included two exploratory factor analyses and a confirmatory factor analysis. It yielded a model of a student perspective on e-assessment, consisting of four factors: Transparency and fairness of assessment, Formative and summative assessment and feedback, Meaningful use of technology in assessment and The difficulty of learning outcomes.

In this paper we present a systematic literature review of studies related to digital maturity of schools. The focus is on the identification of the most significant papers from the field of digital maturity of schools, essential elements of digital maturity frameworks and their validation, as well as the connection with other sectors related to the school. The analysis shows that ICT infrastructure is an important but not sufficient element for digital transformation. Digital transformation is also influenced by digital competences of schools’ employees, use of ICT in teaching and learning and administration, and the strategic approach to digital transformation.

We present a comprehensive learning design (LD) concept and tool, motivated by the needs identified by higher education (HE) practitioners. The concept and tool aim at implementing contemporary research findings and theory to support balanced LD planning (BDP). The student-centered BDP concept and tool provide innovation to LD planning by strongly focusing on learning outcomes (LOs) and student workload, aligning study program and course level LOs, ensuring constructive alignment and assessment validity, enhancing LD by using learning analytics, and enabling flexible use in different contexts and pedagogical approaches. The ongoing work has been done according to design science methodology, with positive first feedback from HE practitioners. We identify areas for further research and improvement, including testing the BDP tool in real-world HE contexts and its integration with learning management systems. This could help close the gap between intended (often innovative) LDs and their implementation in real teaching and learning environments.

Assessment is one of the central components of a curriculum and guides students' learning. E- assessment involves the use of ICT to present and deliver assessment tasks, receive and record students' answers, as well as to provide feedback to students. E-assessment has been used considerably more in many higher education institutions since the onset of the COVID-19 pandemic and therefore our aim is to explore the specifities of e-assessment in mathematics. We will present the benefits and downsides of e- assessment in mathematics in higher education. Our study focused on students' perspectives on e- assessment supported by a Learning Management System (LMS) in mathematics courses. It incorporated 631 students' responses to questionnaires conducted three times and it lasted for one academic year. We used descriptive statistics for close-ended questions and qualitative analysis for open-ended questions. The students' perspectives on e- assessment were generally positive. The students appreciated the student-centered approach based on sound pedagogical alignment of all teaching and learning elements with assessment. However, they found it important that technology should not distract them during assessment tasks. Finally, they did not find that there was more cheating and plagiarism in e- assessment in comparison to on-campus face-to-face assessments.

The assessment should be constructed and analysed holistically. The role of assessment should be critically examined in acquisition of knowledge, skills and attitudes. We postulate two aspects: 1. The complete assessment programme with carefully prepared assessment set should be considered. 2. Assessment programme should be evaluated according to an overarching framework. According to (Van der Vleuten & Schuwirth, 2005) the utility framework for assessment depends on 5 factors: Validity (the most frequent quality criterion), Reliability (the accuracy of pass and fail decisions), Educational impact, Acceptability and Cost of assessment. Our aim is to use learning analytics to support evaluation of assessment set according to the utility framework.

Due to the COVID{;19 pandemic the educational institutions around the world have been temporarily closed by their governments which resulted in rapid transition to remote learning and online assessment. We critically examine it's role in acquisition of knowledge, skills and attitudes. We advocate construction of the complete assessment programme with carefully prepared assessment set, but also its evaluation according to an overarching framework. For that purpose we have chosen the so-called utility framework introduced by Van der Vleuten & Schurwith, 2005. We will present the utility framework which consists of five elements (validity, reliability, educational impact, acceptability and costs). The focus will be on the reliability which we broadly understand as the accuracy of pass and fail decisions. More precisely, we will introduce and analyse several approaches for construction of the composite index for the assessment programme on the course level as a possible measure of the reliability. To do that, we use data available in the learning management system for past two academic years.

This invited lecture covers a choice of research within the field of learning analytics, as well as the applications in educational systems which includes the strategic decision-making (national) but also the analysis of the examples at the study programme and course level. I propose some research questions for future research.

TEM Journal; Vol 6, No 4, 2017. ISSN 2217-8309, The paper describes and interprets differences among social network structures in the private sector, politics (local administration) and non-governmental organisations (NGO) in Croatia. Social network analysis (SNA) methodology was used to examine networks in three sectors, which were found to differ in terms of centrality measures and substructures. In the private sector network, centrality values of network members were related to their respective positions in the organisational hierarchy. In the politics network, centrality measures values of members were influenced both by hierarchy and by the number of same-party members. In the NGO network, all members had high centrality values. Substructure sizes were low in the private sector and high in the NGO sector; in the politics network, they were dependent on the number of same-party members.

Strategic planning using the balanced scorecard (BSC) is a usual activity in many higher education institutions (HEIs). In BSC, goals are deployed into BSC perspectives. In non- profit HEIs, those perspectives are (from bottom to top): the finance perspective, the learning and growth perspective, the internal processes perspective, the costumers perspective and the mission perspective. In terms of profit HEIs (ex. private universities), the finance perspective is at the top of the map (instead of the mission perspective). In the literature, we can also find totally new BSC perspectives that are adjusted to the particular HEI’s needs. Nevertheless, when HEI’s administration selects a specific strategic map design, it places strategic goals into the map and connects them with respect to the existence of the influences between goals. In theory, a goal from a particular perspective can influence on any other goal from the same perspective, as well as to any goal from any perspective that is above its perspective. After the strategic map of goals is created, there is often a request to prioritize strategic goals in the strategic map in to determine the most crucial goals. In this paper, we present several possible ways to prioritize BSC goals. Three of them are the direct applications of three methods: the analytic hierarchy process (AHP), the analytic network process (ANP) and the PageRank centrality. Additionally, we propose an integrated approach. The main disadvantage of applying the AHP is that influences between goals are not included in goals’ priorities calculations. The main disadvantage of the ANP is that many BSC goals will weigh 0.0 because the graph that is associated with the BSC strategic map of goals is almost always reducible. This issue can be solved by applying the PageRank centrality. The procedure of this method ensures that the original BSC map will become irreducible. However, applying the ANP or PageRank does only consider the influences between goals in BSC map. It does not include the importance of each goal for the HEI into consideration. To ensure both, we proposed and applied the integrated approach which combines the AHP with PageRank centrality. In this paper, all these concepts are applied and demonstrated on the case of the BSC strategic map of goals of the Faculty of organization and informatics, University of Zagreb and can be applied for other HEIs’ strategic maps of goals.

Strategic decision making about education and open and distance education (ODE) issues requires analytic approach supported by a suitable method. The Analytic Network Process (ANP) meets all the requirements of decision making in higher education (HE), but it is very rarely used in practice because of its weaknesses. The weaknesses are related to the complexity of the method and demand on resources in the process of implementation. An upgraded ANP method that combines the ANP with the Social Network Analysis (SNA), which diminishes some of the weaknesses of the original ANP, is presented in this paper.

Rapid diffusion of Information and Communication Technologies (ICT) in all professional and personal areas require digital maturity from schools. In order to assess this aspect of school, a concept of digitally mature school has been developed and translated into different frameworks. In this paper, we describe development methodology of Framework for Digitally Mature Schools (FDMS) in Croatia. The FDMS, together with the accompanying instrument and software, represents a unique and comprehensive tool set for the assessment of digital maturity of a school. The FDMS recognized five areas divided into 38 elements that are described on five digital maturity levels in the form of a rubric.

Digital technologies in educational institutions have the potential to be one of the main catalysts of quality education. The purpose of this paper is to analyse digital maturity frameworks, to describe the irdevelopment methodologies and to propose a Digital Maturity Framework for Higher Education Institutions (DMFHEI). The methodology for framework development consists of five main steps: literature search and data extraction ; two focus group analyses ; applying theQ-sorting method ; reviews by two experts ; and a summary of results. Results presented in this paper show that the DMFHEI has recognised seven areasthatcan bedivided into 43 describable elements.

Digital competences should be considered in teacher education but also in the field of teacher professional development. Technological Pedagogical Content Knowledge (TPACK) framework represents teacher knowledge for technology integration and is represented by seven domains. Survey of Preservice Teachers’ Knowledge of Teaching and Technology (SPTKTT) is one of the inventories developed to measure TPACK framework. Authors of the inventory have found empirical support for all seven domains of TPACK framework by the use of exploratory factor nalysis. Aim of this study is to validate SPTKTT inventory in the context of Croatian educational system by the use of exploratory and confirmatory factor analysis. Results showed that structure of SPTKTT inventory consists of nine factors, while confirmatory factor analysis confirmed that three factors which refers to Content Knowledge domain fall under the one single factor. Internal consistency of the inventory shows high reliability.

This workshop presents the developed maturity model for supporting graduates early careers within HEIs taking into account diversity of student body. The final model was developed based on the case study research at four HEIs in Europe, inputs from different higher education stakeholders (teaching and non- teaching staff, students, alumni, employers, policy makers etc.) and testing at four HEIs in Croatia.

MOOCs were initially designed as stand-alone products but blended learning programs have been developed to utilize the benefits of MOOCs. This paper presents case study research of incorporating a MOOC into traditionally delivered course with specific goals of giving students opportunity to experience learning in virtual environment and helping part-time students in achieving particular learning outcomes. Benefits and obstacles of such an effort were researched in a case study where part of the traditionally delivered learning was replaced with a MOOC at a compulsory graduate level course. Students were asked to choose between completing a MOOC and taking project-based activity to work towards the final grade. While going through the MOOC, it was required from students to keep a learning diary to showcase the understanding of the content but also to provide insight for qualitative analysis based on a set of questions that the students were required to answer. The case study research was based on qualitative analysis of the learning diaries and on quantitative review of achieved results. Qualitative analysis of the answers to open ended questions showed that students’ overall feedback has been positive. Students were generally satisfied with this form of learning, despite some difficulties, such as language barrier or sufficient prior knowledge or required workload. Most appreciated characteristics of MOOCs were self-paced learning and the option to assess knowledge on regular basis, especially among part-time students.

Learning analytics should go beyond data analysis and include approaches and algorithms that are meaningful for learner performance and that can be interpreted by teacher and related to learning outcomes. Assessment analytics has been lagging behind other research in learning analytics. This holds true especially for peer- assessment analytics. In this paper we present a mathematical model for peerassessment based on the use of scoring rubrics for criteriabased assessment. We propose methods for the calculation of the final grade along with reliability measurues of peerassessment. Modeling is motivated and driven by the identi- fied peer-assessment scenarios. Use of peer- assessment based on a sound model provides benefits of the deeper learning while addressing the issues of validity and reliability.

This paper gives a brief overview of the changes within higher education system during the last few decades. Further, the paper brings critical analysis of approaches to strategic decision-making and strategic planning in higher education and most importantly, it brings examples of strategic problems and possible strategic decisions as well as strategic decision evaluation possibilities within higher education and the corresponding impact evaluation within Croatian HE institutions.

The aim of learning analytics is to use available data from different systems and data bases in order to support students and teachers in learning and teaching processes. In order to ensure that main groups of users get what they need the most by using learning analytics, there is an imperative to develop feasible needs analysis methodology as well as to perform the needs analysis according to it. There are far less research and prediction about the role and implementation of learning analytics in pretertiary education (schools) than in higher education. The research presented in this paper was performed in primary and secondary schools in Croatia. Final results are presented in form of most relevant questions posed by students and teachers that learning analytics system supposed to answer.

Strategic decision making implementation is still an important problem in higher education (HE). The shift in research moved from goals and activities towards recognizing decision making methods used for decision making (DM) and evaluation of the strategy implementation. The purpose of this paper is to investigate which decision making methods and methodologies are used in the decision making processes in higher education, especially strategic decision making problems connected to the implementation of e-learning. In order to achieve this goal we reviewed 40 research papers. Results show diversity of methods, methodologies and approaches used in the strategic decision making in HE which proves complexity of the topic. We summarize them in four phases and also recommend methods that can be successfully applied based on the literature review presented in this paper and authors’ practical experiences.

Five European Union targets until 2020 include the necessity to increase the share of population having completed tertiary education (at least 40% of 30-34–year-olds completing third level education), to raise the employment rates (75% of the 20-64 year-olds to be employed) but also to reduce the poverty and social exclusion. In order to achieve those goals, higher education institutions should pay attention to improve their practices focused on widening access, to support retention for all students, especially for those underrepresented and unprivileged, as well as to enhance the employment capability of their graduates. This should be done, not only through the provision of support services, but integrating that support into strategy development, curriculum design and delivery and different extra- curricular activities. This paper brings the preliminary results from case studies research conducted at four higher education institutions in Europe that differ in their size, funding, government and support they provide for students from their access through the whole study period. Case studies research was conducted at: University of Belgrade, Faculty of Organizational Sciences (Serbia), University of Edinburgh (UK), Vienna University of Economics and Business (Austria) and University West (Sweden). The preliminary results of this study provide insight into good practices supporting student retention and employment capabilities within higher education institution. The initial list of practices will serve for further development of the strategic framework with accompanying maturity model, as a guidance tool for higher education institution striving to improve the maturity of their practices related to student support.

We investigate and advocate a geometrical approach to learning analytics with a focus on peer assesment. Also, we present a case study for course improvement.

U ovom radu su istraženi stavovi prema matematici i korištenju novih tehnologija studenata prve godine Informacijskih i poslovnih sustava na Fakultetu organizacije i informatike. Konfirmatorna faktorska anliza pokazala je da je stav prema e-učenju novi faktor koji se razlikuje od pouzdanja u sposobnost upotrebe tehnologije i kao takav treba naći svoje mjesto u teoretskom modelu stavova prema matematici.

Learning outcomes are considered to be a key tool for student-centered teaching and learning. They can be successfully implemented in teaching and learning mathematics on higher educational level and together with appropriate level of technology enhanced learning can provide the framework for successful learning process even for students that have not been primarily interested in mathematics. One of the most prominent goals of mathematics in non- mathematics major study programs is providing students with mathematical modelling skills. In this respect assessment methods should be carefully chosen and implemented in concrete learning context to take into account problem posing and problem solving. This paper describes the way in which rubrics can be used in a mathematical course in order to assure the fulfillment of the learning outcome concerning mathematical modeling and problem solving. Problem posing out of the real world context confirmed to be hard task for students.

Learning analytics deals with the data that occurs from students' interaction with ICT: collecting data, analyzing and reporting can influence learning and teaching. Application of learning analytics for analyzing assessment has lagged behind other areas of application. We argue here for the need for learning analytics for assessment with a special focus on peer and self-assessment. All forms of assessment should encourage deep learning. Reliability and validity of peer assessment will be discussed. In this context, a case study will be presented.