Search

Assessment is among the inevitable components of a curriculum and directs students’ learning. E- assessment, as prepared and administered with the use of ICT, provides opportunities to make the process easier in some aspects, but also brings certain challenges. This paper presents an e- assessment framework from a student perspective. Our study lasted for one academic year and incorporated a total of 631 students’ responses to questionnaires conducted three times in two different mathematics courses. The courses are compulsory for undergraduate informatics students. The research included a three-step factor analysis: two exploratory factor analyses and a confirmatory factor analysis. It yielded a model of a student perspective on e-assessment consisting of four factors: Transparency and fairness of assessment, Formative and summative assessment and feedback, Meaningful use of technology in assessment and Difficulty of learning outcomes. The students’ opinion about the use of e-assessment was generally positive. The students appreciated the student-centred approach based on sound pedagogical alignment of all teaching and learning elements with assessment. They found it important that technology should not distract students during assessment tasks and considered cheating in a controlled e-assessment environment to be no more frequent than in face- to-face assessment.

The aim of this half-day face-to-face workshop, organized in cooperation of three universities, is twofold. First, it will provide a platform for sharing of experiences, research and challenges related to the link between learning analytics (LA) and learning design (LD). Second, the workshop will enable participants to engage with an innovative, free-to-use LD tool (learning-design.eu), and create advanced learning analytics on LD using the tool. Participants will be invited to work collaboratively on LD of their own courses, reflect on the LA generated by the LD tool and improve the course LD accordingly. This will also contribute to the further development of the concept and tool, based on a pre-established research protocol. Participants will take away recommendations for improvement of their own courses, as well as know-how on how to use an innovative LD tool at their own institutions. Ahead of the workshop, if interested, participants will be invited to apply for a short presentation (5-10 minutes). They will also be asked to consider their courses and particular learning outcome(s) which could be redesigned at the workshop.

There are different ways games can enrich teaching and learning. The two basic approaches are gamification and game-based learning (GBL). If a teaching and learning process is implemented as a game, it is considered GBL. But when game-like elements enrich only some aspect of a teaching and learning process, it is gamification. This paper presents the case study: Who wants to be a millionaire? (WWTBM) implemented as GBL. In most cases, a quiz is just an example of gamification. In the case study presented in this paper, WWTBM was used in form of well-known TV show but for a special audience. The case study was implemented at the University of Zagreb, Faculty of Organization and Informatics (FOI) as a part of the professional development training for FOI’s teachers. The training covered many topics related to applying innovative teaching and learning strategies and methods, such as flipped classroom, work-based learning, project-based learning, and laboratory-based learning. The topics were delivered as separate workshops, based on the teach as you preach principle. The training was organized as a joint venture of three projects: Erasmus+ RAPIDE and two projects, DIP2Future and Study4Career, funded by the European Social Fund. The workshop related to using games in teaching and learning was implemented in two steps: (1) the participants were asked to watch a WWTBM video (asynchronous) before the workshop, and (2) the video was analyzed in the synchronous hybrid (face-2-face and online) part of the workshop. The duration of the WWTBM video was 43 minutes. The host and the quiz contestant were the same person (the GBL workshop author). The video was recorded in FOI’s video studio using chroma key features to achieve visual authenticity of the WWTBM in the postproduction. The visual elements of the real- life TV WWTBM show were complemented with authentic sound elements. All questions in the show were related to gamification and GBL. The idea was to introduce the viewers to the topic’s content gradually. The questions were not answered right after they were presented – the drama and uncertainty characteristic of the actual TV show were also present. The viewers were motivated to think together with the contestant and play the game and learn. The contestant elaborated his reasoning, which helped the viewers understand the content. In addition, some explanations were given by the host. The contestant could use three jokers. The final question was created to summarize the topic: if the viewers were able to answer it correctly before the contestant, it could be concluded that they achieved the goal of the show: to learn about the topic and have fun. The workshop participants valuated the video using 15 questions during the synchronous session. They underestimated the time the author needed to create the video, declared that they successfully answered the final question, answered the control questions correctly, evaluated positive and negative aspects of the show, recognized the basic elements of a game in the show, and strongly agreed that the video was successful. Finally, it would be of utmost importance to collect data from the teachers on whether they were motivated by the training to use game-based elements and other innovative strategies in their teaching.

This paper presents a study conducted within an international project to investigate the experiences of higher education (HE) teachers with the shift to digital teaching during the COVID-19 pandemic. The study was based on a survey conducted among HE teachers (n = 167) in autumn 2021 in four European countries: Spain, Portugal, Finland and Croatia. Analysis of Variance (ANOVA) was used to investigate the differences in responses with respect to the country and the study field, with more significant differences found among the countries than among the study fields. The results will contribute to a MOOC for teachers’ professional development.

Zbog pandemije COVID-a obrazovne institucije diljem svijeta (pa tako i u Republici Hrvatskoj) privremeno su zatvorile svoje zgrade i učionice. To je rezultiralo brzim prelaskom na učenje i poučavanje na daljinu te online vrednovanje. Veliki izazov je u hibridnom ili potpuno online modelu osigurati i održati kvalitetnu nastavu matematike, posebno prilikom uvođenja novih koncepata koji su studentima zahtjevni. Posebno je zahtjevna priprema, organizacija i provođenje online vrednovanja koje mora udovoljavati stručnim i metodičkim zahtjevima, ali i biti vjerodostojno. Kritički ispitujemo ulogu učenja na daljinu i online vrednovanja u stjecanju znanja, vještina i stavova. Zalažemo se za konstrukciju cjelovitog programa vrednovanja s pomno pripremljenim skupom metoda vrednovanja, ali i njegovu evaluaciju prema sveobuhvatnom okviru. U tu svrhu odabrali smo takozvani Model korisnosti vrednovanja koji su uveli Van der Vleuten i Schurwith (2005). Spomenuti model sadrži 5 kriterija: pouzdanost, valjanost i prihvatljivost vrednovanja, utjecaj vrednovanja na učenje, te troškovi modela vrednovanja. Fokus našeg rada je na pouzdanosti koju vrlo općenito shvaćamo kao razinu točnosti odluka za prolaz i neuspjeh. Analizu pouzdanosti provodimo na razini ispita, ali i na razini cijelog programa vrednovanja za pojedini predmet. Preciznije, konstruiramo kompozitni indeks za program vrednovanja na razini cijelog predmeta na dva načina. Taj kompozitni indeks predstavlja moguću mjeru pouzdanosti cijelog programa vrednovanja. Za testiranje modela koristili smo analitike učenja dostupne u sustavu za upravljanje e-učenjem (Moodle) za posljednjih nekoliko akademskih godina. Podatke o rezultatima vrednovanja analizirali smo uz pomoć statistika testa u Moodleu te odredili koeficijent interne konzistentnosti (tzv. Crombach alpha) na razini pojedinog dijela u planu vrednovanja (npr. za kolokvij) koje smo zatim težinski usrednjili. Statistike testa u Moodleu, omogućavaju upotrebu i drugih parametra kvalitete ispitnih zadataka na razini testa, ali i na razini pojedinog pitanja.

Technology can support teaching and learning processes in many ways. In particular, e-assessment can provide a meaningful upgrade to standard approaches. We will present a model of a student perspective on e-assessment in mathematics in higher education. The focus of our presentation will be on e-assessment supported by a Learning Management System (LMS). Within our mathematics courses paper-based assessments were replaced with computer-based ones. E-assessment questions were individualized, taken from a systematized online question bank, in which questions are organized according to a taxonomy of learning outcomes. E-assessments were conducted in a controlled online environment. We collected 631 students' responses to questionnaires on three occasions, in two different mathematics courses during one academic year. The research included two exploratory factor analyses and a confirmatory factor analysis. It yielded a model of a student perspective on e-assessment, consisting of four factors: Transparency and fairness of assessment, Formative and summative assessment and feedback, Meaningful use of technology in assessment and The difficulty of learning outcomes.

In this paper we present a systematic literature review of studies related to digital maturity of schools. The focus is on the identification of the most significant papers from the field of digital maturity of schools, essential elements of digital maturity frameworks and their validation, as well as the connection with other sectors related to the school. The analysis shows that ICT infrastructure is an important but not sufficient element for digital transformation. Digital transformation is also influenced by digital competences of schools’ employees, use of ICT in teaching and learning and administration, and the strategic approach to digital transformation.

Tijekom pandemije je veliki izazov osigurati i održati kvalitetnu nastavu matematike i drugih predmeta koji su studentima konceptualno zahtjevni, u hibridnom ili potpuno online modelu. Posebno je zahtjevna priprema, organizacija i provođenje online vrednovanja koja moraju udovoljavati stručnim i metodičkim zahtjevima, ali i biti vjerodostojna. Principi kojima se vodimo u dizajnu nastave i online vrednovanja su jasna veza vrednovanja s planiranim ishodima učenja predmeta, upotreba kompleksnih i/ili problemskih zadataka strukturiranih pomoću prikladne taksonomije u velike baze zadataka i pitanja (tako da svaki student dobije “svoje zadatke”). Uz nabrojeno, razumno su odabrane i druge mjere za prevenciju i sprečavanje varanja na (online) ispitima.

Assessment is one of the central components of a curriculum and guides students' learning. E- assessment involves the use of ICT to present and deliver assessment tasks, receive and record students' answers, as well as to provide feedback to students. E-assessment has been used considerably more in many higher education institutions since the onset of the COVID-19 pandemic and therefore our aim is to explore the specifities of e-assessment in mathematics. We will present the benefits and downsides of e- assessment in mathematics in higher education. Our study focused on students' perspectives on e- assessment supported by a Learning Management System (LMS) in mathematics courses. It incorporated 631 students' responses to questionnaires conducted three times and it lasted for one academic year. We used descriptive statistics for close-ended questions and qualitative analysis for open-ended questions. The students' perspectives on e- assessment were generally positive. The students appreciated the student-centered approach based on sound pedagogical alignment of all teaching and learning elements with assessment. However, they found it important that technology should not distract them during assessment tasks. Finally, they did not find that there was more cheating and plagiarism in e- assessment in comparison to on-campus face-to-face assessments.

The assessment should be constructed and analysed holistically. The role of assessment should be critically examined in acquisition of knowledge, skills and attitudes. We postulate two aspects: 1. The complete assessment programme with carefully prepared assessment set should be considered. 2. Assessment programme should be evaluated according to an overarching framework. According to (Van der Vleuten & Schuwirth, 2005) the utility framework for assessment depends on 5 factors: Validity (the most frequent quality criterion), Reliability (the accuracy of pass and fail decisions), Educational impact, Acceptability and Cost of assessment. Our aim is to use learning analytics to support evaluation of assessment set according to the utility framework.

Due to the COVID{;19 pandemic the educational institutions around the world have been temporarily closed by their governments which resulted in rapid transition to remote learning and online assessment. We critically examine it's role in acquisition of knowledge, skills and attitudes. We advocate construction of the complete assessment programme with carefully prepared assessment set, but also its evaluation according to an overarching framework. For that purpose we have chosen the so-called utility framework introduced by Van der Vleuten & Schurwith, 2005. We will present the utility framework which consists of five elements (validity, reliability, educational impact, acceptability and costs). The focus will be on the reliability which we broadly understand as the accuracy of pass and fail decisions. More precisely, we will introduce and analyse several approaches for construction of the composite index for the assessment programme on the course level as a possible measure of the reliability. To do that, we use data available in the learning management system for past two academic years.

This invited lecture covers a choice of research within the field of learning analytics, as well as the applications in educational systems which includes the strategic decision-making (national) but also the analysis of the examples at the study programme and course level. I propose some research questions for future research.

TEM Journal; Vol 6, No 4, 2017. ISSN 2217-8309, The paper describes and interprets differences among social network structures in the private sector, politics (local administration) and non-governmental organisations (NGO) in Croatia. Social network analysis (SNA) methodology was used to examine networks in three sectors, which were found to differ in terms of centrality measures and substructures. In the private sector network, centrality values of network members were related to their respective positions in the organisational hierarchy. In the politics network, centrality measures values of members were influenced both by hierarchy and by the number of same-party members. In the NGO network, all members had high centrality values. Substructure sizes were low in the private sector and high in the NGO sector; in the politics network, they were dependent on the number of same-party members.

Strategic planning using the balanced scorecard (BSC) is a usual activity in many higher education institutions (HEIs). In BSC, goals are deployed into BSC perspectives. In non- profit HEIs, those perspectives are (from bottom to top): the finance perspective, the learning and growth perspective, the internal processes perspective, the costumers perspective and the mission perspective. In terms of profit HEIs (ex. private universities), the finance perspective is at the top of the map (instead of the mission perspective). In the literature, we can also find totally new BSC perspectives that are adjusted to the particular HEI’s needs. Nevertheless, when HEI’s administration selects a specific strategic map design, it places strategic goals into the map and connects them with respect to the existence of the influences between goals. In theory, a goal from a particular perspective can influence on any other goal from the same perspective, as well as to any goal from any perspective that is above its perspective. After the strategic map of goals is created, there is often a request to prioritize strategic goals in the strategic map in to determine the most crucial goals. In this paper, we present several possible ways to prioritize BSC goals. Three of them are the direct applications of three methods: the analytic hierarchy process (AHP), the analytic network process (ANP) and the PageRank centrality. Additionally, we propose an integrated approach. The main disadvantage of applying the AHP is that influences between goals are not included in goals’ priorities calculations. The main disadvantage of the ANP is that many BSC goals will weigh 0.0 because the graph that is associated with the BSC strategic map of goals is almost always reducible. This issue can be solved by applying the PageRank centrality. The procedure of this method ensures that the original BSC map will become irreducible. However, applying the ANP or PageRank does only consider the influences between goals in BSC map. It does not include the importance of each goal for the HEI into consideration. To ensure both, we proposed and applied the integrated approach which combines the AHP with PageRank centrality. In this paper, all these concepts are applied and demonstrated on the case of the BSC strategic map of goals of the Faculty of organization and informatics, University of Zagreb and can be applied for other HEIs’ strategic maps of goals.

Agencija za znanost i visoko obrazovanje (AZVO) tijekom 2017. godine provela je anketu o zapošljivosti diplomanada na nacionalnoj razini u okviru projekta SKAZVO – Unapređenje sustava osiguravanja i unapređenje kvalitete visokog obrazovanja. Cilj je provođenja ankete o zapošljivosti diplomanada bio prikupljanje podataka o generaciji studenata koji su diplomirali na visokim učilištima u Republici Hrvatskoj 2016. godine, o njihovu zadovoljstvu pripremom za tržište rada u okviru visokih učilišta te o prijelazu iz sustava visokog obrazovanja na tržište rada i prvom zaposlenju. Rezultati analize prikupljenih podataka predstavljaju temelj za unapređenje sustava visokog obrazovanja i poticanje aktivnosti usmjerenih boljoj povezanosti s tržištem rada. Ovo izvješće donosi kratak opis anketnog upitnika i strukture ispitanika te osnovne rezultate deskriptivne statistike. Osim zbirnog izvješća pripremljena su i pojedinačna izvješća za visoka učilišta kako bi mogli dobiti detaljniji uvid u strukturu zapošljivosti svojih diplomanada.

Strategic decision making about education and open and distance education (ODE) issues requires analytic approach supported by a suitable method. The Analytic Network Process (ANP) meets all the requirements of decision making in higher education (HE), but it is very rarely used in practice because of its weaknesses. The weaknesses are related to the complexity of the method and demand on resources in the process of implementation. An upgraded ANP method that combines the ANP with the Social Network Analysis (SNA), which diminishes some of the weaknesses of the original ANP, is presented in this paper.

Rapid diffusion of Information and Communication Technologies (ICT) in all professional and personal areas require digital maturity from schools. In order to assess this aspect of school, a concept of digitally mature school has been developed and translated into different frameworks. In this paper, we describe development methodology of Framework for Digitally Mature Schools (FDMS) in Croatia. The FDMS, together with the accompanying instrument and software, represents a unique and comprehensive tool set for the assessment of digital maturity of a school. The FDMS recognized five areas divided into 38 elements that are described on five digital maturity levels in the form of a rubric.

Digital technologies in educational institutions have the potential to be one of the main catalysts of quality education. The purpose of this paper is to analyse digital maturity frameworks, to describe the irdevelopment methodologies and to propose a Digital Maturity Framework for Higher Education Institutions (DMFHEI). The methodology for framework development consists of five main steps: literature search and data extraction ; two focus group analyses ; applying theQ-sorting method ; reviews by two experts ; and a summary of results. Results presented in this paper show that the DMFHEI has recognised seven areasthatcan bedivided into 43 describable elements.

Digital competences should be considered in teacher education but also in the field of teacher professional development. Technological Pedagogical Content Knowledge (TPACK) framework represents teacher knowledge for technology integration and is represented by seven domains. Survey of Preservice Teachers’ Knowledge of Teaching and Technology (SPTKTT) is one of the inventories developed to measure TPACK framework. Authors of the inventory have found empirical support for all seven domains of TPACK framework by the use of exploratory factor nalysis. Aim of this study is to validate SPTKTT inventory in the context of Croatian educational system by the use of exploratory and confirmatory factor analysis. Results showed that structure of SPTKTT inventory consists of nine factors, while confirmatory factor analysis confirmed that three factors which refers to Content Knowledge domain fall under the one single factor. Internal consistency of the inventory shows high reliability.

This workshop presents the developed maturity model for supporting graduates early careers within HEIs taking into account diversity of student body. The final model was developed based on the case study research at four HEIs in Europe, inputs from different higher education stakeholders (teaching and non- teaching staff, students, alumni, employers, policy makers etc.) and testing at four HEIs in Croatia.

MOOCs were initially designed as stand-alone products but blended learning programs have been developed to utilize the benefits of MOOCs. This paper presents case study research of incorporating a MOOC into traditionally delivered course with specific goals of giving students opportunity to experience learning in virtual environment and helping part-time students in achieving particular learning outcomes. Benefits and obstacles of such an effort were researched in a case study where part of the traditionally delivered learning was replaced with a MOOC at a compulsory graduate level course. Students were asked to choose between completing a MOOC and taking project-based activity to work towards the final grade. While going through the MOOC, it was required from students to keep a learning diary to showcase the understanding of the content but also to provide insight for qualitative analysis based on a set of questions that the students were required to answer. The case study research was based on qualitative analysis of the learning diaries and on quantitative review of achieved results. Qualitative analysis of the answers to open ended questions showed that students’ overall feedback has been positive. Students were generally satisfied with this form of learning, despite some difficulties, such as language barrier or sufficient prior knowledge or required workload. Most appreciated characteristics of MOOCs were self-paced learning and the option to assess knowledge on regular basis, especially among part-time students.