Search

In intelligent e-learning systems that adapt a learning and teaching process to student knowledge, it is important to adapt the system as quickly as possible. However, adaptation is not possible until the student model is initialized. In this paper, a new approach to student model initialization using domain knowledge representative subset is described. The approach defines which concepts from domain knowledge should be included in the initial test so the system can make conclusions about what students truly know about domain knowledge. This representative subset of domain knowledge is defined using non-semantic mathematical approach based on graph theory. The initial test, created over a domain knowledge representative subset, guarantees encompassing all concepts that are relevant to domain knowledge. A two-level case study is conducted on what would be the representative subset of one selected domain knowledge. It compares semantically selected domain knowledge representative subsets (semantical analysis was done by domain area experts) to a non-semantical, mathematically selected domain knowledge representative subset. The results of the case study show that problems of inequality of semantically selected domain knowledge representative subsets are easily overcome using the presented approach.

Various approaches to text simplification have been proposed in an attempt to increase text readability. The rephrasing of syntactically and semantically complex structures is still challenging. A pedagogically motivated simplified version of the same text can have both positive and negative side effects. On the one hand, it can facilitate reading comprehension because of much shorter sentences and a limited vocabulary, but on the other hand, the simplified text often lacks coherence, unity and style. Therefore, reasonable trade-offs among linguistic simplicity, naturalness and informativeness are highly needed. This is a survey paper that discusses state-of-the-art approaches to sentence/text simplification and evaluation methods, along with an empirical evaluation of our approach. The quality of sentence splitting, using the knowledge extraction tool SAAT was compared to state-of-the-art syntactic simplification systems. The research was carried out on the WikiSplit, the HSplit and MinWikiSplit simplification corpora. Automatic metrics for the HSplit showed that the SAAT outperformed other TS systems in all categories. For the WikiSplit dataset, automatic metrics scores were slightly lower than that of the baseline system DisSim. However, the human evaluation showed that DisSim outperformed the SAAT in terms of simplicity and grammar. The quality of AG18copy output corresponded to that of the SAAT. The inter-annotator agreement was calculated. Research limitations as well as suggestions for future research were also provided.

In this article we present an knowledge extraction approach that can be used in systems that implement teaching in a fully automated manner. These systems are called Intelligent Tutoring Systems (ITS) and are conceived around the idea of one-to-one teaching. Many such systems use natural language processing to improve the communication interface between student and the system. These techniques can be also used on the content creator side to semi- automate or fully automate the task of teaching content creation. In such systems the knowledge representation plays a crucial role to successfully implement teaching and encourage learning. The output of the knowledge extraction phase is a knowledge in the form of a hyper graph that can be used for adaption to the students current knowledge level. We present a deep neural network architecture for precise POS tagging of words written in languages that are morphologically rich. Using sparse representations for words in this task increases the vector space and makes learning more complex. This problem can be solved to some extent by using traditional vector representations but there is also the problem with representing words that are ambiguous. Proposed architecture uses a Bidirectional Encoder Representations from Transformers (BERT) model that is pre-trained on Croatian language to achieve state-of-the-art accuracy for POS tagging.

The idea of clustering students according to their online learning behavior has the potential of providing more adaptive scaffolding by the intelligent tutoring system itself or by a human teacher. With the aim of identifying student groups who would benefit from the same intervention in AC-ware Tutor, this research examined online learning behavior using 8 tracking variables: the total number of content pages seen in the learning process; the total number of concepts; the total online score; the total time spent online; the total number of logins; the stereotype after the initial test, the final stereotype, and the mean stereotype variability. The previous measures were used in a four-step analysis that consisted of data preprocessing, dimensionality reduction, the clustering, and the analysis of a posttest performance on a content proficiency exam. The results were also used to construct the decision tree in order to get a human-readable description of student clusters.

The reasoning process about the level of student’s knowledge can be challenging even for experienced human tutors. The Bayesian networks are a formalism for reasoning under uncertainty, which has been successfully used for various artificial intelligence applications, including student modeling. While Bayesian networks are a highly flexible graphical and probabilistic modeling framework, its main challenges are related to the structural design and the definition of “a priori” and conditional probabilities. Since the AC&NL Tutor’s authoring tool automatically generates tutoring elements of different linguistic complexity, the generated sentences and questions fall into three difficulty levels. Based on these levels, the probability- based Bayesian student model is proposed for mastery-based learning in intelligent tutoring system. The Bayesian network structure is defined by generated questions related to the node representing knowledge in a sentence. Also, there are relations between inverse questions at the same difficulty level. After the structure is defined, the process of assigning “a priori” and conditional probabilities is automated using several heuristic expert-based rules.

Automatic knowledge acquisition is a rather complex and challenging task. This paper focuses on the description and evaluation of a semi-automatic authoring tool (SAAT) that has been developed as a part of the Adaptive Courseware based on Natural Language AC&NL Tutor project. The SAAT analyzes a natural language text and, as a result of the declarative knowledge extraction process, it generates domain knowledge that is presented in a form of natural language sentences, questions and domain knowledge graphs. Generated domain knowledge presents expert knowledge in the intelligent tutoring system Tutomat. The natural language processing techniques are applied and the tool’s functionalities are thoroughly explained. This tool is, to our knowledge, the only one that enables natural language question and sentence generation of different levels of complexity. Using an unstructured and unprocessed Wikipedia text in computer science, evaluation of domain knowledge extraction algorithm, i.e. the correctness of extraction outcomes and the effectiveness of extraction methods, was performed. The SAAT outputs were compared with the gold standard, manually developed by two experts. The results showed that 68.7% of detected errors referred to the performance of the integrated linguistic resources, such as CoreNLP, Senna, WordNet, whereas 31.3% of errors referred to the proposed extraction algorithms.

The idea of clustering students according to their online learning behavior has the potential of providing more adaptive scaffolding by the intelligent tutoring system itself or by a human teacher. With the aim of identifying groups of students who would benefit from the same intervention, in this paper, we study a set of 104 weekly behaviors observed for 26 students in a blended learning environment with AC-ware Tutor, an ontology-based intelligent tutoring system. Online learning behavior in AC-ware Tutor is described using 8 tracking variables: (i) the total number of content pages seen in the learning process ; (ii) the total number of concepts seen in the learning process ; (iii) the total content proficiency score gained online ; (iv) the total time spent online ; (v) the total number of student logins to AC-ware Tutor ; (vi) the stereotype value after the initial test in AC-ware Tutor, (vii) the final stereotype value in the learning process, and (viii) the mean stereotype variability in the learning process. The previous measures are used in a four-step analysis process that includes the following elements: data preprocessing (Z-score normalization), dimensionality reduction (Principal component analysis), the clustering (K-means), and the analysis of a posttest performance on a content proficiency exam. By using the Euclidean distance in K-means clustering, we identified 4 distinct online learning behavior clusters, which we designate by the following names: Engaged Pre-knowers, Pre-knowers Non-finishers, Hard-workers, and Non-engagers. The posttest proficiency exam scores were compared among the aforementioned clusters using the Mann-Whitney U test.

Over the last few decades, researchers put efforts to improve intelligent tutoring systems' abilities with the aim to get them as close as possible to the ultimate goal of one-to-one tutoring. CoLaB Tutor and AC-ware Tutor are intelligent tutoring systems based on conceptual knowledge learning and are notable due to the fact they are relatively easy to generalize to multiple knowledge domains. CoLaB Tutor's forte lies in teacher-learner communication in controlled natural language, while AC-ware Tutor focuses on the automatic and dynamic generation of adaptive courseware. In order to compare various intelligent tutoring system supported education environments, in this chapter, the authors summarize several empirical evaluations of CoLaB Tutor and AC-ware Tutor. The results of intelligent tutoring systems' effectiveness in these environments offer the possibility to observe the specific intelligent tutoring system across various education levels, as well as to compare the intelligent tutoring systems' supported education environments.

Probabilistic student model based on Bayesian network enables making conclusions about the state of student’s knowledge and further learning and teaching process depends on these conclusions. To implement the Bayesian network into a student model, it is necessary to determine “a priori” probability of the root nodes, as well as, the conditional probabilities of all other nodes. In our approach, we enable non-empirical mathematical determination of conditional probabilities, while “a priory” probabilities are empirically determined based on the knowledge test results. The concepts that are believed to have been learned or not learned represent the evidence. Based on the evidence, it is concluded which concepts need to be re-learned, and which not. The study described in this paper has examined 15 ontologically based Bayesian student models. In each model, special attention has been devoted to defining “a priori” probabilities, conditional probabilities and the way the evidences are set in order to test the successfulness of student knowledge prediction. Finally, the obtained results are analyzed and the guidelines for ontology based Bayesian student model design are presented.

This paper presents the current state of research in the fields of intelligent tutoring systems and intelligent agents. The architecture of intelligent tutoring systems typically consists of four major components: the domain knowledge module, the learner diagnosis module, the tutoring module, and the communication module. An agent-based approach to intelligent tutoring systems research and development is applied as the set of agents that constitute the general architecture, and as animated characters - pedagogical agents. Therefore, the paper focuses on the components of intelligent tutoring systems that can be implemented as agents, as well as on functions that these intelligent agents can perform in each intelligent tutoring system component. Also, we present the recent endeavours in the intelligent tutoring systems community that moves from standalone systems focused on a single instructional domain to adaptive instructional systems that comprise learners, intelligent tutoring systems, and external (non-adaptive) environment.

Purpose– In order to initialize a student model in intelligent tutoring systems, some form of initial knowledge test should be given to a student. Since the authors cannot include all domain knowledge in that initial test, a domain knowledge subset should be selected. The paper aims to discuss this issue.Design/methodology/approach– In order to generate a knowledge sample that represents truly a certain domain knowledge, the authors can use sampling algorithms. In this paper, the authors present five sampling algorithms (Random Walk, Metropolis-Hastings Random Walk, Forest Fire, Snowball and Represent algorithm) and investigate which structural properties of the domain knowledge sample are preserved after sampling process is conducted.Findings– The samples that the authors got using these algorithms are compared and the authors have compared their cumulative node degree distributions, clustering coefficients and the length of the shortest paths in a sampled graph in order to find the best one.Originality/value– This approach is original as the authors could not find any similar work that uses graph sampling methods for student modeling.

U radu su prikazani rezultati istraživanja o učinku učenja i poučavanja učenika sedmih razreda osnovne škole pomoću sustava „Moodle“ za područno znanje nastavnog predmeta hrvatski jezik. Istraživanje su provele učiteljice hrvatskoga jezika i studentice druge godine diplomskog studija informatike Prirodoslovno-matematičkog fakulteta Sveučilišta u Splitu iz kolegija „Vrednovanje sustava e-učenja“ u zimskom semestru akademske godine 2011./2012. Obuhvaćen je uzorak od dvaju razrednih odjela (43-oje učenika) sedmih razreda OŠ Lučac u Splitu. Istraživanje je pokazalo da postoji mala veličina učinka, tj. da primjena ovakvog oblika nastave ne pridonosi statistički značajno povećanju učinkovitosti u procesu učenja i poučavanja. Uporaba suvremenih tehnologija u nastavi hrvatskoga jezika u kombinaciji s tradicionalnom nastavom bio bi najbolji izbor jer pomaže učiteljima da lakše organiziraju kvalitetnu nastavu., This paper presents the results of a research on the impact of using the system Moodle in learning and teaching seventh graders the domain knowledge of the course in the Croatian language. The research was conducted by the Croatian language teachers and the second year graduate students from the Faculty of Natural Sciences and Mathematics, University of Split, that have enrolled on the course “Evaluation of e-learning systems” in the winter semester of the academic year 2011/2012. The sample included two seventh grade classes (43 students) from Elementary School of Lučac in Split. The research has shown that there is a small effect size, i.e. the usage of this form of teaching does not contribute significantly to improving the efficiency in the learning and teaching process. The use of modern technologies in teaching Croatian in combination with traditional approaches would be the best choice as this helps teachers to organize high-quality teaching more easily.

As the acquisition of knowledge is often an expensive and time-consuming process, it is important to know whether it actually improves the student performance. The e-learning is a revolutionary paradigm that has lately been significantly evolving and it is closely related to the intelligent tutoring systems. Methodology for evaluating the educational influence of learning and teaching process, questions whether and in what amount, students learn effectively. Our contribution to this compulsive field of research is a meta-analysis of a series of experiments based on the same two-group methodology that reveals a more precise effect size of one particular e-learning system - eXtended Tutor-Expert System, a representative of web-based authoring shells for building intelligent tutoring systems.

Intelligent tutoring systems are kind of asynchronous e-learning systems designed to support and improve learning and teaching process in particular domain knowledge. An authoring shells are kind of e-learning systems that feature authoring environments for system users. Domain knowledge in such systems can be represented by using different knowledge representation specifications and presentation of tests mainly depends on the type of domain knowledge. We propose templates for dynamical generation of questions as a test over previously formalized domain knowledge. In our approach we encourage expressiveness of ontology for describing domain knowledge. Tests described in this paper entails declarative knowledge formalized by Web Ontology Language (OWL) and are realized as a dynamic quiz. By pronouncing OWL ontology as domain knowledge formalism we deal with the problem of generating tests and understanding presentation of the tests.

We believe that every effectiveness evaluation should be replicated at least in order to verify the original results and to indicate evaluated e-learning system’ s advantages or disadvantages. This paper presents the methodology for conducting controlled experiment replication, as well as, results of a controlled experiment and an internal replication that investigated the effectiveness of intelligent authoring shell eXtended Tutor– Expert System (xTEx-Sys). The initial and the replicated experiment were based on our approach that combines classical two-group experimental design and with factoral design. A trait that distinguishes this approach from others is the existence of arbitrary number of checkpoint-tests to determine the effectiveness in intermediate states. We call it a pre-and-post test control group experimental design with checkpoint-tests. The gained results revealed small or even negative effect sizes, which could be explained by the fact that the xTEx-Sys’ s domain knowledge presentation is rather novel for students and therefore difficult to grasp and apply in earlier phases of the experiment. In order to develop and improve the xTEx-Sys, further experiments must be conducted.

All instructional software should be evaluated before being used in educational process because it is important to know whether it actually improves the student performance. Within the context of evaluating the educational influence of learning and teaching process, we measure educational influence by using the effect size as metric. In this paper we present methodology for evaluating educational influence, as well as, an architecture and functionality of a model for the automatic evaluation of educational influence. The model’ s evaluation engine relies itself on methodology that has been originally developed for the intelligent tutoring systems effectiveness evaluation, and it will, generally, be applicable on any e-learning system, as the e-learning paradigm is closely related to the intelligent tutoring systems. That would enable calculation of overall e-learning systems effectiveness, as well as, effectiveness of different categories of e-learning systems, using meta-analysis.

CoLaB Tutor and AC-ware Tutor are Intelligent Tutoring Systems (ITSs) that are based on concept-based learning and are notable due to the fact they are relatively easy to generalize to multiple knowledge domains. In this research study we investigate the performance of CoLaB Tutor, AC-ware Tutor, and Moodle in a blended learning environment for an introductory computer programming course. In our study, regular face-to-face lectures and laboratory exercises were complemented with online learning at the students’ own pace, time and location. Our study revealed that CoLaB Tutor students had moder- ately higher knowledge gains than those students in the AC-ware and Moodle groups. The prediction of student success (pass/fail) for a basic knowledge post- test revealed an overall classification rate of 73, 5% for the CoLaB Tutor group (completed knowledge and online score as predictors), 71, 4% for the AC-ware group (completed knowledge as predictor) and 70% for the Moodle group (time spent online as predictor). Additionally, students that used ITSs on average passed through more knowledge online than students that used LMS, while students that used LMS on average spent more time online.

We present results of empirical evaluation of intelligent tutoring systems (ITS) with ontological domain knowledge representation. This research was done as a first step in the process of developing a new model of intelligent tutoring system that will include all the characteristics of evaluated systems: adaptive content, communication based on controlled natural language, graphical presentation of ontological domain knowledge representation. The case study results revealed extraordinary effectiveness of evaluated adaptive intelligent tutoring systems when compared with traditional learning and teaching process.

Since involvement of computers in educational, academic and business communities has increased, e-learning systems present involvement of information and communication technology as constituent part of education that supports a teacher in traditional learning and teaching process. In this paper, our focus is placed on learning materials in e-learning systems, how they are selected, sequenced and presented in a form of courseware. The main idea of this preliminary research related to intelligent courseware generation is in dynamic planning of courseware elements based on a student’s knowledge. A term “intelligent courseware generation” has not been used so far in this form and with the semantics described later on. Here, we define a multi-agent system model for intelligent courseware generation that would enable collaboration of different intelligent agents that would, eventually, enhance student’s learning and teaching process.

In this research we propose a comprehensive set of knowledge indicators aimed to enhance learners’ selfreflection and awareness in the learning and testing process. Since examined intelligent tutoring systems do not include additional messaging features, the introduction of common set of knowledge indicators differentiates our approach from the previous studies. In order to investigate the relation between proposed knowledge indicators and learner performance, the correlation and regression analysis were performed for 3 different courses and each examined intelligent tutoring system. The results of correlation and regression analysis, as well as learners’ feedback, guided us in discussion about the introduction of knowledge indicators in dashboard-like visualizations of integrated intelligent tutoring system.