Search

In this paper, a novel approach to automatic question generation (AQG) using semantic role labeling (SRL) for morphologically rich languages is presented. A model for AQG is developed for our native speaking language, Croatian. Croatian language is a highly inflected language that belongs to Balto-Slavic family of languages. Globally this article can be divided into two stages. In the first stage we present a novel approach to SRL of texts written in Croatian language that uses Conditional Random Fields (CRF). SRL traditionally consists of predicate disambiguation, argument identification and argument classification. After these steps most approaches use beam search to find optimal sequence of arguments based on given predicate. We propose the architecture for predicate identification and argument classification in which finding the best sequence of arguments is handled by Viterbi decoding. We enrich SRL features with custom attributes that are custom made for this language. Our SRL system achieves F1 score of 78% in argument classification step on Croatian hr 500k corpus. In the second stage the proposed SRL model is used to develop AQG system for question generation from texts written in Croatian language. We proposed custom templates for AQG that were used to generate a total of 628 questions which were evaluated by experts scoring every question on a Likert scale. Expert evaluation of the system showed that our AQG achieved good results. The evaluation showed that 68% of the generated questions could be used for educational purposes. With these results the proposed AQG system could be used for possible implementation inside educational systems such as Intelligent Tutoring Systems.

Various approaches to text simplification have been proposed in an attempt to increase text readability. The rephrasing of syntactically and semantically complex structures is still challenging. A pedagogically motivated simplified version of the same text can have both positive and negative side effects. On the one hand, it can facilitate reading comprehension because of much shorter sentences and a limited vocabulary, but on the other hand, the simplified text often lacks coherence, unity and style. Therefore, reasonable trade-offs among linguistic simplicity, naturalness and informativeness are highly needed. This is a survey paper that discusses state-of-the-art approaches to sentence/text simplification and evaluation methods, along with an empirical evaluation of our approach. The quality of sentence splitting, using the knowledge extraction tool SAAT was compared to state-of-the-art syntactic simplification systems. The research was carried out on the WikiSplit, the HSplit and MinWikiSplit simplification corpora. Automatic metrics for the HSplit showed that the SAAT outperformed other TS systems in all categories. For the WikiSplit dataset, automatic metrics scores were slightly lower than that of the baseline system DisSim. However, the human evaluation showed that DisSim outperformed the SAAT in terms of simplicity and grammar. The quality of AG18copy output corresponded to that of the SAAT. The inter-annotator agreement was calculated. Research limitations as well as suggestions for future research were also provided.

In this article state of the art semantic role labeling (SRL) tools have been compared on published lexical resources and custom hand annotated previously unseen texts. Semantic role labeling is one of the first steps into understanding the meaning of the sentence. The process of identifying semantic roles includes several steps from which the main are predicate identification, argument identification and argument classification. All the approaches compared in this article suggest supervised machine learning algorithms on lexical resources. The comparison of the tools for semantic role labeling are tested against hand annotated corpora independent of FrameNet or PropBank corpus. The precision, recall tests were made on PropBank data as well. Because the results of classification can be biased, the objective of this research was to comprise all the mentioned tools with hand annotated previously unseen data. The results have shown that mate-tools has achieved best results on independent hand annotated data with the best micro average F-score of 78% on custom hand annotated dataset. Results from this article were used to evaluate which tool can be used for knowledge extraction tasks and question generation in communication module of intelligent tutoring system. The overview of all freely available tools, mostly from classifier implementation point of view is given in this article.

In this article we present an knowledge extraction approach that can be used in systems that implement teaching in a fully automated manner. These systems are called Intelligent Tutoring Systems (ITS) and are conceived around the idea of one-to-one teaching. Many such systems use natural language processing to improve the communication interface between student and the system. These techniques can be also used on the content creator side to semi- automate or fully automate the task of teaching content creation. In such systems the knowledge representation plays a crucial role to successfully implement teaching and encourage learning. The output of the knowledge extraction phase is a knowledge in the form of a hyper graph that can be used for adaption to the students current knowledge level. We present a deep neural network architecture for precise POS tagging of words written in languages that are morphologically rich. Using sparse representations for words in this task increases the vector space and makes learning more complex. This problem can be solved to some extent by using traditional vector representations but there is also the problem with representing words that are ambiguous. Proposed architecture uses a Bidirectional Encoder Representations from Transformers (BERT) model that is pre-trained on Croatian language to achieve state-of-the-art accuracy for POS tagging.

Finding similarity of textual data is very important task in natural language processing. In this article we present approach to finding similarity of words, paragraphs, sentences and documents. Semantic similarity is one of the central tasks in many applications, including text summarization, Intelligent Tutoring Systems (ITS) etc. In ITS sentence similarity is used to compare the student’s response with the correct answer. The result is used to gain information about student’s level of knowledge. We propose three different methods that measure text to text semantic relatedness. There are multiple approaches to finding the right measure to determine the similarity of the sentences. Some measure the alignment of characters, and other measure semantic similarity between sentences. In this work we present and evaluate methods for finding not just similarity of sentences but even also similarity of whole paragraphs and documents. We have evaluated these methods using the data from the Yahoo Question and Answer of the Non-Factual Data Set.

The reasoning process about the level of student’s knowledge can be challenging even for experienced human tutors. The Bayesian networks are a formalism for reasoning under uncertainty, which has been successfully used for various artificial intelligence applications, including student modeling. While Bayesian networks are a highly flexible graphical and probabilistic modeling framework, its main challenges are related to the structural design and the definition of “a priori” and conditional probabilities. Since the AC&NL Tutor’s authoring tool automatically generates tutoring elements of different linguistic complexity, the generated sentences and questions fall into three difficulty levels. Based on these levels, the probability- based Bayesian student model is proposed for mastery-based learning in intelligent tutoring system. The Bayesian network structure is defined by generated questions related to the node representing knowledge in a sentence. Also, there are relations between inverse questions at the same difficulty level. After the structure is defined, the process of assigning “a priori” and conditional probabilities is automated using several heuristic expert-based rules.

Automatic knowledge acquisition is a rather complex and challenging task. This paper focuses on the description and evaluation of a semi-automatic authoring tool (SAAT) that has been developed as a part of the Adaptive Courseware based on Natural Language AC&NL Tutor project. The SAAT analyzes a natural language text and, as a result of the declarative knowledge extraction process, it generates domain knowledge that is presented in a form of natural language sentences, questions and domain knowledge graphs. Generated domain knowledge presents expert knowledge in the intelligent tutoring system Tutomat. The natural language processing techniques are applied and the tool’s functionalities are thoroughly explained. This tool is, to our knowledge, the only one that enables natural language question and sentence generation of different levels of complexity. Using an unstructured and unprocessed Wikipedia text in computer science, evaluation of domain knowledge extraction algorithm, i.e. the correctness of extraction outcomes and the effectiveness of extraction methods, was performed. The SAAT outputs were compared with the gold standard, manually developed by two experts. The results showed that 68.7% of detected errors referred to the performance of the integrated linguistic resources, such as CoreNLP, Senna, WordNet, whereas 31.3% of errors referred to the proposed extraction algorithms.

Over the last few decades, researchers put efforts to improve intelligent tutoring systems' abilities with the aim to get them as close as possible to the ultimate goal of one-to-one tutoring. CoLaB Tutor and AC-ware Tutor are intelligent tutoring systems based on conceptual knowledge learning and are notable due to the fact they are relatively easy to generalize to multiple knowledge domains. CoLaB Tutor's forte lies in teacher-learner communication in controlled natural language, while AC-ware Tutor focuses on the automatic and dynamic generation of adaptive courseware. In order to compare various intelligent tutoring system supported education environments, in this chapter, the authors summarize several empirical evaluations of CoLaB Tutor and AC-ware Tutor. The results of intelligent tutoring systems' effectiveness in these environments offer the possibility to observe the specific intelligent tutoring system across various education levels, as well as to compare the intelligent tutoring systems' supported education environments.

Recent advancements in natural language processing (NLP) improved many systems that were relying on natural language to achieve better communication with user using this system. One of the main problem in application of NLP in multiple languages is lack of tools that can be used to develop such systems. Croatian language is highly inflected language from Slavic language family and traditional models used that give great results for English language behave poorly for morphology rich languages. In this article we present model for creating word embeddings for morphologically rich languages such as Croatian. We evaluate the generated word embeddings on newly created word similarity corpus, that is based on English similarity corpus. In the evaluation of word embeddings we compare with two of the best word representation models for English language. We also evaluate our approach with multi-language models such as FastText. The word embeddings created in this article will be used for developing component in training neural models for semantic understanding of sentences written in Croatian language. These language tools can be utilized in many systems where natural language understanding (NLU) and natural language generation (NLG) is needed. In the introduction we give global insight about word embeddings, what are the models for creating such representations and where these representations could be used. In the second section we mention some of the best models for creating word embeddings. In the third section we give a frame-work for development and evaluation of word embeddings for Croatian language. In the conclusion we emphasis the importance of developing tools in Croatian language and announcement of future research.

This article presents an approach to examining the similarity of the sentences. In our approach, Euler algorithm was used to generate a series of words based on tree and S⊘rensen-Dice coefficient was applied to determine the similarity between compared trees. The emphasis is on defining the similarity between the correct and incorrect answers from the Yahoo Question and Answer of the Non-Factual Data Set. Proposed algorithm was used on two types of trees. First is the constituency tree generated by Stanford CoreNLP, and second is custom-made algorithm that produces second type of tree, called knowledge tree which is derived from parse tree. In our comparison, Zhuang-Sasha algorithm was also used. Second approach that was used for sentence comparison uses Word2Vec model for finding word embedding's and calculating sentence average vector, after that cosine distance was applied to determine similarity between two sentences. Results generated with this method were compared with our method in finding sentence similarity based on knowledge tree. Approach described in this paper can be used in evaluation of correct answers which will be used in our implementation of Intelligent Tutoring System.

The crucial information about learner in an E learning system is in fact the information about learner’s level of knowledge. In everyday practice teacher must know what the student knows to adapt to learner’s individual needs, and his features. In adaptive e learning system, the system has to have information about student’s knowledge to implement learning strategies to achieve maximum effect. Many researches show that learning using e learning system shows best learning effects. This article shows how to use semi-automatic tool for teaching main concepts of programming using concept map. Concept map is used as an ontology that teachers use to construct domain knowledge that is used to asses’ student’s knowledge and construct student model. Based on student model system decides which concepts to include in teaching process. We utilize CM Tutor (Content Modeling Tutor) (Volaric in Oblikovanje modela nastavnih lekcija u inteligentnom sustavu e-ucenja. Split, 2014) module to evaluate student’s performance through qualitative and quantitative means. Experiments are made on 2 generations on students of University of Mostar to evaluate the systems effectiveness and improve systems performance based on student’s feedback.

This paper describes an approach for modeling and recognition of human actions within videos. With millions of videos that are published almost every day, there are new opportunities for research in the field of search and recognition within the video sequence. Statistical approaches and approaches based on the description of the model are described in detail in this paper and compared to a series of videos taken from various on-line databases (KTH, Weizmann, MSR-Action). There are various approaches to identify actions within video sequences. Approaches that are described within this paper are based on recognition of the action of a series of images obtained segmentation and motion picture history by constructing movement (Motion History Images MHI). In this paper we apply the technique to construct MHI on a series of images obtained from the database used for the analysis of movement in order to recognize the action within a video (greeting of human in video).

Teaching is complex activity which requires professors to employ the most effective and efficient teaching strategies to enable students to make progress. Main problem in teaching professors should consider different teaching approaches and learning techniques to suit every student. Today, in computer age, electronic learning (e-learning) is widely used in practice. Development of World Wide Web, especially Web2.0 has led to revolution in education. Student interaction with Learning management systems - LMS result in creating large data sets which are interesting for research. LMS systems also provide tools for following every individual student and statistical view for deeper analyzing result of student - system interaction. However, these tools do not include artificial intelligence algorithms as a support mechanism for decision. In this article we provide framework for student modeling trained on large sets of data using Hadoop and Mahout. This kind of system would provide insight into each individual student's activity. Based on that information, professors could adjust course materials according to student interest and knowledge.

In this research we propose a comprehensive set of knowledge indicators aimed to enhance learners’ selfreflection and awareness in the learning and testing process. Since examined intelligent tutoring systems do not include additional messaging features, the introduction of common set of knowledge indicators differentiates our approach from the previous studies. In order to investigate the relation between proposed knowledge indicators and learner performance, the correlation and regression analysis were performed for 3 different courses and each examined intelligent tutoring system. The results of correlation and regression analysis, as well as learners’ feedback, guided us in discussion about the introduction of knowledge indicators in dashboard-like visualizations of integrated intelligent tutoring system.