Search

Systematic reviews are powerful methods used to determine the state-of-the-art in a given field from existing studies and literature. They are critical but time-consuming in research and decision making for various disciplines. When conducting a review, a large volume of data is usually generated from relevant studies. Computer-based tools are often used to manage such data and to support the systematic review process. This paper describes a comprehensive analysis to gather the required features of a systematic review tool, in order to support the complete evidence synthesis process. We propose a framework, elaborated by consulting experts in different knowledge areas, to evaluate significant features and thus reinforce existing tool capabilities. The framework will be used to enhance the currently available functionality of CloudSERA, a cloud-based systematic review tool focused on Computer Science, to implement evidence-based systematic review processes in other disciplines., This research was funded by the Spanish Research Agency (Agencia Estatal de Investigacion) with ERDF funds grant number TIN2017-85797-R (VISAIGLE project). The research stay of T. Person in the SFU was funded by Erasmus+ KA107 grant number 2017-1-ES01-KA107-037422. The APC was funded by the VISAIGLE project.

A visual programming language allows users and developers to create programs by manipulating program elements graphically. Several studies have shown the bene ts of visual languages for learning purposes and their applicability to robot programming. However, at present, there are not enough comparative studies on the suitability of textual and visual languages for this purpose. In this paper, we study if, as with a textual language, the use of a visual language could also be suitable in the context of robot programming and, if so, what the main advantages of using a visual language would be. For our experiments, we selected a sample of 60 individuals among students with adequate knowledge of procedural programming, that was divided into three groups. For the rst group of 20 students, a learning scenario based on a textual objectoriented language was used for programming a speci c commercial robotic ball with sensing, wireless communication, and output capabilities, whereas for the second and the third group, two learning scenarios based on visual languages were used for programming the robot. After taking a course for programming the robot in the corresponding learning scenario, each group was evaluated by completing three programming exercises related to the robot features (i.e. motion, lighting, and collision detection). Our results show that the students that worked with visual languages perceived a higher clarity level in their understanding of the course exposition, and a higher enjoyment level in the use of the programming environment. Moreover, they also achieved an overall better mark. Agencia Española de Investigación (AEI) TIN2017-85797-R

Virtual Worlds (VWs) have been widely used to support learning processes. One main advantage is providing valuable data on student behaviour and interaction. Nonetheless, most platforms provide only limited access to student logs. Moreover, accessing logs usually requires technical skills most teachers do not have. In this context, the authors present a Domain Specific Language (DSL) designed to allow teachers to generate queries for retrieving valuable log information with a view to obtain evidence on learner behaviour and interaction; hence, to aid in the analysis of in-world behaviour and learning processes. Since this data is automatically retrieved, the teacher can easily run new queries to refine indicators or contrast hypotheses. The authors describe a case study carried out with undergraduate German language students using a VW-based video game. The results provide a set of indicators for analysing individual and group behaviour measuring student competence to communicate in the target language.

When a legacy system’s software architecture cannot be redesigned, implementing additional privacy requirements is often complex, unreliable and costly to maintain. This paper presents a privacy-by-design approach to reengineer web applications as linked data-enabled and implement access control and privacy preservation properties. The method is based on the knowledge of the application architecture, which for the Web of data is commonly designed on the basis of a model-view-controller pattern. Whereas wrapping techniques commonly used to link data of web applications duplicate the security source code, the new approach allows for the controlled disclosure of an application’s data, while preserving non-functional properties such as privacy preservation. The solution has been implemented and compared with existing linked data frameworks in terms of reliability, maintainability and complexity.

The Internet of Things enables experts of given domains to create smart user experiences for interacting with the environment. However, development of such experiences requires strong programming skills, which are challenging to develop for non-technical users. This paper presents several extensions to the block-based programming language used in App Inventor to make the creation of mobile apps for smart learning experiences less challenging. Such apps are used to process and graphically represent data streams from sensors by applying map-reduce operations. A workshop with students without previous experience with Internet of Things (IoT) and mobile app programming was conducted to evaluate the propositions. As a result, students were able to create small IoT apps that ingest, process and visually represent data in a simpler form as using App Inventor's standard features. Besides, an experimental study was carried out in a mobile app development course with academics of diverse disciplines. Results showed it was faster and easier for novice programmers to develop the proposed app using new stream processing blocks., Spanish National Research Agency (AEI) - ERDF funds

Organizations need employees who perform satisfactorily in generic skills, such as teamwork, leadership, problem solving or interpersonal abilities, among others. In organizational environments, employees perform work that is not always visible for supervisors and, thus, they can hardly assess their performance in generic skills. By using a knowledge management system, the users are able to leave a trace of their activity in the system’s records. This research aims to address a computer supported assessment of the user’s generic skills from the perspective of Model-Driven engineering. First, a systematic mapping study is carried out to understand the state of the art. Second, a proposal based on Model-Driven engineering is presented and is then validated through an organizational learning process model. Our results are promising and we are able to conduct a scalable assessment based on objective indicators of the employee’s planning, time management and problem solving skills.

The heterogeneity of external systems that can be connected in an e-learning environment can impose interoperability and performance requirements for recording and storing the learning data. Web-based protocols have been developed to improve e-learning systems’ interoperability and capability to perform meaningful analytics. The present paper describes a web-based learning environment aimed at training how to command and control unmanned autonomous vehicles, provided with analytic capabilities. It integrates an external web content management system and a simulation engine that present different performance requirements for recording all significant events that occur during the learning process. Its record store construction, based on standard interoperability protocols, is explored here from the performance viewpoint. The tests that were conducted to assess regular data stores used for learning analytics show that performance should not be overlooked when constructing and deploying learning analytics systems.

Purpose In a project-based learning experience, the detailed monitoring of the activities in which team members participate can be useful to evaluate their work. However, the project activity produces a large amount of data that can be hardly assessed by a single project supervisor. This poses a scalability issue if the number of users or projects size increases. In this vein, the purpose of this paper is to make the assessment of online learning experiences more sustainable. Design/methodology/approach This paper describes a learning-oriented collaborative assessment method, supported by an open data framework. Also, an architecture for the extraction of different indicators to facilitate the assessment process is presented. Findings The assessment method and the open data framework were applied to a project-based course on web engineering. This experience has provided positive evidences because the grade measurement was backed up with assessment evidences and calculated with less effort. Research limitations/implications At the moment, results indicate that apparently there are no significant evidences against the sustainable evaluation practices for students’ summative evaluation. Nevertheless, when more data become available, a more statistically significant analysis could be made to determine the influence of the assessment practices in the final result of the evaluated skills. Originality/value In contrast to various existing proposals for e-assessment, the strategy focuses on assessing learning experiences in software development projects. Also, the approach is based on the reuse of information from external process supporting tools by integrating a number of metrics in a non-intrusive way.

This paper describes the design and validation of a game based on a platform for easy deployment of collaborative educational games, named BECO Games platform. As an example of its potential, a learning experience for an Economics subject was created through a collaborative game to understand the concept of common goods. The effectiveness of the game was tested by comparing the performance of Bachelor students who used the platform and those who did not (137 students vs. 92 students). In addition, it was controlled that in previous years when students played the game through forums and an Excel sheet, these differences did not exist. Results indicate that the performance differences between students who participated in the online game and those who did not were greater than in previous years. In addition, a satisfaction survey was delivered to the students to understand their impressions better. This survey assessed student opinion about the platform, about the educational experience, and about their behavior during the game.

In recent years, 360° videos combined with virtual reality (VR) techniques have been used to recreate highly interactive and immersive environments, allowing its users to directly experience and interact with its content and environment. Unlike traditional video recordings, which usually provide a very restrictive perspective of an object, person or environment, 360° videos provide users with full spherical views. In this paper, the authors share their recent experience with the design and implementation of a 360° video application. The application provides foreign language learners with an immersive learning environment that allows them to interact with the environment and its content, while practicing, at the same time, different language skills. The purpose of the study is to explore the possibilities of 360° video applications and chatbots for creating immersive environments in which language learners can foster their language competencies by means of real world-like situations.

The appropriate categorisation of written information by health professionals is very important to guarantee its accessibility. Unfortunately, the information technology tools that support professionals on that task imply a heavy workload, so that the responsibility for categorising the written content is often delegated to administrative staff. Well-known health ontologies such as SNOMED-CT or MeSH provide a representation of the clinical contents to be used by the information systems. This research proposes a computer based method to automatically extract and code the diagnostics, procedures and treatments according to health ontologies. A Knowledge Management System based on an extended version of Drupal is used to implement and evaluate this proposal. Results provide a positive evidence on the application of the method to support medical professionals.

A Higher Education degree is composed by courses which can be organized in areas or modules. Over last years, time invested by students to complete Higher Education degrees has increased. This increment can be caused by the existence of bottlenecks in the courses of academic programs. We aim to carry out an analysis of students’ performance to detect courses which represent bottlenecks in the process of completing a degree, because of many student failing compulsory courses. Students’ performance can be stored in data sets. Unfortunately, analysis of large data set can lead to scalability problems not being comprehensive applying manual analysis methods because of its extension. We applied Process Mining techniques to overtake these scalability problems. Process Mining is a set of Sequence Analysis techniques to analyze event logs. In this paper, we conducted an analysis of a data set which includes the performance of 612 students applying Process Mining tools. We obtained frequencies of students according time invested to complete a course. We compared these frequencies to detect possible bottlenecks. Finally, some requirements to consider courses as bottlenecks are proposed.

The acquisition of foreign language competencies has become one of the main concerns of current ICT educational policies. Mobile smart devices allow teachers to provide students with personalized learning environments in line with their needs. However, most of the available apps, especially in the area of foreign language learning, still focus on form-based learning supporting mainly one-way interaction. In this chapter, the authors designed a learning system based on a dynamic, asynchronous and constructive learning approach. The chapter illustrates how the system helped students to get involved in their learning process by creating, sharing, and assessing their own learning resources and how teachers could benefit from students' logs to retrieve indicators for assessment processes. Finally, two algorithms that guide students' learning processes are compared: the first algorithm is based on community-driven behaviour, the second one on students' individual behaviour. Results show that both algorithms provide similar outcomes.

One of the most valuable skills for teachers is the ability to produce their own digital solutions, translating teaching concepts into end-user computer systems. This often requires the involvement of computing specialists. As a result, the development of educational programming environments remains a challenge. Learning experiences based multimodal interaction applications (gesture interaction, voice recognition or artificial vision) are becoming commonplace in education because they motivate and involve students. This chapter analyses the state-of-the-art in LA techniques and user-friendly authoring tools. It presents a tool to support the creation of multimodal interactive applications equipped with non-intrusive monitoring and analytics capabilities. This tool enables teachers with no programming skills to create interactive LA-enriched learning scenarios. To this end, several components that manage LA activities are included in the tool, they range from automatically capturing users’ interaction with mobile applications, to querying data and retrieving metrics, to visualising tables and charts.

ContextSoftware Process Engineering promotes the systematic production of software by following a set of well-defined technical and management processes. A comprehensive management of these processes involves the accomplishment of a number of activities such as model design, verification, validation, deployment and evaluation. However, the deployment and evaluation activities need more research efforts in order to achieve greater automation. ObjectiveWith the aim of minimizing the required time to adapt the tools at the beginning of each new project and reducing the complexity of the construction of mechanisms for automated evaluation, the Software Process Deployment & Evaluation Framework (SPDEF) has been elaborated and is described in this paper. MethodThe proposed framework is based on the application of well-known techniques in Software Engineering, such as Model Driven Engineering and Information Integration through Linked Open Data. It comprises a systematic method for the deployment and evaluation, a number of models and relationships between models, and some software tools. ResultsAutomated deployment of the OpenUP methodology is tested through the application of the SPDEF framework and support tools to enable the automated quality assessment of software development or maintenance projects. ConclusionsMaking use of the method and the software components developed in the context of the proposed framework, the alignment between the definition of the processes and the supporting tools is improved, while the existing complexity is reduced when it comes to automating the quality evaluation of software processes.

Nowadays, virtual learning environments are developed as digital ecosystems based on existing resources, applications, and web services. Even if they are not hosted in a centralized course management system, they are usually highly coupled. In this paper, we show how semantic technologies can help decoupling them. We build an e-learning web ecosystem enriched with an educational information according to an educational information model and a domain-specific model, so we provide teachers and students with a common user interface. We implemented our proposal in the ASCETA project, integrating well-known open-source systems, such as wikis, blogs, microblogging services, content management systems, and task management systems.

SUMMARY Software process engineering is a discipline, which aims to study and improve software development and maintenance processes. The explicit definition of software processes is essential. To this end, the Object Management Group consortium proposed the Software & Systems Process Engineering Meta-Model (SPEM) that exploits the benefits of the Model Driven Architecture paradigm applied to software process models, instead of software specification models. The aim of this study is to discover evidence clusters and evidence deserts in the use and application of SPEM from a business process management point of view. To reach the proposed objective, we have undertaken a systematic mapping study of the existing scientific literature. The reviewed literature deals mainly with process modeling and, to a lesser extent, with process adaptability, verification, and validation, enactment and evaluation. Wide agreement exists in using the SPEM meta-model to develop different types of methods and processes. Further research efforts are needed in areas related to enactment and evaluation of software processes. There is a need to evolve to a new version of the meta-model that incorporates the improvements proposed by different authors. Copyright © 2013 John Wiley & Sons, Ltd.

With increasing globalization, foreign language skills have become one of the main requirements when applying for a job in leading ICT companies. Due to their ubiquity and multi-functionality, mobile devices allow teachers to provide technology-friendly students with highly dynamic learning contents in line with their lifestyle. Unfortunately, most of the available APPs still support a mainly one-way interaction (teacher to learner or computer-client interaction). The authors have designed an APP based on a highly interactive, ubiquitous and constructive learning approach. The current paper illustrates firstly how the APP has helped learners to get actively involved in their own learning process sharing and assessing their foreign language knowledge. And secondly, how students' logs can be used to automate the assessment of different skills such as the ability to explain terms in a foreign language or to assess definitions created by other game players.

Learning experiences based on multimodal interactive applications are becoming common at all educational levels. Designing assessments for learning applications is often addressed through learning analytics. Multimodal interactive applications generate a large amount of data about students' interaction that can provide insights about their profile, behavior and performance. Unfortunately, this information is usually not accessible or difficult to collect from such applications, especially for teachers without computer programming skills. In this work, we present a visual development environment that supports the creation of multimodal interactive applications for learning with non-intrusive monitoring capacities, thus providing teachers the opportunity to create their own learning analytics tools even if they are not skilled in computer programming.

With increasing globalization, foreign language skills have become one of the main requirements when applying for a job in leading ICT companies. Due to their ubiquity and multi-functionality, mobile devices allow teachers to provide technology-friendly students with highly dynamic learning contents in line with their lifestyle. Unfortunately, most of the available APPs still support a mainly one-way interaction (teacher to learner or computer-client interaction). The authors have designed an APP based on a highly interactive, ubiquitous and constructive learning approach. The current paper illustrates firstly how the APP has helped learners to get actively involved in their own learning process sharing and assessing their foreign language knowledge. And secondly, how students' logs can be used to automate the assessment of different skills such as the ability to explain terms in a foreign language or to assess definitions created by other game players.

The widespread adoption of computers and Internet in our life has reached the classrooms, where computer-supported collaborative learning (CSCL) based on wikis offers new ways of collaboration and encourages student participation. When the number of contributions from students increases, traditional assessment procedures of e-learning settings suffer from scalability problems. In a wiki-based learning experience, automatic tools are required to support the assessment of such huge amounts of data. In this work, we present StatMediaWiki, a tool that collects and aggregates information that helps to analyze a MediaWiki installation. It generates charts, tables and different statistics enabling easy analysis of wiki evolution. We have used StatMediaWiki in a Higher Education course and present the results obtained in this case study.

Research Information Systems (RIS) play a critical role in the sharing of scientific information and provide researchers, professionals and decision makers with the required data for their activities. Existing RIS standards have proposed data models to represent the main entities for storage and exchange. These account for the needs of multiple stakeholders through a high flexibility based on a formal syntax and declared semantics, but for techno-historical reasons they assume the completeness of information within system boundaries. The distributed nature of research information across systems calls for a mechanism to link the local entities from the closed world of concrete RISs with other possibly underspecified entities exposed through other means, as for example, the Linked Open Data Web. By transformation of a relational model into an open graph model, differences between the two system paradigms are revealed. The main principles and techniques for exposing CERIF-driven relational data as linked data will be provided as a first step demonstrating effective RISs interconnection through the linked open data (LOD) Web.

In a project-based learning experience, the detailed monitoring of the activities in which team members participate can be useful to evaluate their work. Using learning-oriented assessment procedures, supervisors can assess the teamwork abilities with a formative purpose. Evaluation strategies such as self-assessment, peer assessment and co-assessment are often used to make evaluation formative and sustainable. Conducting an assessment strategy is not easy for team members, since they need before to have a reasonable understanding of the evaluation process and criteria. This paper describes a learning-oriented evaluation methodology and an open data framework that can be applied to collaborative project settings. An evaluation rubric and a series of indicators that provide evidences about the developed skills have been elaborated and applied in a small-scale project-based course. Projects were managed and developed with the help of an open source software forge that contains a ticketing tool for planning and tracking of tasks, a version control repository to save the software outcomes, and using a wiki to host text deliverables. The experience provides evidences in favor of using the assessment method and open data framework to make teamwork evaluation more sustainable., En las experiencias de aprendizaje basadas en proyectos, la monitorización detallada de las actividades de los miembros de cada equipo puede resultar de utilidad para la evaluación de su trabajo. Mediante procedimientos de evaluación, los supervisores pueden evaluar las capacidades de trabajo en equipo con una finalidad formativa. Suelen aplicarse estrategias como la auto-evaluación, la evaluación entre iguales y la co-evaluación para hacer que la evaluación formativa sea sostenible. Seguir una estrategia de evaluación no es siempre sencillo para los miembros de los equipos, pues necesitan un conocimiento razonable del proceso y los criterios de evaluación. Este artículo describe una metodología de evaluación orientada al aprendizaje y un framework software de datos abiertos aplicable a la evaluación de proyectos de desarrollo colaborativo. Se han elaborado una rúbrica y una serie de indicadores que proporcionan evidencias sobre las habilidades y, posteriormente, se han aplicado a un curso basado en proyectos de pequeña escala. Los proyectos se gestionaron y ejecutaron con la ayuda de una forja de software libre que incluye una herramienta de gestión de tareas para la planificación y el seguimiento, un repositorio de control de versiones para almacenar el software entregable y una wiki para almacenar los entregables textuales. La experiencia ha proporcionado evidencias a favor del método de evaluación y el framework de datos abiertos para hacer la evaluación del trabajo en equipo más sostenible.

Virtual worlds (VWs) have become increasingly popular to support students' foreign language learning, especially beyond the classroom. Unfortunately students' interaction in VWs is not always available for the supervisor and thus is not easy to analyse. Nonetheless, it provides interesting information not only in terms of assessment, but also to detect general learner profiles and trends in their use. Therefore we propose a computer system that accepts queries in a simple but specific language for VWs, allowing the supervisor to design assessment formulas, generate reports on students' interaction in VWs and refine the formulas from these results until obtaining a valid indicator for analysing and assessing the retrieved data. In this paper, we describe a case study which has been carried out in a German foreign language course, using a VW implemented video-game and assessing students' foreign language skills. The study has been deployed in a scalable way through our computer system. Preliminary results provide a set of indicators for individual and group behaviour which can be used to assess student's ability to communicate in the target language.

The focus on assessment of learning experiences has shifted from knowledge to competences. Unfortunately, assessing certain competences is mainly a subjective task, being problematic for both the evaluators and the evaluated. Additionally, when the learning process is computer-supported and the number of students increases, traditional assessment procedures suffer from scalability problems. In this paper we introduce a system that supports grading learning competences according to students' performance in an online course. We automatically extract different objective indicators about students' work in a Learning Management System (LMS). Evaluators can use an assessment-specific query language to express a number of required indicators. Such indicators are automatically extracted from the activity logs generated by the LMS. A case study with Moodle LMS-based courses is carried out to explain how such indicators can be obtained and how to analyze the assessment results.

In a project-based learning experience, the detailed monitoring of the activities in which team members participate can be useful to evaluate their work. Using learning-oriented assessment procedures, supervisors can assess the teamwork abilities with a formative purpose. Evaluation strategies such as self-assessment, peer assessment and co-assessment are often used to make evaluation formative and sustainable. Conducting an assessment strategy is not easy for team members, since they need before to have a reasonable understanding of the evaluation process and criteria. This paper describes a learning-oriented evaluation methodology and a open data framework that can be applied to collaborative software development project settings. An evaluation rubric and a series of indicators that provide evidences about the developed skills have been elaborated and applied in a small-scale project-based course on Web Engineering. Projects were managed and developed with the help of an open source software forge that contains a ticketing tool for planning and tracking of tasks, a version control repository to save the software deliverables, and using a wiki to host text deliverables. The experience provides evidences in favor of using the assessment method and open data framework to make teamwork evaluation more sustainable.