Search

VirtualWorlds (VWs) are popular tools for teaching/learning in the twenty-first century classroom. The challenge remains however, to provide the means by which teachers could sustainably analyse and assess the performance of large groups of students in such environments. Unfortunately, external game features such as game scores and play duration have turned out to be unfair in some assessments. In this context, a case study was carried out in a foreign language course, illustrating how teachers could easily retrieve a number of performance indicators from VW-interaction logs and harness them to conduct a fine-grained analysis of students’ performance, while facilitating at the same time valuable tools for their assessment. Objective performance indicators in a server database were made accessible using an end-user development programming language. This way, a range of data visualisation methods could be employed to contrast different assumptions regarding learner performance when playing a VW-based game, which was designed to help CEFR A1 level students to learn German. This way, factors such as randomisation of game tasks, which could negatively affect learner performance, were alleviated., This research was funded by Spanish National Research Agency (AEI), through the projects VISAIGLE (TIN2017-85797-R) and DeepBio (TIN2017-85727-C4-3-P) with ERDF funds more...

© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works., Software engineering uses models throughout most phases of the development process. Models are defined using modelling languages. To make these languages applicable to a wider set of scenarios and customizable to specific needs, researchers have proposed using product lines to specify modelling language variants. However, there is currently a lack of efficient techniques for ensuring correctness with respect to properties of the models accepted by a set of language variants. This may prevent detecting problematic combinations of language variants that produce undesired effects at the model level. To attack this problem, we first present a classification of instantiability properties for language product lines. Then, we propose a novel approach to lifting the satisfiability checking of model properties of individual language variants, to the product line level. Finally, we report on an implementation of our proposal in the Merlin tool, and demonstrate the efficiency gains of our lifted analysis method compared to an enumerative analysis of each individual language variant, This work has been funded by the Spanish Ministry of Science (RTI2018-095255-B-I00), the R&D programme of Madrid (P2018/TCS-4314), and by NSERC. We thank the anonymous referees for their useful comments more...

Falls can have serious consequences for people, leading to restrictions in mobility or, in the worst case, to traumatic-based cases of death. To provide rapid assistance, a portable fall detection system has been developed that is capable of detecting fall situations and, if necessary, alerting emergency services without any user interaction. The prototype is designed to facilitate reliable fall detection and to classify several fall types and human activities. This solution represents a life-saving service for every person that will significantly improve assistance in the case of fall events, which are a part of daily life. Additionally, this approach facilitates independent system operation, since the system does not depend on sensor or network units located within a building structure. This article also introduces fall analysis. To guarantee functional safety, a hazard analysis method named system-theoretic accident model and processes (STAMP) is applied. more...

© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, Chatbots are programs that supply services to users via conversation in natural language, acting as virtual assistants within social networks or web applications. Here, we review the most representative chatbot development tools with a focus on technical and managerial aspects, This work was partially funded by the R&D program of the Madrid Region (project FORTE, S2018/TCS4314), and the Spanish Ministry of Science (project MASSIVE, RTI2018-095255-B-I00) more...

The Internet of Things (IoT) paradigm keeps growing, and many different IoT devices, such as smartphones and smart appliances, are extensively used in smart industries and smart cities. The benefits of this paradigm are obvious, but these IoT environments have brought with them new challenges, such as detecting and combating cybersecurity attacks against cyber-physical systems. This paper addresses the real-time detection of security attacks in these IoT systems through the combined used of Machine Learning (ML) techniques and Complex Event Processing (CEP). In this regard, in the past we proposed an intelligent architecture that integrates ML with CEP, and which permits the definition of event patterns for the real-time detection of not only specific IoT security attacks, but also novel attacks that have not previously been defined. Our current concern, and the main objective of this paper, is to ensure that the architecture is not necessarily linked to specific vendor technologies and that it can be implemented with other vendor technologies while maintaining its correct functionality. We also set out to evaluate and compare the performance and benefits of alternative implementations. This is why the proposed architecture has been implemented by using technologies from different vendors: firstly, the Mule Enterprise Service Bus (ESB) together with the Esper CEP engine; and secondly, the WSO2 ESB with the Siddhi CEP engine. Both implementations have been tested in terms of performance and stress, and they are compared and discussed in this paper. The results obtained demonstrate that both implementations are suitable and effective, but also that there are notable differences between them: the Mule-based architecture is faster when the architecture makes use of two message broker topics and compares different types of events, while the WSO2-based one is faster when there is a single topic and one event type, and the system has a heavy workload., This work was supported by the Spanish Ministry of Science, Innovation and Universities and the European Union FEDER Funds [grant numbers FPU 17/02007, RTI2018-093608-B-C33, RTI2018-098156-B-C52 and RED2018-102654-T] . This work was also supported by the JCCM [grant number SB-PLY/17/180501/000353] and the Research Plan from the University of Cadiz and Grupo Energetico de Puerto Real S.A. under project GANGES [grant number IRTP03' UCA] . The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. more...

Competitions are being widely used to motivate students in diverse learning processes, including those in computer programming. This paper presents a methodology for designing and assessing competitive learning scenarios that allow students to develop three different coding skills: the ability to compete against unknown competitors, the ability to compete against known competitors and the ability to compete against refined versions of known competitors. The proposal is based on peer code review, implemented as an improvement cycle after the dissemination of the code among participants. A case study evaluating the methodology was conducted with two cohorts of students in an undergraduate course. The analysis of the obtained grades suggests that while performance after our assistance was improved, students could still fail or succeed independently of the assistance. Complementary data from student questionnaires and supervisor observations are aligned with this finding. As a conclusion, the evidence supports the validity of the methodology. Additionally, several guidelines based on the experience are provided to transfer the proposal to other environments., This research was funded by Spanish National Research Agency (AEI), through the project VISAIGLE (TIN2017-85797-R) with ERDF funds. more...

14 páginas; 11 tablas; 1 figura, Nowadays, information and communications technology systems are fundamental assets of our social and economical model, and thus they should be properly protected against the malicious activity of cybercriminals. Defence mechanisms are generally articulated around tools that trace and store information in several ways, the simplest one being the generation of plain text files coined as security logs. Such log files are usually inspected, in a semi-automatic way, by security analysts to detect events that may affect system integrity, confidentiality and availability. On this basis, we propose a parameter-free method to detect security incidents from structured text regardless its nature. We use the Normalized Compression Distance to obtain a set of features that can be used by a Support Vector Machine to classify events from a heterogeneous cybersecurity environment. In particular, we explore and validate the application of our method in four different cybersecurity domains: HTTP anomaly identification, spam detection, Domain Generation Algorithms tracking and sentiment analysis. The results obtained show the validity and flexibility of our approach in different security scenarios with a low configuration burden., This research has received funding from the European Union’s Horizon 2020 Research and Innovation Programme under grant agreement No. 872855 (TRESCA project), from the Comunidad de Madrid (Spain) under the projects CYNAMON (P2018/TCS-4566) and S2017/BMD-3688, co-financed with FSE and FEDER EU funds, by the Consejo Superior de Investigaciones Científicas (CSIC) under the project LINKA20216 (“Advancing in cybersecurity technologies”, i-LINK+ program), and by Spanish project MINECO/FEDER TIN2017-84452-R. more...

Cloud computing has emerged as a cutting-edge technology which is widely used by both private and public institutions, since it eliminates the capital expense of buying, maintaining, and setting up both hardware and software. Clients pay for the services they use, under the so-called Service Level Agreements (SLAs), which are the contracts that establish the terms and costs of the services. In this paper, we propose the CloudCost UML profile, which allows the modeling of cloud architectures and the users’ behavior when they interact with the cloud to request resources. We then investigate how to increase the profits of cloud infrastructures by using price schemes. For this purpose, we distinguish between two types of users in the SLAs: regular and high-priority users. Regular users do not require a continuous service, so they can wait to be attended to. In contrast, high-priority users require a constant and immediate service, so they pay a greater price for their services. In addition, a computer-aided design tool, called MSCC (Modeling SLAs Cost Cloud), has been implemented to support the CloudCost profile, which enables the creation of specific cloud scenarios, as well as their edition and validation. Finally, we present a complete case study to illustrate the applicability of the CloudCost profile, thus making it possible to draw conclusions about how to increase the profits of the cloud infrastructures studied by adjusting the different cloud parameters and the resource configuration., This work was supported by the Spanish Ministry of Science and Innovation (co-financed by European Union FEDER funds) project “FAME (Formal modeling and advanced testing methods. Applications to medicine and computing systems) and MASSIVE (Engineering adaptive software by and for the people in a highly connected world)”, references RTI2018-093608-B-C32 and RTI2018-095255-B-I00. There was also support from the Junta de Comunidades de Castilla-La Mancha project SBPLY/17/180501/000276/ 01 (cofunded with FEDER funds, EU), the Region of Madrid (grant number FORTE-CM, S2018/TCS-4314), and the Madrid Government (Comunidad de Madrid-Spain) under the Multiannual Agreement with the Complutense University as part of the Program to Stimulate Research for Young Doctors in the context of the V PRICIT (Regional Programme of Research and Technological Innovation) under grant PR65/19-22452 more...

Mutation testing is becoming increasingly widely used to evaluate the quality of test suites, especially to test programs coded in widely used programming languages in the industry. Mutation tools have arisen to automate the technique in different languages, including C++. With the increasing use of this technique, new mutation operators modeling possible faults often emerge to improve its abilities and adapt the tools to new advanced features. In this work, mutation operators for the new C++ standards, defined in previous work, are implemented and applied to generate and execute mutants in real programs. With this study, the MuCPP mutation tool is updated with the inclusion of these new operators. In addition, the improvements suggested in the definition of those operators can be finally tested, and conclusions about their utility in practice can be drawn. The implemented operators are checked on a set of four C++ programs that use these advanced features. The results show significant differences with the previous manual analysis: the number of invalid mutants was reduced by 64%, and we found fewer alive mutants (88%) and an increase in dead mutants (31%). In summary, both the number of mutants incorrectly classified in the previous manual analysis and the number of mutants generated (particularly equivalent mutants) have been reduced. more...

Modelling is an essential activity in software engineering. Ittypically involves two meta-levels: one includes meta-models thatdescribe modelling languages, and the other contains models built byinstantiating those meta-models. Multi-level modelling generalizes this approach by allowing models to span an arbitrarynumber of meta-levels. A scenario that profits from multi-levelmodelling is the definition of language families that can bespecialized (e.g., for different domains) by successive refinementsat subsequent meta-levels, hence promoting language reuse. Thisenables an open set of variability options given by allpossible specializations of the language family. However,multi-level modelling lacks the ability to express closed variability regarding the availability of language primitives or thepossibility to opt between alternative primitive realizations. Thislimits the reuse opportunities of a language family. To improve thissituation, we propose a novel combination of product lines withmulti-level modelling to cover both open and closed variability. Ourproposal is backed by a formal theory that guarantees correctness,enables top-down and bottom-up language variability design, and isimplemented atop the MetaDepth multi-level modelling tool, This work has been funded by the SpanishMinistry of Science (project MASSIVE, RTI2018-095255- B-I00), by the R&D programme of Madrid (project FORTE, P2018/TCS-4314), and the Universidad Autónoma de Madrid more...

Machine learning methods are growing in relevance for biometrics and personal information processing in domains such as forensics, e-health, recruitment, and e-learning. In these domains, white-box (human-readable) explanations of systems built on machine learning methods become crucial. Inductive logic programming (ILP) is a subfield of symbolic AI aimed to automatically learn declarative theories about the processing of data. Learning from interpretation transition (LFIT) is an ILP technique that can learn a propositional logic theory equivalent to a given black-box system (under certain conditions). The present work takes a first step to a general methodology to incorporate accurate declarative explanations to classic machine learning by checking the viability of LFIT in a specific AI application scenario: fair recruitment based on an automatic tool generated with machine learning methods for ranking Curricula Vitae that incorporates soft biometric information (gender and ethnicity). We show the expressiveness of LFIT for this specific problem and propose a scheme that can be applicable to other domains. In order to check the ability to cope with other domains no matter the machine learning paradigm used, we have done a preliminary test of the expressiveness of LFIT, feeding it with a real dataset about adult incomes taken from the US census, in which we consider the income level as a function of the rest of attributes to verify if LFIT can provide logical theory to support and explain to what extent higher incomes are biased by gender and ethnicity, This work was supported by projects: PRIMA (H2020-MSCA-ITN-2019-860315), TRESPASS-ETN(H2020-MSCA-ITN-2019-860813), IDEA-FAST (IMI2-2018-15-853981), BIBECA(RTI2018-101248-B-I00MINECO/FEDER), RTI2018-095232-B-C22MINECO, PLeNTaS project PID2019-111430RBI00MINECO; and also by Pays de la Loire Region through RFI Atlanstic 2020 more...

Virtual learning environments contain valuable data about students that can be correlated and analyzed to optimize learning. Modern learning environments based on data mashups that collect and integrate data from multiple sources are relevant for learning analytics systems because they provide insights into students' learning. However, data sets involved in mashups may contain personal information of sensitive nature that raises legitimate privacy concerns. Average privacy preservation methods are based on preemptive approaches that limit the published data in a mashup based on access control and authentication schemes. Such limitations may reduce the analytical utility of the data exposed to gain students' learning insights. In order to reconcile utility and privacy preservation of published data, this research proposes a new data mashup protocol capable of merging and k-anonymizing data sets in cloud-based learning environments without jeopardizing the analytical utility of the information. The implementation of the protocol is based on linked data so that data sets involved in the mashups are semantically described, thereby enabling their combination with relevant educational data sources. The k-anonymized data sets returned by the protocol still retain essential information for supporting general data exploration and statistical analysis tasks. The analytical and empirical evaluation shows that the proposed protocol prevents individuals' sensitive information from re-identifying., The Spanish National Research Agency (AEI) funded this research through the project CREPES (ref. PID2020-115844RB-I00) with ERDF funds. more...

Modern Deep Neuronal Network backends allow a great flexibility to define network architectures. This allows for multiple outputs with their specific losses which can make them more suitable for particular goals. In this work we shall explore this possibility for classification networks which will combine the categorical cross-entropy loss, typical of softmax probabilistic outputs, the categorical hinge loss, which extends the hinge loss standard on SVMs, and a novel Fisher loss which seeks to concentrate class members near their centroids while keeping these apart, The authors acknowledge financial support from the European Regional Development Fund and the Spanish State Research Agency of the Ministry of Economy, Industry, and Competitiveness under the project PID2019-106827GB-I00. They also thank the UAM–ADIC Chair for Data Science and Machine Learning and gratefully acknowledge the use of the facilities of Centro de Computaci´on Cient´ıfica (CCC) at UAM more...

Given the impact of renewable sources in the overall energy production, accurate predictions are becoming essential, with machine learning becoming a very important tool in this context. In many situations, the prediction problem can be divided into several tasks, more or less related between them but each with its own particularities. Multitask learning (MTL) aims to exploit this structure, training several models at the same time to improve on the results achievable either by a common model or by task-specific models. In this paper, we show how an MTL approach based on support vector regression can be applied to the prediction of photovoltaic and wind energy, problems where tasks can be defined according to different criteria. As shown experimentally with three different datasets, the MTL approach clearly outperforms the results of the common and specific models for photovoltaic energy, and are at the very least quite competitive for wind energy, The authors acknowledge financial support from Spain’s grants TIN2016-76406-P and PID2019-106827GB- I00/AEI/10.13039/501100011033. Work supported also by The first author is also supported by a FPU–UAM grant more...

Esta versión del artículo ha sido aceptada para su publicación, después de la revisión por pares (cuando corresponda) y está sujeta a los términos de uso de AM de Springer Nature, pero no es la Versión de Registro y no refleja mejoras posteriores a la aceptación, ni ninguna corrección. La versión del registro está disponible en línea en: https://doi.org/10.1007/978-3-030-62522-1_15, Chatbots are software services accessed via conversation in natural language. They are increasingly used to help in all kinds of procedures like booking flights, querying visa information or assigning tasks to developers. They can be embedded in webs and social networks, and be used from mobile devices without installing dedicated apps. While many frameworks and platforms have emerged for their development, identifying the most appropriate one for building a particular chatbot requires a high investment of time. Moreover, some of them are closed – resulting in customer lock-in – or require deep technical knowledge. To tackle these issues, we propose a model-driven engineering approach to chatbot development. It comprises a neutral meta-model and a domainspecific language (DSL) for chatbot description; code generators and parsers for several chatbot platforms; and a platform recommender. Our approach supports forward and reverse engineering, and model-based analysis. We demonstrate its feasibility presenting a prototype tool and an evaluation based on migrating third party Dialogflow bots to Rasa, Work funded by the Spanish Ministry of Science (RTI2018095255-B-I00) and the R&D programme of Madrid (P2018/TCS-4314) more...

Providing useful information to the users by recommending highly demanded products and services is a fundamental part of the business of many top tier companies. Recommender Systems make use of many sources of information to provide users with accurate predictions and novel recommendations of items. Here we propose, DeepMF, a novel collaborative filtering method that combines the Deep Learning paradigm with Matrix Factorization (MF) to improve the quality of both predictions and recommendations made to the user. Specifically, DeepMF performs successive refinements of a MF model with a layered architecture that uses the acquired knowledge in a layer as input for subsequent layers. Experimental results showed that the quality of both the predictions and recommendations of DeepMF overcome the baselines., This work has been supported by Spanish Ministry of Science and Education and Competitivity (MINECO) and European Regional Development Fund (FEDER) under grants TIN2017-85727-C4-3-P (DeepBio). more...

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works, The increasing presence of photovoltaic (PV) generation in the energy mix demands improved forecasting tools which can be updated in an almost continuous basis. Satellite-based information lends itself naturally to this purpose and here it is used to nowcast hourly PV energy production for horizons up to six hours over Peninsular Spain and two islands, Majorca in the Mediterranean Sea and Tenerife in the Atlantic Ocean. This paper compares a single model-based on same-day numerical weather prediction (NWP) with hourly refreshed models which either only use satellite-based measurements or which combine both NWP same-day forecasts and satellite data. As shown in the experiments, the satellite-NWP combination gives very good nowcasting results, clearly superior to those achievable separately by either approach, The authors would like to thank Red Elctrica de Espaa for useful discussions and making available PV energy data and gratefully acknowledge the use of the facilities of Centro de Computación Científica (CCC) at UAM, and also would like to thank the Agencia Española de Meteorologa, AEMET, and the ECMWF for access to the MARS repository more...

Event-related potentials (ERPs) are important neurophysiological markers widely used in scientific, medical and engineering contexts. Proper ERP detection contributes to widening the scope of use and, in general, improving functionality. The morphology and latency of ERPs are variable among subject sessions, which complicates their detection. Although variability is an intrinsic feature of neuronal activity, it can be addressed with novel views on ERP detection techniques. In this paper, we propose an agile method for characterizing and thus detecting variable ERPs, which keeps track of their temporal and spatial information through the continuous measurement of the area under the curve in ERP components. We illustrate the usefulness of the proposed ERP characterization for electrode selection in brain-computer interfaces (BCIs) and compare the results with other standard methods. We assess ERP classification for BCI use with Bayesian linear discriminant analysis (BLDA) and cross-validation. We also evaluate performance with both the information transfer rate and BCI utility. The results of our validation tests show that this characterization helps to take advantage of the information on the evolution of positive and negative ERP components and, therefore, to efficiently select electrodes for optimized ERP detection. The proposed method improves the classification accuracy and bitrate of all sets of electrodes analyzed. Furthermore, the method is robust between different day sessions. Our work contributes to the efficient detection of ERPs, manages inter- and intrasubject variability, decreases the computational cost of classic detection methods and contributes to promoting low-cost personalized brain-computer interfaces., This work was supported by the Predoctoral Research Grants of the Ecuador Government through of the Secretaría de Educación Superior, Ciencia, Tecnología e Innovación (SENESCYT) under Grant 2015-AR2Q9086, and by the Ministerio de Ciencia, Innovación y Universidades/FEDER under the Spanish Government Grants: TIN2017-84452-R, DPI2015-65833-P and PGC2018-095895-B-I00. more...

Over the last years, the Internet of Things has fostered a growing interest in context-aware mobile applications; this fact is mainly due to highly favoring information provision from multiple Internetconnected devices. To identify user context, these applications collect information from the user and his/her environment and typically lter app information, so that the user receives only the interesting and relevant information. However, such a task usually implies further resource consumption on user mobile devices, not only regarding battery usage but also in terms of network traf c. Accordingly, although context-aware applications can improve user experiences in their daily lives, they must ensure the maintenance of lowlevel resource consumption; otherwise, the applications are promptly replaced by less consuming ones, and therefore, removed from the mobile market. In this paper, we evaluate and discuss several architectural styles for context-aware mobile applications, as well as, providing a set of guidelines to decide on the right architecture for a particular app depending on its characteristics. The use of such guidelines when choosing the right architectural style can strongly in uence the resource consumption of context-aware mobile applications. Following these guidelines, user satisfaction of a context-aware mobile application may be improved, thus guaranteeing the app success. more...

Modeling user profiles is a necessary step for most information filtering systems – such as recommender systems – to provide personalized recommendations. However, most of them work with users or items as vectors, by applying di erent types of mathematical operations between them and neglecting sequential or content-based information. Hence, in this paper we study how to propose an adaptive mechanism to obtain user sequences using di erent sources of information, allowing the generation of hybrid recommendations as a seamless, transparent technique from the system viewpoint. As a proof of concept, we develop the Longest Common Subsequence (LCS) algorithm as a similarity metric to compare the user sequences, where, in the process of adapting this algorithm to recommendation, we include di erent parameters to control the e - ciency by reducing the information used in the algorithm (preference filter), to decide when a neighbor is considered useful enough to be included in the process (confidence filter), to identify whether two interactions are equivalent ( -matching threshold), and to normalize the length of the LCS in a bounded interval (normalization functions). These parameters can be extended to work with any type of sequential algorithm. We evaluate our approach with several state-of-the-art recommendation algorithms using di erent evaluation metrics measuring the accuracy, diversity, and novelty of the recommendations, and analyze the impact of the proposed parameters. We have found that our approach o ers a competitive performance, outperforming content, collaborative, and hybrid baselines, and producing positive results when either content- or rating-based information is exploited, This article has been co-funded by the European Social Fund (ESF) within the 2017 call for predoctoral contracts and the Spanish Ministry of Economy, Industry and Competitiveness (project reference: TIN2016-80630-P) more...

© Owner/Author 2020. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in EASE '20: Proceedings of the Evaluation and Assessment in Software Engineering, https://doi.org/10.1145/3383219.3383246, Modelling is a fundamental activity in software engineering, which is often performed in collaboration. For this purpose, on-line tools running on the cloud are frequently used. However, recent advances in Natural Language Processing have fostered the emergence of chatbots, which are increasingly used for all sorts of software engineering tasks, including modelling. To evaluate to what extent chatbots are suitable for collaborative modelling, we conducted an experimental study with 54 participants, to evaluate the usability of a modelling chatbot called SOCIO, comparing it with the on-line tool Creately. We employed a within-subjects cross-over design of 2 sequences and 2 periods. Usability was determined by attributes of efficiency, effectiveness, satisfaction and quality of the results. We found that SOCIO saved time and reduced communication effort over Creately. SOCIO satisfied users to a greater extent than Creately, while in effectiveness results were similar. With respect to diagram quality, SOCIO outperformed Creately in terms of precision, while solutions with Creately had better recall and perceived success. However, in terms of accuracy and error scores, both tools were similar, Work funded by the Spanish Ministry of Science (project MASSIVE, RTI2018-095255-B-I00) and the R&D programme of Madrid (project FORTE, P2018/TCS-4314) more...

This version of the article has been accepted for publication, after peer review (when applicable) and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://doi.org/10.1007/s11063-017-9613-7, Deep Learning models are recently receiving a large attention because of their very powerful modeling abilities, particularly on inputs that have a intrinsic one- or two-dimensional structure that can be captured and exploited by convolutional layers. In this work we will apply Deep Neural Networks (DNNs) in two problems, wind energy and daily solar radiation prediction, whose inputs, derived from Numerical Weather Prediction systems, have a clear spatial structure. As we shall see, the predictions of single deep models and, more so, of DNN ensembles can improve on those of Support Vector Regression, a Machine Learning method that can be considered the state of the art for regression, With partial support from Spain’s Grants TIN2013-42351-P (MINECO), S2013/ICE2845 CASI-CAM-CM (Comunidad de Madrid), FACIL (Ayudas Fundación BBVA a Equipos de Investigación Científica 2016) and the UAM–ADIC Chair for Data Science and Machine Learning. The second author is also kindly supported by the FPU-MEC Grant AP-2012-5163. The authors gratefully acknowledge access to the MARS repository granted by the ECMWF, the use of the facilities of Centro de Computación Científica (CCC) at UAM and thank Red Eléctrica de España for kindly supplying wind energy production data and to Sotavento for making their production data publicly available. more...

We undertake a detailed examination of the steps that make up offline experiments for recommender system evaluation, including the manner in which the available ratings are filtered and split into training and test; the selection of a subset of the available users for the evaluation; the choice of strategy to handle the background effects that arise when the system is unable to provide scores for some items or users; the use of either full or condensed output lists for the purposes of scoring; scoring methods themselves, including alternative top-weighted mechanisms for condensed rankings; and the application of statistical testing on a weighted-by-user or weighted-by-volume basis as a mechanism for providing confidence in measured outcomes. We carry out experiments that illustrate the impact that each of these choice points can have on the usefulness of an end-to-end system evaluation, and provide examples of possible pitfalls. In particular, we show that varying the split between training and test data, or changing the evaluation metric, or how target items are selected, or how empty recommendations are dealt with, can give rise to comparisons that are vulnerable to misinterpretation, and may lead to different or even opposite outcomes, depending on the exact combination of settings used, The frst two authors were funded in part by Grant TIN2016-80630-P from the Spanish Ministry of Science, Innovation and Universities more...

Presently, we are living in a hyper-connected world where millions of heterogeneous devices are continuously sharing information in different application contexts for wellness, improving communications, digital businesses, etc. However, the bigger the number of devices and connections are, the higher the risk of security threats in this scenario. To counteract against malicious behaviours and preserve essential security services, Network Intrusion Detection Systems (NIDSs) are the most widely used defence line in communications networks. Nevertheless, there is no standard methodology to evaluate and fairly compare NIDSs. Most of the proposals elude mentioning crucial steps regarding NIDSs validation that make their comparison hard or even impossible. This work firstly includes a comprehensive study of recent NIDSs based on machine learning approaches, concluding that almost all of them do not accomplish with what authors of this paper consider mandatory steps for a reliable comparison and evaluation of NIDSs. Secondly, a structured methodology is proposed and assessed on the UGR’16 dataset to test its suitability for addressing network attack detection problems. The guideline and steps recommended will definitively help the research community to fairly assess NIDSs, although the definitive framework is not a trivial task and, therefore, some extra effort should still be made to improve its understandability and usability further., The authors would like to acknowledge the Spanish Ministerio de Ciencia, Innovación y Universidades and ERDF for the support provided under contracts RTI2018-100754-B-I00 (iSUN) and RTI2018-098160-B-I00 (DEEPAPFORE). more...

Deep learning has become a breathtaking technology in the last years, overcoming traditional handcrafted approaches and even humans for many different tasks. However, in some tasks, such as the verification of handwritten signatures, the amount of publicly available data is scarce, what makes difficult to test the real limits of deep learning. In addition to the lack of public data, it is not easy to evaluate the improvements of novel proposed approaches as different databases and experimental protocols are usually considered. The main contributions of this study are: i) we provide an in-depth analysis of state-of-the-art deep learning approaches for on-line signature verification, ii) we present and describe the new DeepSignDB on-line handwritten signature biometric public database,1 iii) we propose a standard experimental protocol and benchmark to be used for the research community in order to perform a fair comparison of novel approaches with the state of the art, and iv) we adapt and evaluate our recent deep learning approach named Time-Aligned Recurrent Neural Networks (TA-RNNs)2. for the task of on-line handwritten signature verification. This approach combines the potential of Dynamic Time Warping and Recurrent Neural Networks to train more robust systems against forgeries. Our proposed TA-RNN system outperforms the state of the art, achieving results even below 2.0% EER when considering skilled forgery impostors and just one training signature per user.1https://github.com/BiDAlab/DeepSignDB 2Spanish Patent Application (P202030060), This work was supported in part by PRIMA under Grant H2020-MSCA-ITN-2019-860315; in part by TRESPASS-ETN under Grant H2020-MSCA-ITN-2019-860813; in part by BIBECA under Grant RTI2018-101248-B-I00 MINECO/FEDER; in part by Bio-Guard (Ayudas Fundación BBVA a Equipos de Investigación Científica 2017); and in part by UAM-Cecabank. The work of Ruben Tolosana was supported by the Comunidad de Madrid and Fondo Social Europeo. Spanish Patent Application (P202030060) more...

The main goal of a Recommender System is to suggest relevant items to users, although other utility dimensions – such as diversity, novelty, confidence, possibility of providing explanations – are often considered. In this work, we investigate about confidence but from the perspective of the system: what is the confidence a system has on its own recommendations; more specifically, we focus on different methods to embed awareness into the recommendation algorithms about deciding whether an item should be suggested. Sometimes it is better not to recommend than fail because failure can decrease user confidence in the system. In this way, we hypothesise the system should only show the more reliable suggestions, hence, increasing the performance of such recommendations, at the expense of, presumably, reducing the number of potential recommendations. Different from other works in the literature, our approaches do not exploit or analyse the input data but intrinsic aspects of the recommendation algorithms or of the components used during prediction are considered. We propose a taxonomy of techniques that can be applied to some families of recommender systems allowing to include mechanisms to decide if a recommendation should be generated. In particular, we exploit the uncertainty in the prediction score for a probabilistic matrix factorisation algorithm and the family of nearest-neighbour algorithms, the support of the prediction score for nearest-neighbour algorithms, and a method independent of the algorithm. We study how the performance of a recommendation algorithm evolves when it decides not to recommend in some situations. If the decision of avoiding a recommendation is sensible – i.e., not random but related to the information available to the system about the target user or item –, the performance is expected to improve at the expense of other quality dimensions such as coverage, novelty, or diversity. This balance is critical, since it is possible to achieve a very high precision recommending only one item to a unique user, which would not be a very useful recommender. Because of this, on the one hand, we explore some techniques to combine precision and coverage metrics, an open problem in the area. On the other hand, a family of metrics (correctness) based on the assumption that it is better to avoid a recommendation rather than providing a bad recommendation is proposed herein. In summary, the contributions of this paper are twofold: a taxonomy of techniques that can be applied to some families of recommender systems allowing to include mechanisms to decide if a recommendation should be generated, and a first exploration to the combination of evaluation metrics, mostly focused on measures for precision and coverage. Empiric results show that large precision improvements are obtained when using these approaches at the expense of user and item coverage and with varying levels of novelty and diversity, This work was funded by the national Spanish Government under project TIN2016-80630-P. The authors also acknowledge the very helpful feedback from the three anonymous reviewers more...

© ACM 2020. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings, http://dx.doi.org/10.1145/10.1145/3417990.3420205, Domain-specific languages (DSLs) are languages tailored for particular domains. Many frameworks and tools have been proposed to develop editors for DSLs, especially for desktop IDEs, like Eclipse. We are witnessing the advent of low-code development platforms, which are cloud-based environments supporting rapid application development by using graphical languages and forms. While this approach is very promising, the creation of new low-code platforms may require the migration of existing desktop-based editors to the web. However, this is a technically challenging task. To fill this gap, we present ROCCO, a tool that migrates Eclipse-based graphical modelling editors to the web, to facilitate their integration with low-code platforms. The tool reads a meta-model annotated with EuGENia annotations, and generates a web editor using the DPG web framework used by the UGROUND company. In this paper, we present the approach, including tool support and an evaluation based on migrating nine editors created by third parties, which shows the usefulness of the tool, This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement n° 813884, Lowcomote [33]. The work has also been supported by the Spanish Ministry of Science (project MASSIVE, RTI2018-095255-B-I00) and the R&D programme of Madrid (project FORTE, P2018/TCS-4314) more...

Modeling the temporal context efficiently and effectively is essential to provide useful recommendations to users. In this work, we focus on improving neighborhood-based approaches where we integrate three different mechanisms to exploit temporal information. We first present an improved version of a similarity metric between users using a temporal decay function, then, we propose an adaptation of the Longest Common Subsequence algorithm to be used as a time-aware similarity metric, and we also redefine the neighborhood-based recommenders to be interpreted as ranking fusion techniques where the neighbor interaction sequence can be exploited by considering the last common interaction between the neighbor and the user. We demonstrate the effectiveness of these approaches by comparing them with other state-of-the-art recommender systems such as Matrix Factorization, Neural Networks, and Markov Chains under two realistic time-aware evaluation methodologies (per user and community-based). We use several evaluation metrics to measure both the quality of the recommendations – in terms of ranking relevance – and their temporal novelty or freshness. According to the obtained results, our proposals are highly competitive and obtain better results than the rest of the analyzed algorithms, producing improvements under the two evaluation dimensions tested consistently through three real-world datasets, This work has been funded by the Ministerio de Ciencia, Innovación y Universidades (reference TIN2016-80630-P) more...

La creciente expansión de las tecnologías móviles ha creado nuevas oportunidades y desafíos para la enseñanza y el aprendizaje de idiomas. No obstante, a pesar de que en la última década se hayan desarrollado numerosas aplicaciones comerciales y no comerciales (Apps) para fomentar el aprendizaje de idiomas, pocas explotan el potencial que ofrecen hoy en día los smartphones o tabletas para crear nuevos entornos de aprendizaje. En cambio, la mayoría siguen centrándose en un aprendizaje formal e individual, sin aprovechar las posibilidades que brindan los actuales dispositivos móviles para incrementar la interacción y comunicación entre varios usuarios. De ahí nuestro propósito de desarrollar una app que no solo permita la interacción entre sus usuarios proponiendo un enfoque basado en el aprendizaje comunitario sino que además explote el potencial de la gamificación para incrementar la motivación del alumnado. Para ello se ha implementado una app gamificada que permite al usuario introducir nuevo contenido (acorde a sus necesidades de aprendizaje) además de evaluar y reportar el contenido creado por otros usuarios y de recibir feedback regular sobre su propio proceso de aprendizaje. The increasing expansion of mobile technologies (smartphones, tablets, etc.) has created new opportunities and challenges for the teaching and learning of languages. However, despite the numerous commercial and non-commercial apps that have been developed in the last decade, very few applications (apps), explore the potential that current smartphones and tablets offer to create new learning environments. In fact, most of the apps still focus its interest on supporting form-focused and individual learning rather than exploiting the potential of current mobile devices to enhance students’ interaction and communication amongst each other. Therefore, the present study aims at developing a gamified app that not only might let its users to interact amongst each other providing a community-driven learning approach but also exploits the potential of gamification to enhance students’ motivation. To this end we have implemented a system that allows its users to add new content based on their learning needs, evaluate and report the learning content that is gradually created by other users. more...

Evaluating the extent to which computer-produced stories are structured like human-invented narratives can be an important component of the quality of a story plot. In this paper, we report on an empirical experiment in which human subjects have invented short plots in a constrained scenario. The stories were annotated according to features commonly found in existing automatic story generators. The annotation was designed to measure the proportion and relations of story components that should be used in automatic computational systems for matching human behaviour. Results suggest that there are relatively common patterns that can be used as input data for identifying similarity to human-invented stories in automatic storytelling systems. The found patterns are in line with narratological models, and the results provide numerical quantification and layout of story components. The proposed method of story analysis is tested over two additional sources, the ROCStories corpus and stories generated by automated storytellers, to illustrate the valuable insights that may be derived from them. more...

This paper analyzes a sample of patients hospitalized with COVID-19 in the region of Madrid (Spain). Survival analysis, logistic regression, and machine learning techniques (both supervised and unsupervised) are applied to carry out the analysis where the endpoint variable is the reason for hospital discharge (home or deceased). The different methods applied show the importance of variables such as age, O2 saturation at Emergency Rooms (ER), and whether the patient comes from a nursing home. In addition, biclustering is used to globally analyze the patient-drug dataset, extracting segments of patients. We highlight the validity of the classifiers developed to predict the mortality, reaching an appreciable accuracy. Finally, interpretable decision rules for estimating the risk of mortality of patients can be obtained from the decision tree, which can be crucial in the prioritization of medical care and resources., This research was funded by Agencia Estatal de Investigación AEI/FEDER Spain, Project PGC2018-095895-B-I00, and Comunidad Autónoma de Madrid, Spain, Project S2017/BMD-3688 more...

Modelling is an essential activity in software engineering processes. It typically involves two meta-levels: one includes meta-models that describe modelling languages, and the other contains models built by instantiating those meta-models. Multi-level modelling generalizes this approach by allowing models to span an arbitrary number of meta-levels. A scenario that profits from multi-level modelling is the definition of language families that become specialized by successive refinements at subsequent meta-levels, hence promoting language reuse. This enables an open set of variability options for the possible specializations of a given language. However, multi-level modelling lacks the ability to express closed variability regarding the supported language primitives and their realizations. This limits the reuse opportunities of a language family. To improve this situation, we propose a novel combination of product lines with multi-level modelling to cover both open and closed variability. Our proposal is backed by a formal theory that guarantees correctness, and is implemented atop the MetaDepth multi-level modelling tool., Work funded by the Spanish Ministry of Science (project MASSIVE, RTI2018-095255-B-I00) and the R&D programme of Madrid (project FORTE, P2018/TCS-4314). more...

Reproduced with permission from the authors, Applied Sciences; published by Multidisciplinary Digital Publishing Institute, 2019., This paper presents a tutoring system which uses three different granularities for helping students to classify animals from bone fragments in zooarchaeology. The 3406 bone remains, which have 64 attributes, were obtained from the excavation of the Middle Palaeolithic site of El Salt (Alicante, Spain). The coarse granularity performs a five-class prediction, the medium a twelve-class prediction, and the fine a fifteen-class prediction. In the coarse granularity, the results show that the first 10 most relevant attributes for classification are width, bone, thickness, length, bone fragment, anatomical group, long bone circumference, X, Y, and Z. Based on those results, a user-friendly interface of the tutor has been built in order to train archaeology students to classify new remains using the coarse granularity. A pilot has been performed in the 2019 excavation season in Abric del Pastor (Alicante, Spain), where the automatic tutoring system was used by students to classify 51 new remains. The pilot experience demonstrated the usefulness of the tutoring system both for students when facing their first classification activities and also for seniors since the tutoring system gives them valuable clues for helping in difficult classification problems, This research was funded by ADeAPTIVE (Advanced Design of e-Learning Applications Personalizing Teaching to Improve Virtual Education) project with the support of the Erasmus+ programme of the European Union (grant number 2017-1-ES01-KA203-038266). This work was also part of the project Explora “Application of Soft Computing techniques and 3D modeling in massive data processing in Archeology” that was supported by the Spanish Ministry of Science and Innovation (Grant TIN2015-72682-EXP). This study has been partially funded by ACCIÓ, Spain (Pla d’Actuació de Centres Tecnològics 2019) under the project Augmented Workplace more...

The Internet of Things enables experts of given domains to create smart user experiences for interacting with the environment. However, development of such experiences requires strong programming skills, which are challenging to develop for non-technical users. This paper presents several extensions to the block-based programming language used in App Inventor to make the creation of mobile apps for smart learning experiences less challenging. Such apps are used to process and graphically represent data streams from sensors by applying map-reduce operations. A workshop with students without previous experience with Internet of Things (IoT) and mobile app programming was conducted to evaluate the propositions. As a result, students were able to create small IoT apps that ingest, process and visually represent data in a simpler form as using App Inventor's standard features. Besides, an experimental study was carried out in a mobile app development course with academics of diverse disciplines. Results showed it was faster and easier for novice programmers to develop the proposed app using new stream processing blocks., Spanish National Research Agency (AEI) - ERDF funds more...

Para aprovechar las plataformas paralelas, se necesitan herramientas de programación para poder representar apropiadamente los algoritmos paralelos. Además, los entornos paralelos requieren sistemas en tiempo de ejecución que ofrezcan diferentes paradigmas de computación. Existen diferentes áreas a estudiar con el fin de construir un sistema en tiempo de ejecución completo para un entorno paralelo. Esta Tesis aborda dos problemas comunes: el soporte unificado de datos densos y dispersos, y la integración de paralelismo orientado a mapeo de datos y paralelismo orientado a flujo de datos. Esta Tesis propone una solución que desacopla la representación, partición y reparto de datos, del algoritmo y de la estrategia de diseño paralelo para integrar manejo para datos densos y dispersos. Además, se presenta un nuevo modelo de programación basado en el paradigma de flujo de datos, donde diferentes actividades pueden ser arbitrariamente enlazadas para formar redes genéricas pero estructuradas que representan el cómputo global, Departamento de Informática (Arquitectura y Tecnología de Computadores, Ciencias de la Computación e Inteligencia Artificial, Lenguajes y Sistemas Informáticos) more...

Trivial Compiler Equivalence (TCE) has been recently proposed as an effective technique to detect equivalences between programs, where two or more programs are equivalent if the compiler pro duces the same binary code. Mutation testing can greatly benefit from TCE as a way to reveal some equivalent and duplicate mu tants, which traditionally hinder the applicability of the technique. For instance, previous research has shown that about 28% of the mutants generated by traditional mutation operators in C programs can be removed using TCE. However, the effectiveness of TCE has not been assessed with class-level operators, where the percentage of equivalent mutants is known to be higher than when using tra ditional ones. In this paper, we present an empirical study on the effectiveness of TCE at identifying equivalent and duplicate mutants using C++ class operators. The results show that TCE is helpful to discard equivalent and duplicate mutants: 241 out of 1,987 (12%) in our study, including 189 out of 684 (27.6%) manually-identified equivalent mutants. Large differences were observed among the different case studies, especially in the detection rate of equivalent mutants, which ranged from 4% to 45%. Ministerio de Economía y Competitividad TIN2015-65845-C3-3-R (DarDos) Ministerio de Economía y Competitividad TIN2015-70560-R (BELI) more...

The miniaturization and price reduction of sensors have encouraged the proliferation of smart environments, in which multitudinous sensors detect and describe the activities carried out by inhabitants. In this context, the recognition of activities of daily living has represented one of the most developed research areas in recent years. Its objective is to determine what daily activity is developed by the inhabitants of a smart environment. In this field, many proposals have been presented in the literature, many of them being based onad hocontologies to formalize logical rules, which hinders their reuse in other contexts. In this work, we propose the use of class expression learning (CEL), an ontology-based data mining technique, for the recognition of ADL. This technique is based on combining the entities in the ontology, trying to find the expressions that best describe those activities. As far as we know, it is the first time that this technique is applied to this problem. To evaluate the performance of CEL for the automatic recognition of activities, we have first developed a framework that is able to convert many of the available datasets to all the ontology models we have found in the literature for dealing with ADL. Two different CEL algorithms have been employed for the recognition of eighteen activities in two different datasets. Although all the available ontologies in the literature are focused on the description of the context of the activities, the results show that the sequence of the events produced by the sensors is more relevant for their automatic recognition, in general terms. more...

The impact of data characteristics on the performance of classical recommender systems has been recently investigated and produced fruitful results about the relationship they have with recommendation accuracy. This work provides a systematic study on the impact of broadly chosen data characteristics (DCs) of recommender systems. This is applied to the accuracy and fairness of several variations of CF recommendation models. We focus on a suite of DCs that capture properties about the structure of the user–item interaction matrix, the rating frequency, item properties, or the distribution of rating values. Experimental validation of the proposed system involved large-scale experiments by performing 23,400 recommendation simulations on three real-world datasets in the movie (ML-100K and ML-1M) and book domains (BookCrossing). The validation results show that the investigated DCs in some cases can have up to 90% of explanatory power – on several variations of classical CF algorithms –, while they can explain – in the best case – about 40% of fairness results (measured according to user gender and age sensitive attributes). Therefore, this work evidences that it is more difficult to explain variations in performance when dealing with fairness dimension than accuracy, This work was supported in part by the Ministerio de Ciencia, Innovación y Universidades (reference: PID2019-108965GB-I00), and in part by H2020 PASSEPARTOUT No. 101016956, Servizi Locali S.P.A. Servizi Locali 2.0, and Ministry of Education, University and Research (references: PON ARS01_00876 Bio-D, PON ARS01_00821 FLET4.0, PON ARS01_00917 OK-INSAID) more...

This is the author’s version of a work that was accepted for publication in Information Systems. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Information Systems, VOL 62, (2016)] DOI 10.1016/j.is.2016.06.008, Meta-models play a cornerstone role in Model-Driven Engineering as they are used to define the abstract syntax of modelling languages, and so models and all sorts of model transformations depend on them. However, there are scarce tools and methods supporting their Validation and Verification (V&V), which are essential activities for the proper engineering of meta-models. In order to fill this gap, we propose two complementary meta-model V&V languages. The first one has similar philosophy to the xUnit framework, as it enables the definition of meta-model unit test suites comprising model fragments and assertions on their (in-)correctness. The second one is directed to express and verify expected properties of a meta-model, including domain and design properties, quality criteria and platform-specific requirements. As a proof of concept, we have developed tooling for both languages in the Eclipse platform, and illustrate its use within an example-driven approach for meta-model construction. The expressiveness of our languages is demonstrated by their application to build a library of meta-model quality issues, which has been evaluated over the ATL zoo of meta-models and some OMG specifications. The results show that integrated support for meta-model V&V (as the one we propose here) is urgently needed in meta-modelling environments., This work has been funded by the Spanish Ministry of Economy and Competitivity with project “Flexor” (TIN2014-52129-R), the region of Madrid with project “SICOMORO-CM” (S2013/ICE-3006), and the EU commission with project “MONDO” (FP7- ICT-2013-10, #611125). more...

An accurate prediction of freight volume at the sanitary facilities of seaports is a key factor to improve planning operations and resource allocation. This study proposes a hybrid approach to forecast container volume at the sanitary facilities of a seaport. The methodology consists of a three-step procedure, combining the strengths of linear and non-linear models and the capability of a clustering technique. First, a self-organizing map (SOM) is used to decompose the time series into smaller clusters easier to predict. Second, a seasonal autoregressive integrated moving averages (SARIMA) model is applied in each cluster in order to obtain predicted values and residuals of each cluster. These values are finally used as inputs of a support vector regression (SVR) model together with the historical data of the cluster. The final prediction result integrates the prediction results of each cluster. The experimental results showed that the proposed model provided accurate prediction results and outperforms the rest of the models tested. The proposed model can be used as an automatic decision-making tool by seaport management due to its capacity to plan resources in advance, avoiding congestion and time delays. more...

Complex event processing (CEP) is a computational intelligence technology capable of analyzing big data streams for event pattern recognition in real time. In particular, this technology is vastly useful for analyzing multicriteria conditions in a pattern, which will trigger alerts (complex events) upon their fulfillment. However, one of the main challenges to be faced by CEP is how to define the quantitative analysis to be performed in response to the produced complex events. In this paper, we propose the use of the MEdit4CEP-CPN model-driven tool as a solution for conducting such quantitative analysis of events of interest for an application domain, without requiring knowledge of any scientific programming language for implementing the pattern conditions. Precisely, MEdit4CEP-CPN facilitates domain experts to graphically model event patterns, transform them into a Prioritized Colored Petri Net (PCPN) model, modify its initial marking depending on the application scenario, and make the quantitative analysis through the simulation and monitor capabilities provided by CPN tools. more...

The choice of mutation operators is a fundamental aspect in mutation testing to guide the tester to an effective test suite. Designing a set of mutation operators is subject to a trade-off between effectiveness and computational cost: a larger mutation population might uncover more faults, but will take longer to analyse. With the aim of resolving this trade-off, several authors have defined an assortment of metrics to determine the most valuable operators. In this work, we extend an existing quality metric by incorporating an additional source of data and coverage information and therefore investigate the extent to which mutants that are often covered but rarely killed can improve the evaluation of mutation operators for the refinement of the test suite. As a case study, we analyse C++ class-level operators based on the new coverage-based quality metric to assess whether the original metric is enhanced. The results when selecting the best-valued operators show that this metric has great potential to help the tester in finding effective mutation operators. In comparison with the metric from which it is derived, the use of coverage data allows to reduce the number of mutants but often loses fewer test cases and, in addition, retains those that seem hard to design. more...

This version of the article has been accepted for publication, after peer review (when applicable) and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://doi.org/10.1007/978-3-030-19651-6_4, Support Vector Machines, SVM, are one of the most popular machine learning models for supervised problems and have proved to achieve great performance in a wide broad of predicting tasks. However, they can suffer from scalability issues when working with large sample sizes, a common situation in the big data era. On the other hand, Deep Neural Networks (DNNs) can handle large datasets with greater ease and in this paper we propose Deep SVM models that combine the highly non-linear feature processing of DNNs with SVM loss functions. As we will show, these models can achieve performances similar to those of standard SVM while having a greater sample scalability, With partial support from Spain's grants TIN2016-76406-P and 82013/ICE-2845 CASI-CAM-CM. Work partially supported also by project FACILAyudas Fundación BBVA a Equipos de Investigación Científica 2016, and the UAMADIC Chair for Data Science and Machine Learning. We also gratefully acknowledge the use of the facilities of Centro de Computación Científica (CCC) at UAM more...

During the last few years, computer performance has reached a turning point where computing power is no longer the only important concern. This way, the emphasis is shifting from an exclusive focus on the optimisation of the computing system to optimising other systems, like the memory system. Broadly speaking, testing memory systems entails two main challenges: the oracle problem and the reliable test set problem. The former consists in deciding if the outputs of a test suite are correct. The latter refers to providing an appropriate test suite for determining the correctness of the system under test. In this paper we propose an expert system for checking the correctness of memory systems. In order to face these challenges, our proposed system combines two orthogonal techniques – simulation and metamorphic testing – enabling the automatic generation of appropriate test cases and deciding if their outputs are correct. In contrast to conventional expert systems, our system includes a factual database containing the results of previous simulations, and a simulation platform for computing the behaviour of memory systems. The knowledge of the expert is represented in the form of metamorphic relations, which are properties of the analysed system involving multiple inputs and their outputs. Thus, the main contribution of this work is two-fold: a method to automatise the testing process of memory systems, and a novel expert system design focusing on increasing the overall performance of the testing process. To show the applicability of our system, we have performed a thorough evaluation using 500 memory configurations and 4 di erent memory management algorithms, which entailed the execution of more than one million of simulations. The evaluation used mutation testing, injecting faults in the memory management algorithms. The developed expert system was able to detect over 99% of the critical injected faults, hence obtaining very promising results, and outperforming other standard techniques like random testing, This work was supported by the Spanish Ministerio de Economía, Industria y Competitividad, Gobierno de España/FEDER (grant numbers DArDOS, TIN2015-65845-C3-1-R and FAME, RTI2018-093608-B-C31) and the Comunidad de Madrid project FORTE under Grant S2018/TCS-4314. The first author is also supported by the Universidad Complutense de Madrid - Santander Universidades grant (CT17/17-CT18/17) more...

Context: Search-based techniques have been applied to almost all areas in software engineering, especially to software testing, seeking to solve hard optimization problems. However, the problem of selecting mutants to improve the test suite at a lower cost has not been explored to the same extent as other problems, such as mutant selection for test suite evaluation or test data generation. Objective: In this paper, we apply search-based mutant selection to enhance the quality of test suites efficiently. Namely, we use the technique known as Evolutionary Mutation Testing (EMT), which allows reducing the number of mutants while preserving the power to refine the test suite. Despite reported benefits of its application, the existing empirical results were derived from a limited number of case studies, a particular set of mutation operators and a vague measure, which currently makes it difficult to determine the real performance of this technique. Method: This paper addresses the shortcomings of previous studies, providing a new methodology to evaluate EMT on the basis of the actual improvement of the test suite achieved by using the evolutionary strategy. We make use of that methodology in new experiments with a carefully selected set of real-world C++ case studies. Results: EMT shows a good performance for most case studies and levels of demand of test suite improvement (around 45% less mutants than random selection in the best case). The results reveal that even a reduced subset of mutants selected with EMT can serve to increase confidence in the test suite, especially in programs with a large set of mutants. Conclusions: These results support the use of search-based techniques to solve the problem of mutant selection for a more efficient test suite refinement. Additionally, we identify some aspects that could foreseeably help enhance EMT. more...

This is the author’s version of a work that was accepted for publication in Neurocomputing. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Neurocomputing, VOL 163, (2015) DOI 10.1016/j.neucom.2014.08.090, The growing interest in big data problems implies the need for unsupervised methods for data visualization and dimensionality reduction. Diffusion Maps (DM) is a recent technique that can capture the lower dimensional geometric structure underlying the sample patterns in a way which can be made to be independent of the sampling distribution. Moreover, DM allows us to define an embedding whose Euclidean metric relates to the sample's intrinsic one which, in turn, enables a principled application of k-means clustering. In this work we give a self-contained review of DM and discuss two methods to compute the DM embedding coordinates to new out-of-sample data. Then, we will apply them on two meteorological data problems that involve time and spatial compression of numerical weather forecasts and show how DM is capable to, first, greatly reduce the initial dimension while still capturing relevant information in the original data and, also, how the sample-derived DM embedding coordinates can be extended to new patterns., The authors acknowledge partial support from Spain's grant TIN2010-21575-C02-01 and the UAM{ADIC Chair for Machine Learning. The first author is also supported by an FPI{UAM grant and kindly thanks the Applied Mathematics Department of Yale University for receiving her during her visits. more...

© 2015 by MDPI (http://www.mdpi.org). Reproduction is permitted for noncommercial purposes., In this paper, we present Virtual Touch, a toolkit that allows the development of educational activities through a mixed reality environment such that, using various tangible elements, the interconnection of a virtual world with the real world is enabled. The main goal of Virtual Touch is to facilitate the installation, configuration and programming of different types of technologies, abstracting the creator of educational applications from the technical details involving the use of tangible interfaces and virtual worlds. Therefore, it is specially designed to enable teachers to themselves create educational activities for their students in a simple way, taking into account that teachers generally lack advanced knowledge in computer programming and electronics. The toolkit has been used to develop various educational applications that have been tested in two secondary education high schools in Spain., We want to thank the IES Ernest Lluch and the IES Joan Coromines for all their support during the experience. The work reported in this paper was partially funded by the Spanish Research plan (project TIN2013-44586-R) and by the Madrid Research Plan (project S2013/ICE-2715). more...

This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in SAC '15 Proceedings of the 30th Annual ACM Symposium on Applied Computing, http://dx.doi.org/10.1145/2695664.2695717., This paper describes a new modelling language for the ef- fective design of Java annotations. Since their inclusion in the 5th edition of Java, annotations have grown from a use- ful tool for the addition of meta-data to play a central role in many popular software projects. Usually they are con- ceived as sets with dependency and integrity constraints within them; however, the native support provided by Java for expressing this design is very limited. To overcome its de ciencies and make explicit the rich conceptual model which lies behind a set of annotations, we propose a domain-speci c modelling language. The proposal has been implemented as an Eclipse plug- in, including an editor and an integrated code generator that synthesises annotation processors. The language has been tested using a real set of annotations from the Java Per- sistence API (JPA). It has proven to cover a greater scope with respect to other related work in di erent shared areas of application., This work was supported by the Spanish Ministry of Economy and Competitivity with project Go-Lite (TIN2011-24139) and the Community of Madrid with project SICOMORO (S2013/ICE-3006). more...

Performance bugs are common, costly, and elusive. Performance tests aim to detect performance bugs by running the program with specific inputs and determining whether the observed behaviour is acceptable. There not exist mechanisms, however, to assess the effectiveness of performance tests. Mutation testing is a technique to evaluate and enhance functional test suites by seeding artificial faults in the program under test. In this new idea paper, we explore the applicability of mutation testing to assess and improve performance tests. This novel approach is motivated with examples and open questions. more...

Open Access funded by Medical Research Council, The MRC binary file format is widely used in the three-dimensional electron microscopy field for storing image and volume data. Files contain a header which describes the kind of data held, together with other important metadata. In response to advances in electron microscopy techniques, a number of variants to the file format have emerged which contain useful additional data, but which limit interoperability between different software packages. Following extensive discussions, the authors, who represent leading software packages in the field, propose a set of extensions to the MRC format standard designed to accommodate these variants, while restoring interoperability. The MRC format is equivalent to the map format used in the CCP4 suite for macromolecular crystallography, and the proposal also maintains interoperability with crystallography software. This Technical Note describes the proposed extensions, and serves as a reference for the standard., We thank Chris Booth and Steffen Meyer from Gatan Inc. for clarifying the format definition used by Digital Micrograph. Acknowledgement for support from National Institute of Health, USA includes: NIGMS grant P41GM103310 (AC and SD), NIBIB grant 5R01-EB005027 (DM), and R01GM080139 (SJL). RH and MW would like to thank the UK Medical Research Council for the award of Partnership Grant MR/J000825/1 to support the establishment of CCP-EM. RH and JS are also supported by MRC grant U105184322. more...