Search

[EN] Traversal-based automated software testing involves testing an application via its graphical user interface (GUI) and thereby taking the user's point of view and executing actions in a human-like manner. These actions are decided on the fly, as the software under test (SUT) is being run, as opposed to being set up in the form of a sequence prior to the testing, a sequence that is then used to exercise the SUT. In practice, random choice is commonly used to decide which action to execute at each state (a procedure commonly referred to as monkey testing), but a number of alternative mechanisms have also been proposed in the literature. Here we propose using genetic programming (GP) to evolve such an action selection strategy, defined as a list of IF-THEN rules. Genetic programming has proved to be suited for evolving all sorts of programs, and rules in particular, provided adequate primitives (functions and terminals) are defined. These primitives must aim to extract the most relevant information from the SUT and the dynamics of the testing process. We introduce a number of such primitives suited to the problem at hand and evaluate their usefulness based on various metrics. We carry out experiments and compare the results with those obtained by random selection and also by Q-learning, a reinforcement learning technique. Three applications are used as Software Under Test (SUT) in the experiments. The analysis shows the potential of GP to evolve action selection strategies.

Software Engineering courses play an important role for preparing students with the right knowledge and attitude for software development in practice. The implication is far reaching, as the quality of the software that we use ultimately depends on the quality of the people that make them. Educating Software Engineering, however, is quite challenging, as the subject is not considered as most exciting by students, while teachers often have to deal with exploding number of students. The EU project IMPRESS seeks to explore the use of gamification in educating software engineering at the university level to improve students' engagement and hence their appreciation for the taught subjects. This paper presents the project, its objectives, and its current progress.

Software engineers working in industry seldom try to apply formal methods to solve problems. There are various reasons for this. Sometimes these reasons are understandable---the cost of using formal methods does not make economic sense in many contexts. However, formal methods are also often greeted with scepticism. Formal methods are assumed to take too much time, require tools that are too academic, or to be too mathematical to be understood by practice-oriented software engineers. We tested these assumptions by designing a small course around a framework for program verification, aimed at regular computer science students enrolled in a Master's programme. After four lectures and associated exercises, students were given a small verification task where they had to model and verify a real, non-trivial, C function in Why3. A significant majority of students managed to prove a non-trivial functional specification of this C function in the time allotted, and many also pointed out inherent flaws of this function discovered during formalization. Participants reported no major difficulties or mental hurdles in learning Why3, and considered its approach to be appropriate for selected components of safety-critical software. While formal verification tools such as Why3 still have lots of room for improvement, this experience shows that in a short amount of time, software engineers can be taught to use a program verification tool, and obtain usable results without being fully proficient in it. We further recommend that courses on formal methods should also let students explore these as techniques to be applied, instead of only focusing on the theory behind them, as we expect this to gradually lower the barrier to wider acceptance.

Service-oriented architecture (SOA) is a popular architectural style centered around services, loose coupling, and interoperability. A recurring problem in SOA development is the Button Problem; how to ensure that whenever a “button is pressed” on some service—no matter what—the performance of other key services remains unaffected? The Button Problem is especially complex to solve in systems that have devolved into hardly comprehensible spaghettis of service dependencies.In a collaborative effort with industry partner First8, we present the first formal framework to help SOA developers solve the Button Problem, enabling automated reasoning about service sensitivities and candidate refactorings. Our formalization provides a rigorous foundation for a tool that was already successfully evaluated in industrial case studies, and it is built against two unique requirements: “whiteboard level of abstraction” and non-quantitative analysis.

Researchers have proposed a number of prioritization techniques to help decision makers select an optimal combination of (non-) functional requirements to implement. [Problem] However, most of them are defined based on an ordinal or nominal scale, which are not reliable because they are limited to simple operations of ranked or ordered requirements. [Principal ideas] We argue that the importance of certain requirements could be determined by their criticality level, which can be assessed using a ratio scale. [Contribution]The main contribution of the paper is the new strategy proposed for prioritizing functional requirements, using early mutation testing and dependency analysis.

Authentication is a major research topic in the information security field. Much has been written about assessing entity (user) authentication methods, but there is a lack of literature concerning the evaluation of financial transaction authentication in online banking. Entity authentication methods have been systematized by quantifying their qualitative aspects, but there is no evaluation mechanism which also places the additional characteristics of transaction authentication in a user-centric context. Based on an existing mechanism which quantifies accessibility, memorability, security and vulnerability characteristics in entity authentication methods, we propose feasibility as an additional dimension which quantifies aspects related to the secure usability of transaction authentication methods. We also propose the use of this evaluation mechanism by multiple raters to reduce personal bias. Four implemented and eight proposed authentication methods for online banking were evaluated by seven experts. The results indicate that the mechanism can be applied on a wide range of authentication methods, since it is able to evaluate methods based on different information schemes. However, care must be taken that evaluations are performed by multiple experts, due to the amount of subjectivity inherent in the mechanism and in the different opinions of the raters. We introduced a qualitative evaluation mechanism for online banking authentication.Seven raters examine 12 online banking authentication methods with our mechanism.Bank-issued authentication devices overall have a qualitative very good fit.Most user-owned devices fit poorly compared to bank-issued devices.

In order to obtain a fair ‘OER picture’ for the Global South a large-scale study has been carried out for a series of countries, including Kenya. In this paper we report on the Kenya study, run at four universities that have been selected with randomly sampled students and lecturers. Empirical data have been generated by the use of a student and a lecturer questionnaire to which in total 798 students and 43 lecturers have responded. Selected from the very rich source two major outcomes are: (i) there is a significant digital differentiation among lecturers and students at urban versus rural universities in terms of their proficiency and internet accessibility; and (ii) the awareness and appreciation of the OER concept and open licensing is low but from the actual processing by respondents of educational resources (not necessarily open) a ‘preparedness for openness’ can be derived that promises well for the future.

A survey was conducted to provide a state of the art of online banking authentication and communications security implementations. Between global regions the applied (single or multifactor) authentication schemes differ greatly, as well as the security of SSL/TLS implementations. Three phases for online banking development are identified. It is predicted that mobile banking will enter a third phase, characterized by the use of standard web technologies to develop mobile banking applications for different platforms. This has the potential to make mobile banking a target for attacks in a similar manner that home banking currently is.

To manage the complexity of today's information systems, we need to investigate novel approaches for automated testing. In this paper we present two extensions to TESTAR, a state-of-the-art tool for testing systems through the GUI. We extend this tool with 1) a systematic and powerful approach for storing and querying test results through a graph database for offline oracles, and 2) support for accessibility evaluation of general applications for stakeholders with disabilities utilizing offline oracles. Furthermore, we conduct a preliminary validation of these extensions through a case study on the popular VLC Media Player.

Education in general and cybersecurity education in particular can be made more attractive by adding hands-on experience to classrooms. This requires new technology, such as virtualisation, to be developed fully geared towards the needs of educational purposes. Over the years, several techniques have been developed by the authors. In this paper, the authors first give a full account of their earlier work on a distributed virtual computer lab for cybersecurity education. Then, this virtual lab is extended with educational enhancements, such as an intelligent tutoring system, which resulted in a prototype for a virtual classroom for cybersecurity education.

Teaching Object-Oriented design on the class diagram level is often a cumbersome effort. Requiring the use of specific design patterns helps the students in structuring their design properly. However, checking whether students used the right design pattern can be a very time-intensive task due to the variety of possibilities of creating structure using design patterns on the high-level class diagrams. For the same reason, it is hard for students to check for themselves whether their solution fulfills the basic requirements that are required by the instructor with respect to the use of design patterns. Efficiency and the quality of design pattern education can be improved by automatic detection of design patterns in \textsc{UML} class diagrams. We introduce a new method to detect design patterns in class diagrams, together with a prototype of a tool that uses this new method. Using this tool, an instructor needs less effort to review solutions of design exercises since the tool can check the basic class requirements automatically. Consequently, an instructor can focus on the more high-level requirements that were set in the exercise and students can easier check for themselves whether their design satisfies the basic required properties on the pattern level. The method offers static decidability for those design patterns, that are identified by structural properties e.c. the names of the classes and their associations. It is non-duplicating. That is a specific occurrence of a design pattern is not reported multiple times. The method not only detects all 16 static Gang of Four design patterns without false positives or false negatives, but also it can detect redundant relations. Our tool contributes tto the quality and efficiency of design pattern education, both for students and instructors.

Piloting an academic test automation tool in an industrial environment provides valuable feedback on practical applicability of the tool, but often requires a significant collaboration effort and an iterative process of feedback, development, and evaluation. In this paper, we propose an approach for continuous delivery of TESTAR open source test automation tool into an industrial continuous integration (CI) environment for piloting and evaluation.

Developing an Android app is a complex skill that is difficult for students to master because many constituent skills have to be processed simultaneously, which might cause a cognitive overload. Cognitive Load Theory (CLT) offers a framework to analyze and reduce the cognitive load, but the question is how to apply CLT in a systematic way during course design. The Four-Component Instructional Design (4C/ID) model incorporates CLT and offers additional instructional methods that improve learning. This paper reports on a case study applying 4C/ID to develop a blended course in Android app development. The following 4C/ID components were designed using the Ten Steps to Complex Learning (Ten Steps) approach: learning tasks, supportive information and procedural information. Each designed component or part thereof was categorized as face-to-face or online, for both lecturer and student, resulting in a blended learning design. The Ten Steps approach proved to be valuable in designing a blended course in Android app development. Our case study will be the starting point of further research.

The introduction of the new computing science curriculum in the Netherlands in 2019 raises the need for new evidence-based teaching materials that include practical assignments and guidelines for their assessment. As a part of our research project on teaching Computational Science (modeling and simulation), we participate in these efforts and developed a curriculum intervention including a practical assignment and an accompanying assessment instrument consisting of grading rubrics based on the SOLO taxonomy. In this research paper we focus on the assessment instrument. We describe its development and report on a pilot study carried out in the secondary computing science course implementing the curriculum intervention. The instrument proved to be reliable and effective in tracing high and low levels of the students' achievements in modeling and simulation projects and exposed the expected differences in performance levels of various groups of students, which renders it useful for both formative and summative assessment. Furthermore, our application of the instrument has provided new insights into the needs of specific groups of students to receive instruction prior to and during the work on the assignments.

Operating Rooms (OR’s) are complex, high tech environments with extensive use of medical equipment and information technology. The implementation of new medical equipment with the aim to increase safety, improve patient outcomes or to improve efficiency may initially cause disruptions in the OR, which influence its success. Between and within hospitals the implementation of medical equipment varies and a generic implementation model omits. The aim of this study is to identify factors for successful implementations according to surgical supportive staff. Results are compared with findings from other published studies. In total 90 out of 235 surveys were returned (38%). Respondents, scrub nurses and circulating nurses, indicate that implementation and integration of new medical equipment in current activities and ICT systems remain a challenge. In this study we identified the following factors: a coherent and holistic implementation approach; integration of medical equipment in processes, systems and organization; knowledge and skill development and effective communication during the implementation process.

Computing is an interdisciplinary field that can be approached from different points of view. Each point of view has its goals, aims and fundamental assumptions. This makes computing a complex discipline. Moreover, new computing disciplines appear regularly. With the trend that ICT-professionals should have non-ICT competences as well, and non-ICT-professionals should have ICT-competences, new computing curricula are often hybrid in nature. As a hybrid computing curriculum cannot cover the full range of computing, it is interesting to investigate the 'computing part' of such curricula. Our analysis framework consists of three elements: the curricular components 'goals and objectives' and 'instructional strategies', and the underlying epistemological view on the discipline ('cultural styles'). Taking a historical perspective, we describe the origins of the ACM/IEEE Curriculum Recommendation series. We discuss the three main cultural styles of computing: theoretical, scientific and engineering. Observing that in a curriculum the above elements should be aligned, we present three trade-offs for the case of hybrid computing curricula. We apply our results to two concrete examples, Liberal Arts and Computer Science and Front End Development. Based on our investigation, we formulate recommendations for designers of hybrid computing curricula. We recommend, for example, discussing disciplinary boundaries and resulting trade-offs explicitly while designing and documenting curricula.

The introduction of the new Informatics curriculum in the Netherlands in 2019 raises the need for new teaching material that includes practical assignments and guidelines for their assessment. As a part of our research project on teaching Computational Science (modeling and simulation), we participate in these efforts and developed a curriculum intervention and an assessment instrument consisting of a practical assignment and grading rubrics to assess student’s level of understanding. The rubrics we developed can be used both for formative and summative assessment. In this paper we describe the design of this assessment instrument and indicate further research directions focusing on validation of this instrument.

Item does not contain fulltext 264 p.

We explore a way to find energy deficiencies in code by model checking, both properties related to utilisation (energy hotspots) and not related to utilisation (energy bugs). Temporal logic properties (expressed using LTL) finds these deficiencies during the development of a system, employing a novel energy-inefficiency metric. The formal model is deduced from hardware specifications, a definition of utilisation and a control software application written in C. Model checking results in a set of traces (formally a counter-example) that can be related to the matching execution path in the source code by the software developer. The traces include energy footprints and enable developers to improve the energy efficiency of the application. A smart light system serves as a case study to evaluate the proposed approach.

In design-based education, students use scientific concepts to inform their artifact-making endeavors. On the other hand, artifact-making activities are meant to deepen students' conceptual understanding. However, no strategy has been described that explicitly links conceptual development and artifact-making endeavors in computer science design projects in an authentic way. To fill this gap, in this 'work in progress' study, we developed an instructional model for fusion designing and conceptual learning in CS education. A key component of the model is the 'intermediate design products' which play a critical role in connecting designing and conceptual learning. The model served to develop exemplary lesson materials meant to enhance students' algorithmic thinking. The materials were implemented and evaluated in four classes. The results suggest the model provides opportunities to improve students' algorithmic thinking. Furthermore, intermediate products turn out to be promising inputs for (formative) assessment of conceptual development in design projects by capturing and revealing students' misconceptions about basic CS concepts.

In 1998, informatics was introduced as an elective subject for all students in the upper grades of senior general secondary education and pre-university education in the Netherlands. Rather than focusing on digital literacy or the use of office applications, it focuses on informatics as a scientific discipline. In its first decade, it faced growing pains while fighting for recognition and necessary facilities from the stakeholders: students, parents, school administrators, politicians and the general public. In 2007, the curriculum was slightly streamlined but not updated. In its second decade, informatics reached adulthood with established teacher training programs and a new curriculum which is to be introduced in 2019. In this paper we describe the events and processes that led to the renewal of the curriculum, the curriculum itself with the principles it is based on and its aims, the current process of teaching material development, the related research, the teacher training, curriculum reform in primary and lower secondary education, and the current situation of informatics as an upper secondary school subject, together with the challenges it still faces.

The goal of this study is to investigate changes in PCK (Pedagogical Content Knowledge) of Dutch primary and secondary computer science teachers participating in a MOOC about Scratch programming. We captured the teachers’ PCK using identical pre- and post-questionnaires and conducted a qualitative deductive-inductive content analysis to identify changes in the PCK of the MOOC attendees. We relate the observed differences between PCK before and after the MOOC to Clarke and Hollingsworth’s model of teacher professional growth and Van Driel and Henze’s model of PCK development. Our analysis gives rise to four design principles meant to inform the pedagogical design of such MOOCs and improve their pedagogical affordances with regard to PCK development of their attendees.

In context-based education, authentic situations ('contexts') are used as starting points for learning content matter ('concepts'). In this way, contexts provide significance and meaning to the concepts taught. The context-based approach has been investigated extensively in the field of science education. Context-based education has the potential to be a useful strategy in computer science (CS), in particular for teaching and learning of fundamental concepts. Initiatives like Informatik im Kontext confirm that context-based teaching and learning is a promising approach. So far, however, little research has been done on particular aspects of context-based learning in CS, such as the effective selection of contexts, principles for connecting concepts and contexts, and mechanisms for fostering knowledge transfer. This work-in-progress paper reports on an ongoing qualitative study on context-based teaching of fundamental CS concepts connected to algorithmic thinking. The study focuses on experiences and ideas of teachers, as they play a key role in the adaptation of contexts stemming from a rapidly changing field. We conducted semi-structured interviews with CS teachers on the above aspects of context-based teaching. The results reveal various ideas that teachers have on the use and effects of context-based learning and raises questions about the selection of contexts.

Proper decision making represent one of an organization’s most important capabilities. To manage decisions and underlying business rules, an increasing number of organizations have begun to use business rules management (BRM). However, given BRM research’s and practice’s nascence, we need to more deeply understand the challenges in implementing BRM capabilities. As such, from collecting and analyzing two three-round focus groups and two three-round Delphi studies, we identified 28 main challenges that five Dutch governmental institutions experienced in eliciting, designing, and specifying business rules. We also discuss directions for future research.

The use of Bayesian networks has been shown to be powerful for supporting decision making, for example in a medical context. A particularly useful inference task is the most probable explanation (MPE), which provides the most likely assignment to all the random variables that is consistent with the given evidence. A downside of this MPE solution is that it is static and not very informative for (medical) domain experts. In our research to overcome this problem, we were inspired by recent research results on augmenting Bayesian networks with argumentation theory. We use arguments to generate explanations of the MPE solution in natural language to make it more understandable for the domain expert. Moreover, the approach allows decision makers to further explore explanations of different scenarios providing more insight why certain alternative explanations are considered less probable than the MPE solution.

Developing software is a team project that does not only rest on technologies and methodologies used, human aspects also play a pivotal role in the success of team projects. We investigate the impact of personality diversity on the success of short-lived student teams who develop software. We define a measurement of personality diversity in terms of personality types using the Myers Briggs Type Indicator (MBTI) personality dimensions. We observe the ratios of students in our cohort belonging to each of the four personality dimensions. Team success is measured in terms of team synergy and the quality of the end product. We hypothesise that diversity in a team is positively correlated with team synergy and the quality of their deliverable. We discuss the factors that may have contributed to our failure to confirm our hypotheses.

We present CoSTest, a tool that supports the validation of Conceptual Schemas by using testing. The tool implements techniques for transforming instantiations from a Requirements Model into test case implementations by supporting a Model-driven architecture.

While modern software development technologies enhance the capabilities of model-based/driven development, they introduce challenges for testers such as how to perform early testing at model level to ensure the quality of the model. In this context, we have developed an early testing technique supported by the CoSTest tool to validate requirements at model level. In this paper we describe an empirical evaluation of CoSTest with respect to its effectiveness in terms of its fault detection and test suite adequacy. This evaluation is carried out by model-level mutation testing using first order mutants (created by injection of a single fault) and high order mutants (containing more than one fault) with seven conceptual schemas (of different sizes) that represent the functionality of different software systems in different domains. Our findings show that the tests generated by CoSTest are effective at killing a large number of mutants. However, there are also some fault types (e.g. delete the references to a class attribute or an operation call in a constraint) that our test suites were not able to detect. CoSTest was more effective in terms of detecting fault types using high order mutants that first order mutants. Thus, CoSTest's effectiveness is affected by the mutant type tested.

In the software testing contest, practitioners and researcher's are invited to test their test approaches against similar approaches to evaluate pros and cons and which is perceivably the best. The 2017 iteration of the contest focused on Graphical User Interface-driven testing, which was evaluated on the testing tool TESTONA. The winner of the competition was announced at the closing ceremony of the international conference on software testing (ICST), 2017.

Energy consumption analysis of IT-controlled systems can play a major role in minimising the overall energy consumption of such IT systems, during the development phase, or for optimisation in the field. Recently, a precise energy analysis was developed, with the property of being parametric in the hardware. In principle, this creates the opportunity to analyse which is the best software implementation for given hardware, or the other way around: choose the best hardware for a given algorithm. The precise analysis was introduced for a very limited language: ECA. In this paper, several important steps are taken towards practical energy analysis. The ECA language is extended with common programming language features. The application domain is further explored, and threats to the validity are identified and discussed. Altogether, this constitutes an important step towards analysing energy consumption of IT-controlled systems in practice., In Proceedings DICE-FOPARA 2017, arXiv:1704.05169

Tools that perform automated software testing via the user interface rely on an action selection mechanism that at each step of the testing process decides what to do next. This mechanism is often based on random choice, a practice commonly referred to as monkey testing. In this work we evaluate a first approach to genetic programming (GP) for action selection that involves evolving IF-THEN-ELSE rules; we carry out experiments and compare the results with those obtained by random selection and also by \(\mathcal {Q}\)-learning, a reinforcement learning technique. Three applications are used as Software Under Test (SUT) in the experiments, two of which are proprietary desktop applications and the other one an open source web-based application. Statistical analysis is used to compare the three action selection techniques on the three SUTs; for this, a number of metrics are used that are valid even under the assumption that access to the source code is not available and testing is only possible via the GUI. Even at this preliminary stage, the analysis shows the potential of GP to evolve action selection mechanisms.

Existing software analysis methods for performanceand coverage are typically tied to the source code of software applications.In this work, we extend these methods to the Graphical User Interfaces (GUI) of applications, motivated by the desire to bring the user perspective into focus of software quality assurance and testing at the GUI level. We present and discuss various profiling procedures, their advantages and disadvantages, the arising challenges and the identified solutions. The identification and classification of the GUI components recorded during the monitoring process posed particular problems. The monitoring and collection of data could be well implemented, while detailed improvements in the evaluation of results are still necessary.

For real-time and embedded systems, limiting the consumption of time and memory resources is often an important part of the requirements. Being able to predict bounds on the consumption of such resources during the development process of the code can be of great value. In this paper, we focus mainly on memory-related bounds. Recent research results have advanced the state of the art of resource consumption analysis. In this paper, we present a toolset that makes it possible to apply these research results in practice for (real-time) systems enabling JAVA developers to analyse symbolic loop bounds, symbolic bounds on heap size and both symbolic and numeric bounds on stack size. We describe which theoretical additions were needed in order to achieve this. We give an overview of the capabilities of the RESANA (Radboud University Nijmegen, The Netherlands) toolset that is the result of this effort. The toolset can not only perform generally applicable analyses, but it also contains a part of the analysis that is dedicated to the developers' (real-time) virtual machine, such that the results apply directly to the actual development environment that is used in practice

Energy is becoming a key resource for IT systems. Hence, it can be essential for the success of a system under development to be able to derive and optimise its resource consumption. For large IT systems, compositionality is a key property in order to be applicable in practice. If such a method is hardware parametric, the effect of using different algorithms or running the same software on different hardware configurations can be studied. This article presents a hardware-parametric, compositional and precise type system to derive energy consumption functions. These energy functions describe the energy consumption behaviour of hardware controlled by the software. This type system has the potential to predict energy consumptions of algorithms and hardware configurations, which can be used on design level or for optimisation.

Online banking relies on user-owned home computers and mobile devices, all vulnerable to man-in-the-middle attacks which are used to steal money from bank accounts. Banks mitigate this by letting users verify information that originates from these untrusted devices. This is not user-friendly since the user has to process the same information twice. It also makes the user an unnecessary critical factor and risk in the security process. This paper concerns a case study of an information scheme which allows the user to enter critical information in a trusted device, which adds data necessary for the recipient to verify its integrity and authenticity. The output of the device is a code that contains the information and the additional verification data, which the user enters in the computer used for online banking. With this, the bank receives the information in a secure manner without requiring an additional check by the user, since the data is protected from the moment the user entered it in the trusted device. This proposal shows that mundane tasks for the user in online banking can be automated, which improves both security and usability.