Search

Gaze-control and gaze-assistance in software development tools have so far been explored in the setting of code editing, but other developer activities like code review could also benefit from this kind of tool support.In this paper, we present GANDER, a platform for user studies on gaze-assisted code review. As a proof of concept, we extend the platform with an assistant that highlights name relationships in the code under review based on gaze behavior, and we perform a user study with 7 participants. While the participants experience the interaction as overwhelming and lacking explicit actions (seen in other similar user studies), the study demonstrates the platform's capability for mobility, real-time gaze interaction, data logging, replay and analysis.

Training deep neural networks (DNNs) takes signifcant time and resources. A practice for expedited deployment is to use pre-trained deep neural networks (PTNNs), often from model zoos -- collections of PTNNs; yet, the reliability of model zoos remains unexamined. In the absence of an industry standard for the implementation and performance of PTNNs, engineers cannot confidently incorporate them into production systems. As a first step, discovering potential discrepancies between PTNNs across model zoos would reveal a threat to model zoo reliability. Prior works indicated existing variances in deep learning systems in terms of accuracy. However, broader measures of reliability for PTNNs from model zoos are unexplored. This work measures notable discrepancies between accuracy, latency, and architecture of 36 PTNNs across four model zoos. Among the top 10 discrepancies, we find differences of 1.23%-2.62% in accuracy and 9%-131% in latency. We also fnd mismatches in architecture for well-known DNN architectures (e.g., ResNet and AlexNet). Our findings call for future works on empirical validation, automated tools for measurement, and best practices for implementation., Proceedings of the 30th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering: Ideas, Visions, and Reflections track (ESEC/FSE-IVR) 2022

A concept inventory (CI) is a standardized assessment tool designed to evaluate a student's understanding of the fundamental concepts of a topic. To create a CI, it is necessary to accurately identify these concepts, but also to identify how poorly students understand them. The aim of this paper is to present an approach used to identify misconceptions related to the concept of Finite State Machine (FSM). In the learning process, identifying the students' misconceptions, i.e., when they appear and how to efficiently correct them, are important aspects of the best learning outcome. Rather than measuring understanding at a specific point in the learning timeline, the CI can be administered to students several times over the course of the learning period to measure how students' understanding of concepts changes. This preliminary study is composed of two main steps. In the first step, four misconceptions were identified about FSM based on multi-year observations and teacher experiences. From these misconceptions, seven statements about FSM are specified. In the second step, a Likert scale questionnaire (composed of seven statements) was administered five times to students according to a specific schedule, allowing to measure the evolution of FSM understanding. A pre-questionnaire is used to determine the students' misconceptions about the FSM concept, based on their learning (self-learning or from previous courses). This measure, which is the starting point of this preliminary study, makes it possible to highlight the changes in the students' positioning in relation to the statements provided and to link these changes to the teaching interventions. Thus, changes are clearly observable after the two theoretical classes, and stabilization is devoted after the delivery of the lab work.

There is often a scarcity of training data for machine learning (ML) classification and regression models in industrial production, especially for time-consuming or sparsely run manufacturing processes. A majority of the limited ground-truth data is used for training, while a handful of samples are left for testing. Here, the number of test samples is inadequate to properly evaluate the robustness of the ML models under test for classification and regression. Furthermore, the output of these ML models may be inaccurate or even fail if the input data differ from the expected. This is the case for ML models used in the Electroslag Remelting (ESR) process in the refined steel industry to predict the pressure in a vacuum chamber. A vacuum pumping event that occurs once a workday generates a few hundred samples in a year of pumping for training and testing. In the absence of adequate training and test samples, this paper first presents a method to generate a fresh set of augmented samples based on vacuum pumping principles. Based on the generated augmented samples, three test scenarios and one test oracle are presented to assess the robustness of an ML model used for production on an industrial scale. Experiments are conducted with real industrial production data obtained from Uddeholms AB steel company. The evaluations indicate that Ensemble and Neural Network are the most robust when trained on augmented data using the proposed testing strategy. The evaluation also demonstrates the proposed method's effectiveness in checking and improving ML algorithms' robustness in such situations. The work improves software testing's state-of-the-art robustness testing in similar settings. Finally, the paper presents an MLOps implementation of the proposed approach for real-time ML model prediction and action on the edge node and automated continuous delivery of ML software from the cloud., Comment: 11 pages

Harmful software has resulted in loss of life, societal and environmental damage alongside economic losses from fines and sales embargoes. When someone perceives their team or organisation is creating or operating harmful software (e.g., defective, vulnerable, malicious or illegal), one way to attempt to change the situation is to “blow the whistle” and disclose the situation internally or externally. Studying harmful situations and the effectiveness of interventions, up to and including whistleblowing, can help identify technical and human successes and failings in software engineering (SE). The aim of this paper is to explore the extent to which whistleblowing is studied in SE with the objective of identifying themes, research approaches, gaps and concerns, and the implications for future SE research and practice. We find that whistleblowing is an under-explored area of SE research, and where research exists, it often takes the view that reporting harm is a matter of individual moral responsibility; we argue this poorly reflects SE collaborative practice where professional responsibilities are distributed across the software development lifecycle. We conclude by 1) recommending approaches that can help a more timely identification and mitigation of harm in SE; 2) suggesting mechanisms for improving the effectiveness and the personal safety of harm-reporting in SE, and 3) reflecting on the role that professional bodies can have in supporting harm reporting, up to and including whistleblowing.

Background: Volvo Cars is pioneering an agile transformation on a large scale in the automotive industry. Social psychological aspects of automotive software development are an under-researched area in general. Few studies on team maturity or group dynamics can be found specifically in the automotive software engineering domain. Objective: This study is intended as an initial step to fill that gap by investigating the connection between issues and problem reports and team maturity. Method: We conducted a quantitative study with 84 participants from 14 teams and qualitatively validated the result with the Release Train Engineer having an overview of all the participating teams. Results: We find that the more mature a team is, the faster they seem to resolve issues as provided through external feedback, at least in the two initial team maturity stages. Conclusion: This study suggests that working on team dynamics might increase productivity in modern automotive software development departments, but this needs further investigation., 5 pages

International audience; Field-testing is costly and time-consuming, hence, simulation-based testing is becoming more and more important to validate autonomous systems. Since autonomous systems can be deployed in diverse environments, a significant amount of diversified test cases has to be created. TAF (Testing Automation Framework) is a test generation tool we developed to serve this purpose. It produces the test cases from a data model that specifies the virtual environments of interest. This paper presents a practitioner's view of the integration of TAF into simulation-based test platforms, through two industrial case studies. The first one is for testing an agricultural robot developed by Naïo Technologies, and the second one for a static perception system by SICK AG that surveils a road crossing to support connected vehicles with tracking data in complex urban scenarios. We report on our experience in the design of the data models, as well as in the automation of the execution, logging, and analysis of the generated tests. We conclude with lessons learned.

Background: Bugs and bug management consumes a significant amount of time and effort from software development organizations. A reduction in bugs can significantly improve the capacity for new feature development. Aims: We categorize and visualize dimensions of bug reports to identify accruing technical debt. This evidence can serve practitioners and decision makers not only as an argumentative basis for steering improvement efforts, but also as a starting point for root cause analysis, reducing overall bug inflow. Method: We implemented a tool, MultiDimEr, that analyzes and visualizes bug reports. The tool was implemented and evaluated at Ericsson. Results: We present our preliminary findings using the MultiDimEr for bug analysis, where we successfully identified components generating most of the bugs and bug trends within certain components. Conclusions: By analyzing the dimensions provided by MultiDimEr, we show that classifying and visualizing bug reports in different dimensions can stimulate discussions around bug hot spots as well as validating the accuracy of manually entered bug report attributes used in technical debt measurements such as fault slip through. © 2022 ACM. open access

Requirements Engineering (RE) is the branch of Software Engineering concerned with the real- world goals for, functions of, and constraints on software systems. RE is also concerned with the relationship of these factors to precise specifications of software behaviour and to their evolution over time and across software families. RE is increasingly recognized as a critical activity in any systems engineering process. Independently of the nature of the software, the elicitation, analysis, negotiation, specification, validation and management of requirements are fundamental for establishing quality in complex software. Only by fully understanding stakeholders' needs, and documenting them in a concise and unambiguous way, can we consistently deliver quality products designed to meet the complexity of our advanced information society.

Context: Software testing requires substantial effort in maintaining GUI test scripts, leading to rework or waste. Therefore, it is important to identify the factors leading to waste in GUI-based automation testing. Objective: To identify testability requirements and factors associated with waste from literature and practice. Methods: We conducted a literature review to identify the state of knowledge and a two-phased interview with practitioners to capture their subjective opinions and gather their views on the state of knowledge. Results: We identified 16 testability requirements and 14 factors associated with waste in GUI test automation. We classified each of them into SUT-related, test-process-related, test-tool-related, human and organizational, environment and cross-cutting. Conclusions: Our results indicate that the generic testability requirements identified in the literature are also valid for GUI automation testing. Our study identifies new findings in addition to the existing literature. © 2022 Owner/Author. Open access

Product line engineering model is suitable for engineering related software products in an efficient manner, taking advantage of their similarities while managing their differences. Our feature driven software product line (SPL) solution based on that model allows for instantiation of different CAVE products based on the set of core assets and driven by a set of common VR features with the minimal budget and time to market.

Inter-Process Communication (IPC) mechanisms are simple, OS-provided communication endpoints that do not typically accommodate program-level needs on latency, resource utilization, and mobility. But modern network-connected devices, particularly in IoT, have a wide variety of custom needs and frugal capabilities that standard networking stacks and programming interfaces do not cater for well. Thus IPC needs to evolve, but programmers would need to commit to new communication choices through their source code, which is difficult to change. This position paper argues for the reimagining of IPC to benefit IoT through program-level tailoring of composable and reusable protocol building-blocks for computation-and data-management in distributed systems. We propose sprockets, a generalization of RPC beyond marshalling and synchronization. It incorporates programmer annotations about code semantics, program-level network-related functions, and performance expectations. This is a stepping stone towards the declarative synthesis of high-level IPC that better meets the program's communication needs.

Building customised IoT applications is often labor intensive. While several frameworks support developers in building sophisticated applications, they mainly focus on the communication with the underlying devices and their automation. Modular programming may further improve the reuse of components in other applications. However, integrating automatic device configuration, connection management, and user support for solving connectivity issues, remains a real endeavor for application developers. Likewise, integrating and enforcing policies in an IoT application is complex, and the dynamic nature of IoT systems makes it even more difficult to develop applications that properly handle changes in the environment. This paper presents a middleware that simplifies the development of smart IoT applications. The middleware hosts a modular, event-based, logic reasoner. The reasoner is developed in Prolog and communicates with the underlying IoT framework. The middleware leverages the actual benefits of Prolog through complex querying and inference capabilities on the IoT system. This, combined with built-in functionality for automatic device and connection management, access control, advanced automation and an abstraction module that decouples the applications from the underlying infrastructure, grants the developer to focus on the specific application. An integration in a server and a mobile application demonstrates the capabilities of the middleware and its modules.

In scientific research, the independent reproduction of experimental results is the source of trust. The release of experimental artifacts enables the reproduction of results; however, additional efforts of researchers are required to prepare and document their experiments accordingly. To honor this increased effort, multiple initiatives were implemented to incentivize the creation and release of experimental artifacts, e.g., awards for papers that provide experimental artifacts. In this work, we want to propose a novel approach toward a reproducible research---a structured experimental workflow that allows the creation of reproducible experiments without requiring additional efforts of the researcher. Moreover, we present our own testbed and toolchain, namely, plain orchestrating service (pos), which enables the creation of such experimental workflows. The experiment is documented by our proposed, fully scripted experiment structure. Further, we consider the entire experimental workflow from experiment orchestration, to data measurement, to result evaluation. In addition, pos provides scripts that enable the automation of the bundling and release of all the created experimental artifacts. In this case study, we release one of our own experiments together with the necessary tools so that others can reproduce our experiment. Additionally, we provide an interactive environment where pos experiments can be executed and reproduced, which is available at https://gallenmu.github.io/pos-artifacts.

Security is a cross-cutting issue in the automotive development process. The nature of cross-cutting issues demands constant coordination between different stakeholders. Changes in the vehicle functionalities lead to reoccurring security analysis steps, rising the complexity of progress tracking. While those process steps are typically done on function level, the vehicle architecture has to be verified as a composite, too. This is mostly done late in the development process by testing. Thus, architectural mismatches between functionalities security demands are often revealed too late. Starting from the definition of integrity as a system property in the information flow, we present the link from the MoRA approach to the architectural modeling and analysis approach. Verifying the no command-up policy is transferred to the temporal logic TLA+ allowing an early and fast architecture verification.

Accessibility in computing, when well applied, allows for the inclusion of users and the breaking of barriers in the interaction process. However, approaches to support accessibility validation are commonly unknown or ignored. The objective of this work is to understand accessibility issues considered in the software development process. This research has carried out an initial literature systematic mapping. The result indicates that there are gaps in the use of smart approaches to assess accessibility during the software development process.

While modern software development tools provide programmers with features that help locating and fixing errors in the code, using these tools is not necessarily easy for novice programmers. We call for methods of instruction that teach students how to fix errors in practice using a modern, industry-grade, IDE. We suggest an approach—utilizing tutorials embedded in an IDE— that we think could be suitable for this purpose. Our ongoing and future work include implementing and evaluating that approach.

By contributing to open source software (OSS), students can gain professional software development experience and learn about applications of computer science (CS) concepts in pragmatic contexts. However, integrating such projects in classrooms requires substantial logistical planning by instructors as well as adequate programming skills from students. To mitigate these challenges, we propose four model curricula to serve as accessible strategies of integrating practicable learning opportunities in lower-level CS classes. Depending on classroom circumstances, instructors can assign projects that involve student contributions to OSS, custom plug-ins, simulated open source communities, or practical code excerpts. As a result, students will be able to explore the utility of CS and discover an exciting future in computing.

We introduce Sinergym, an open-source building simulation and control framework for training reinforcement learning agents. The proposed framework is compatible with EnergyPlus models and allows to implement Python-based controllers, facilitating reproducibility of experiments and generalization to multiple scenarios. A comparison between Sinergym and other existing libraries for building control is included. We describe its design and main functionalities, such as offering a diverse set of environments with different buildings, weather types and action spaces. The provided examples show the usage of the framework for benchmarking reinforcement learning methods for building control.

Battery-less energy-harvesting systems have widened the landscape of Internet-of-Things (IoT) applications by taking computation to hard-to-reach places. Energy-harvesting sensors are perpetual, environment-friendly, cost-effective, and maintenance-free. Despite having such lucrative characteristics, battery-powered devices hold majority share of today's IoT market, since developing energy-harvesting applications require more expert knowledge, careful implementation, and rigorous debugging than applications with stable power. In this paper, we argue that development becomes easier, faster, efficient, and scalable with a standard, re-usable, general purpose platform that ensures the platform's versatility across various application with proper balance between abstraction and accessibility in hardware and software. Such platforms would provide flexibility across both hardware and software layers, at the same time, producing reliable performance. However, realizing this design point pose several research challenges that need to be identified and addressed. We identify the limitations in existing systems, articulate the challenges and provide guidelines for the community to work towards a general purpose platform that would enable new diversified battery-less applications in the future.

This paper proposes a framework for automatic exploit generation in JIT compilers, focusing in particular on heap corruption vulnerabilities triggered by dynamic code, i.e., code generated at runtime by the JIT compiler. The purpose is to help assess the severity of vulnerabilities and thereby assist with vulnerability triage. The framework consists of two components: the first extracts high-level representations of exploitation primitives from existing exploits, and the second uses the primitives so extracted to construct exploits for new bugs. We are currently building a prototype implementation of the framework focusing on JavaScript JIT compilers. To the best of our knowledge, this is the first proposal to consider automatic exploit generation for code generated dynamically by JIT compilers.

Fundamental research into in situ visualization and analysis techniques is difficult to access for visualization researchers due to the overwhelming complexity and effort of constructing and managing in situ software stacks that allow reproducible evaluation of novel in situ visualization and analysis techniques. To address this problem, we describe EZ-ISAV, a work-in-progress towards a framework for easy construction of customizable in situ pipelines in container images. Designed for portability and ease of use, these images are intended to serve as proof-of-concept cases for in situ visualization and analysis research. Furthermore, we describe the EZ-ISAV repository, an open repository of turn-key, ready-made container images that can strongly reduce the overhead of developing and evaluating in situ techniques and provide improved reproducibility and portability of in situ visualization research.

While software protection mechanisms, such as DRM and online services, hinder the unrestrained duplication of games and applications, these mechanisms fail at protecting individual software components from reuse by intellectual property thieves. While conceptually watermarking can discourage software misuse and allows proof of ownership, embedding watermarks requires expert knowledge. This is why software watermarks are barely used. This paper presents LLWM, an LLVM-based watermarking framework that automates the embedding of watermarks and thus enables the widespread use of watermarking. LLWM incorporates several (adapted) implementations of existing watermarking techniques and provides the foundation for IR-Mark, a new watermarking technique based on a modified register allocation. With LLWM, built upon LLVM and its compiler Clang, developers can use a watermark simply by compiling their codes, without the obstacle of having to understand and run existing methods and standalone tools. With the methods included in LLWM, we offer a variety of choices for embedding techniques and evaluate and discuss their characteristics and resilience against common attacks.

Safe system programming is often a crucial requirement due to its critical role in system software engineering. Conventional low-level programming languages such as C and assembly are efficient, but their inherent unsafe nature makes it undesirable for security-critical scenarios. Recently, Rust has become a promising alternative for safe system-level programming. While giving programmers fine-grained hardware control, its strong type system enforces many security properties including memory safety. However, Rust's security guarantee is not a silver bullet. Runtime crashes and memory-safety errors still harass Rust developers, causing damaging exploitable vulnerabilities, as reported by numerous studies. In this paper, we present and evaluate MirChecker, a fully automated bug detection framework for Rust programs by performing static analysis on Rust's Mid-level Intermediate Representation (MIR). Based on the observation of existing bugs found in Rust codebases, our approach keeps track of both numerical and symbolic information, detects potential runtime crashes and memory-safety errors by using constraint solving techniques, and outputs informative diagnostics to users. We evaluate MirChecker on both buggy code snippets extracted from existing Common Vulnerabilities and Exposures (CVE) and real-world Rust codebases. Our experiments show that MirChecker can detect all the issues in our code snippets, and is capable of performing bug finding in real-world scenarios, where it detected a total of 33 previously unknown bugs including 16 memory-safety issues from 12 Rust packages (crates) with an acceptable false-positive rate.

Kernel drivers are a critical part of the attack surface since they constitute a large fraction of kernel codebase and oftentimes lack proper vetting, especially for those closed-source ones. Unfortunately, the complex input structure and unknown relationships/dependencies among interfaces make them very challenging to understand. Thus, security analysts primarily rely on manual audit for interface recovery to generate meaningful fuzzing test cases. In this paper, we present SyzGen, a first attempt to automate the generation of syscall specifications for closed-source macOS drivers and facilitate interface-aware fuzzing. We leverage two insights to overcome the challenges of binary analysis: (1) iterative refinement of syscall knowledge and (2) extraction and extrapolation of dependencies from a small number of execution traces. We evaluated our approach on 25 targets. The results show that SyzGen can effectively produce high-quality specifications, leading to 34 bugs, including one that attackers can exploit to escalate privilege, and 2 CVEs to date.

This presentation is based on the paper "Search-based Local Blackbox Deobfuscation: Understand Improve and Mitigate'' from the same authors, which has been accepted for publication at ACM CCS 2021. Code obfuscation aims at protecting Intellectual Property and other secrets embedded into software from being retrieved. Recent works leverage advances in artificial intelligence (AI) with the hope of getting blackbox deobfuscators completely immune to standard (whitebox) protection mechanisms. While promising, this new field of AI-based, and more specifically search-based blackbox deobfuscation, is still in its infancy. In this article we deepen the state of search-based blackbox deobfuscation in three key directions: understand the current state-of-the-art, improve over it and design dedicated protection mechanisms. In particular, we define a novel generic framework for search-based blackbox deobfuscation encompassing prior work and highlighting key components; we are the first to point out that the search space underlying code deobfuscation is too unstable for simulation-based methods (e.g., Monte Carlo Tree Search used in prior work) and advocate the use of robust methods such as S-metaheuristics; we propose the new optimized search-based blackbox deobfuscator Xyntia which significantly outperforms prior work in terms of success rate (especially with small time budget) while being completely immune to the most recent anti-analysis code obfuscation methods; and finally we propose two novel protections against search-based blackbox deobfuscation, allowing to counter Xyntia powerful attacks.

Smartphones apps aid humans in plenty of situations. There exists an app for everything. However, without the user’s awareness, some apps contain vulnerabilities or leak private data. Static and dynamic app analysis are ways to find these software properties. Especially setting up a dynamic analysis environment is not a trivial task. Several peculiarities of Android have to be considered, existing tools for different aspects have to be evaluated, selected and setup to work together. Existing literature is often outdated and only covers tools for one aspect but doesn’t combine them together in a big picture. This paper presents a generic design for an automated dynamic app analysis environment and highlights the required components as well as functionality to reveal security and privacy issues. Available tools are listed, realizing different aspects of the proposed environment design. Tool features are evaluated and tool usability for an automated large scale dynamic app analysis is compared. This document should serve as a reference to all who need to implement dynamic analysis on Android (or some aspects) and require an overview of available and usable solutions.

Professional fact-checking tends to be costly and scalability challenging. For this reason, a series of methods to automate this process has been emerging. Unfortunately, these methods have been created with monolithic architectures, not making use of pre-built parts, and are harder to be understood by others. This work aims to propose a framework, where the development of methods can be done in a modular manner, creating workflows based on key steps that can be linked together to generate the input classification. For this, a study of the literature was conducted to identify the main processing steps of the methods, seeking to develop a solution that would guarantee flexibility of execution and interoperability between the different processing components. With this, the automatic verification process was described in a limited set of steps that can be performed, creating independent components that implement each of them. The framework makes use of a proposed data traceability ontology to map all generated data during execution under a unified vocabulary, which facilitates communication between these independent components.

Widgets are part of many web applications and implement distinct interaction mechanisms, in reusable user interface components. However, implementing accessible widgets is a challenge, associated to different factors highlighted in the state-of-the-art, such as: lack of developers knowledge of accessibility guidelines, organization support and tools that support accessible development. This paper reports an investigation on JavaScript source-code complexity impacts of implementing accessibility requirements in widgets. We investigated Dialog, Drop-down menu and Tab widgets of 27 open-source JavaScript libraries. The results showed that widget software artifacts reported an increased average of source-code complexity when implementing accessibility requirements. Results also showed that using DOM-oriented JavaScript libraries (eg. jQuery) significantly increased source-code complexity of widgets. Nevertheless, when using Declarative Component-based JavaScript libraries, no significant difference in source-code complexity was observed. These results suggest that using Declarative Component-based widget libraries might reduce maintainability costs of JavaScript front-end projects.

The concept of coverages, generally grasping multi-dimensional space/time varying phenomena, has received impressive attention among service implementers and operators. The biggest reason for this success is that coverages conveniently model datacubes, specifically in combination with powerful APIs such as the Web Coverage Service (WCS) with its datacube analytics language, Web Coverage Processing Service (WCPS). OGC, ISO, and EU INSPIRE capitalize on coverages, and leading tools implement them. In this tutorial, we first briefly recapitulate the coverage model and simple access with WCS Core, address the status of OAPI-Coverages, and then proceed to datacube analytics with WPCS. Practical demos based on the EarthServer datacube federation serve to illustrate; participants can recap and modify most of the demos. Altogether, this workshop constitutes a unique opportunity for getting up to speed with coverages and datacubes.

With the increase of traffic prediction models, there has become an urgent need to develop a standardized framework to implement and evaluate these methods. This paper presents LibCity, a unified, comprehensive, and extensible library for traffic prediction, which provides researchers with a credible experimental tool and a convenient development framework. In this library, we reproduce 42 traffic prediction models and collect 29 spatial-temporal datasets, which allows researchers to conduct comprehensive experiments in a convenient way. To accelerate the development of new models, we design unified model interfaces based on unified data formats, which effectively encapsulate the details of the implementation. To verify the effectiveness of our implementations, we also report the reproducibility comparison results of LibCity, and set up a performance leaderboard for the four kinds of traffic prediction tasks. Our library will contribute to the standardization and reproducibility in the field of traffic prediction. The open source link of LibCity is https://github.com/LibCity/Bigscity-LibCity.

In this paper, we demonstrate how to conduct OD analysis based on big taxi trajectory data with XStar in an efficient manner. XStar, originally developed by the first author, is a standalone software system dedicated to trajectory-data users with little programming skills and affordable computing devices. Since its release in Jan. 2019, it has received downloads of over 4000 by May 2021.

Geospatial research and education have become increasingly dependent on cyberGIS to tackle computation and data challenges. However, the use of advanced cyberinfrastructure resources for geospatial research and education is extremely challenging due to both high learning curve for users and high software development and integration costs for developers, due to limited availability of middleware tools available to make such resources easily accessible. This tutorial describes CyberGIS-Compute as a middleware framework that addresses these challenges and provides access to high-performance resources through simple easy to use interfaces. The CyberGIS-Compute framework provides an easy to use application interface and a Python SDK to provide access to CyberGIS capabilities, allowing geospatial applications to easily scale and employ advanced cyberinfrastructure resources. In this tutorial, we will first start with the basics of CyberGIS-Jupyter and CyberGIS-Compute, then introduce the Python SDK for CyberGIS-Compute with a simple Hello World example. Then, we will take multiple real-world geospatial applications use-cases like spatial accessibility and wildfire evacuation simulation using agent based modeling. We will also provide pointers on how to contribute applications to the CyberGIS-Compute framework.

Microservices Architectures (MSAs) are continuously replacing monolithic systems toward achieving more flexible and maintainable service-oriented software systems. However, the shift toward an MSA also requires a technological and managerial shift for its adopters. Architecting and managing MSAs represent unique challenges, including microservices' identification, interoperability, and reuse. To handle these challenges, we propose an Ontology-driven Conceptual Modelling approach, based on the Ontology of Microservices Architecture Concepts (OMSAC), for modelling and analyzing microservices-based systems. We show, how OMSAC-based conceptual models, stocked in a Stardog triple store, support Stakeholder-specific communication, documentation, and reuse. This paper reports on the application of our approach in three open-source MSA systems with a focus on microservices' discovery based on similarity metrics. Eventually, we compare the extracted similarity metrics derived from the application of machine learning techniques to the OMSAC models with a manual analysis performed by experts.

Cloud services must typically be distributed across a large number of machines in order to make use of multiple compute and storage resources. This opens the programmer to several sources of complexity such as concurrency, order of message delivery, lossy network, timeouts and failures, all of which impose a high cognitive burden. This paper presents evidence that technology inspired by formal-methods, delivered as part of a programming framework, can help address these challenges. In particular, we describe the experience of several engineering teams in Microsoft Azure that used the open-source Coyote Actor programming framework to build multiple reliable cloud services. Coyote Actors impose a principled design pattern that allows writing formal specifications alongside production code that can be systematically tested, without deviating from routine engineering practices. Engineering teams that have been using Coyote have reported dramatically increased productivity (in time taken to push new features to production) as well as services that have been running live for months without any issues in features developed and tested with Coyote.

Companies today increasingly rely on microservice architectures to deliver service for their large-scale mobile or web applications. However, not all developers working on these applications are distributed systems engineers and therefore do not anticipate partial failure: where one or more of the dependencies of their service might be unavailable once deployed into production. Therefore, it is paramount that these issues be raised early and often, ideally in a testing environment or before the code ships to production. In this paper, we present an approach called service-level fault injection testing and a prototype implementation called Filibuster, that can be used to systematically identify resilience issues early in the development of microservice applications. Filibuster combines static analysis and concolicstyle execution with a novel dynamic reduction algorithm to extend existing functional test suites to cover failure scenarios with minimal developer effort. To demonstrate the applicability of our tool, we present a corpus of 4 real-world industrial microservice applications containing bugs. These applications and bugs are taken from publicly available information of chaos engineering experiments run by large companies in production. We then demonstrate how all of these chaos experiments could have been run during development instead, and the bugs they discovered detected long before they ended up in production.

Cloud maturity and popularity have resulted in Open source software (OSS) proliferation. And, in turn, managing OSS code quality has become critical in ensuring sustainable Cloud growth. On this front, AI modeling has gained popularity in source code understanding tasks, promoted by the ready availability of large open codebases. However, we have been observing certain peculiarities with these black-boxes, motivating a call for their reliability to be verified before offsetting traditional code analysis. In this work, we highlight and organize different reliability issues affecting AI-for-code into three stages of an AI pipeline- data collection, model training, and prediction analysis. We highlight the need for concerted efforts from the research community to ensure credibility, accountability, and traceability for AI-for-code. For each stage, we discuss unique opportunities afforded by the source code and software engineering setting to improve AI reliability.

Summarization: FlinkCEP is the Complex Event Processing (CEP) API of the Flink Big Data platform. The high expressive power of the language of FlinkCEP comes at the cost of cumbersome parameterization of the queried patterns, acting as a barrier for FlinkCEP's adoption. Moreover, properly configuring a FlinkCEP program to run over a computer cluster requires advanced skills on modern hardware administration which non-expert programmers do not possess. In this work (i) we build a novel, logical CEP operator that receives CEP pattern queries in the form of extended regular expressions and seamlessly re-writes them to FlinkCEP programs, (ii) we build a CEP Optimizer that automatically decides good job configurations for these FlinkCEP programs. We also present an experimental evaluation which demonstrates the significant benefits of our approach. Παρουσιάστηκε στο: 30th ACM International Conference on Information & Knowledge Management

Machine Learning methods have been adopted for a wide range of real-world applications, ranging from social networks, online image/video-sharing platforms, and e-commerce to education, healthcare, etc. However, in practice, a large amount of effort is required to tune several components of machine learning methods, including data representation, hyperparameter, and model architecture, in order to achieve a good performance. To alleviate the required tunning efforts, Automated Machine Learning (AutoML), which can automate the process of applying machine learning methods, has been studied in both academy and industry recently. In this tutorial, we will introduce the main research topics of AutoML, including Hyperparameter Optimization, Neural Architecture Search, and Meta-Learning. Two emerging topics of AutoML, Automatic Feature Generation and Machine Learning Guided Database, will also be discussed since they are important components for real-world applications. For each topic, we will motivate it with application examples from industry, illustrate the state-of-the-art methodologies, and discuss some future research directions based on our experience from industry and the trends in academy.

We present PyTFL, a library written in Python for the team formation task. In team formation task, the main objective is to form a team of experts given a set of skills. We demonstrate an efficient and well-structured open-source toolkit that can easily be imported into Python. Our toolkit incorporates state-of-the-art approaches for team formation, e.g., neural-based team formation, and supports team formation sub-tasks such as collaboration graph preparation, model training and validation, systematic evaluation based on qualitative and quantitative team metrics, and efficient team formation and prediction. While there are strong research papers on the team formation problem, PyTFL is the first toolkit to be publicly released for this purpose.

This paper reports our experience applying lightweight formal methods to validate the correctness of ShardStore, a new key-value storage node implementation for the Amazon S3 cloud object storage service. By "lightweight formal methods"we mean a pragmatic approach to verifying the correctness of a production storage node that is under ongoing feature development by a full-time engineering team. We do not aim to achieve full formal verification, but instead emphasize automation, usability, and the ability to continually ensure correctness as both software and its specification evolve over time. Our approach decomposes correctness into independent properties, each checked by the most appropriate tool, and develops executable reference models as specifications to be checked against the implementation. Our work has prevented 16 issues from reaching production, including subtle crash consistency and concurrency problems, and has been extended by non-formal-methods experts to check new features and properties as ShardStore has evolved., SOSP '21: Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles, ISBN:978-1-4503-8709-5

In this paper, we report on our 6-year experience developing Eagle Tester (eTester for short) - a mobile battery drain testing and diagnostic tool. We show how eTester evolved from an "academic" prototype to a fully automated tool usable by the mobile industry. We first present the design of our initial research prototype and discuss 8 key requirements for a usable battery drain testing and diagnostic tool gathered from some of the most popular software vendors in the Android ecosystem. These requirements posed interesting scientific and engineering challenges such as how to accurately estimate battery drain without requiring a priori power modeling, work on unmodified devices, and automatically monitor code evolution to generate high-fidelity battery spike alerts with actionable insights. These requirements motivated a complete overhaul of the eTester design and led to the creation of a novel battery drain testing methodology. We show how the redesigned eTester was used to effortlessly find battery bugs in some of the most popular Android apps with hundreds of millions of users, such as Netflix and CNN. We are open-sourcing eTester to encourage further research in battery diagnosis and to empower developers to write battery-efficient mobile software.

Developing a new programming language, constructing a new domain-specific compiler, writing a new verification tool, optimizing a large application, designing a microprocessor, or verifying some of its components, all of these tasks require today a multi-year project. While most of the underlying problems are inherently hard and cannot be accelerated magically, we are additionally slowed down by a lack of well-defined interfaces that prevent us to exploit synergies between CS sub-communities. In this presentation, I raise the question of how we can accelerate the innovation speed of our CS technology stack to levels recently seen in deep learning, battery electric vehicles, or rocket launches. While I won't provide an answer, I will share the latest developments from the LLVM compiler community where the recent introduction of MLIR initiated the design of numerous IR abstractions that can be freely composed to build hybrid tools crossing community boundaries, that can be analyzed to gain a deep understanding of the various IR abstractions, and which may be the seed of a new abstraction sharing economy in our community. I will share some of my very own steps in this space on analyzing and understanding the various IR abstractions already in existence and will point out new cross-community collaboration opportunities. This talk concludes by raising the question of how we as researchers can build impactful and lasting open-source communities to move from interfacing software to towards building bridges between communities.

MoarVM was born out of youthful arrogance. I was contributing to Raku (then Perl 6) compiler development, looking at the Parrot virtual machine we were targeting at the time, considering some of our challenges with it (especially regarding performance and threading), and thought: what if I could design and build something better…in my copious free time? Within a few years, helped along by the contributions of tens of other open source volunteers, MoarVM almost entirely replaced Parrot as the language’s runtime of choice. Nearly a decade has passed since work on MoarVM began. Starting out as a simple register-based bytecode interpreter, MoarVM has steadily incorporated many of the tricks of the trade: type specialization, deoptimization, inlining, on stack replacement, JIT compilation, and basic escape analysis. As these fell into place, they drove a further change: hot operations that had been implemented as complex primitives in the VM for speed steadily became bottlenecks for further improvement, as the optimizer had no visibility into them, and thus were gradually eliminated. Most recently, a new generalized dispatch mechanism has arrived, eliminating numerous special-cased mechanisms (for example, method and multiple dispatch caches) with a single, programmable, approach. Being a multi-language VM has never been a goal - yet achieving better performance while managing complexity has led MoarVM to become ever more abstracted from the Raku language. In this session I will review MoarVM’s journey so far, picking out some of the most interesting challenges faced, lessons learned, and trade-offs encountered. I will also discuss the new generalized dispatch mechanism and its concept of resumable dispatch, which is allowing us to take on optimization of some language features that have thus far been stubbornly slow - but which also brings its own share of new challenges.

In recent times, Machine Learning (ML) techniques have been explored in everyday systems, such as banking systems, virtual assistants, and recommendation systems. To use ML techniques, it is necessary to perform some stages until the construction of the ML model. Within the ML model construction pipeline, there may be a need to develop systems that support the stages of this construction, such as the data labeling stage. For this stage, the User Interface (UI) of the systems must be designed so that the user does not feel bothered or bored during labeling, as this can decrease the quality of data labeling. If the data is wrong labeled, the ML model will learn incorrectly, leaving this model unusable, as it does not match the reality. Previous research focused on supporting the development of data labeling systems, but there is still a gap in guiding developers on how to create such UI for these systems. This paper aims to present the construction of guidelines for the development of data labeling systems UI that meet Nielsen's heuristics for good user interaction. To evaluate these guidelines, we conducted a preliminary study where ML developers used them to build a UI prototype for a labeling system and provided their impressions. Based on the developer's impressions, we analyzed each guideline, associating it to Nielsen's heuristics. Our results showed that the use of guidelines helps the developers during the construction of a UI for data labeling systems and that these guidelines meet some of Nielsen's heuristics.

The increasing use of artificial intelligence technologies in several areas of knowledge highlights the need to design solutions for people with different backgrounds and skills. Since data collection to use such technologies and obtain reliable results is still a challenge, the statistical technique of Design of Experiments (DOE) can be applied to address this issue gathering less biased and more balanced data. However, to apply DOE effectively is required a high level of specific statistical knowledge, making difficult for most people to use the technique. In this paper we discuss the design and evaluation of a software prototype for people to apply DOE without statistical knowledge. The results of prototype tests reveal that some technical terms are difficult to understand, even when explained. Despite this, results also show that real-life examples were helpful for participants understand concepts related to DOE. We hope our results can inspire Human-Computer Interaction community to design solutions for artificial intelligence technologies, making it more accessible for people from different fields. We believe this multidisciplinary approach contributes to solve real people's needs.

Block-based programming languages, and Scratch in particular, are widely used to introduce young learners to programming. As these students progress through their education, they want or need to transition to using text-based systems and encounter a number of challenges as they do so. Issues with syntax, text editing, and memorisation are all significant, but the change of programming paradigm is also a challenge. This paper discusses the design and development of a system to help students make the transition to text-based programming environments more easily. Sprites, animations and sound form the basis of most Scratch programs and these engaging features become unavailable at the same time as students are facing transition difficulties related to text editing and program structure. From programs designed in an actor-based event-driven system with easy concurrency, students have to move to procedural or class-based programs where multimedia features are accessed quite differently and programs are designed around explicit event loops. In this paper we introduce a new programming system, Pytch, which embodies “Scratch-Oriented programming” in Python. Using a web-based environment that requires no local setup, students can build Python programs using the familiar sprites and concurrent event-driven model learned in Scratch. The system offers the programming model inspired by Scratch through a Python library and a runtime augmented with a form of managed concurrency. The motivation and related work are discussed, and the system is presented in its current form. The next stage will be to evaluate the effectiveness of the system with users.