Search

Experimentation is an essential method for causal inference in any empirical discipline. Crossover-design experiments are common in Software Engineering (SE) research. In these, subjects apply more than one treatment in different orders. This design increases the amount of obtained data and deals with subject variability but introduces threats to internal validity like the learning and carryover effect. Vegas et al. reviewed the state of practice for crossover designs in SE research and provided guidelines on how to address its threats during data analysis while still harnessing its benefits. In this paper, we reflect on the impact of these guidelines and review the state of analysis of crossover design experiments in SE publications between 2015 and March 2024. To this end, by conducting a forward snowballing of the guidelines, we survey 136 publications reporting 67 crossover-design experiments and evaluate their data analysis against the provided guidelines. The results show that the validity of data analyses has improved compared to the original state of analysis. Still, despite the explicit guidelines, only 29.5% of all threats to validity were addressed properly. While the maturation and the optimal sequence threats are properly addressed in 35.8% and 38.8% of all studies in our sample respectively, the carryover threat is only modeled in about 3% of the observed cases. The lack of adherence to the analysis guidelines threatens the validity of the conclusions drawn from crossover design experiments

Test-driven development (TDD) is a widely used agile practice. However, very little is known with certainty about TDD's underlying foundations, i.e., the way TDD works. In this paper, we propose a theoretical framework for TDD, with the following characteristics: 1) Each TDD cycle represents a vertical slice of a (probably also small) user story, 2) vertical slices are captured using contracts, implicit in the developers' minds, and 3) the code created during a TDD cycle is a sliced-based specification of a code oracle, using the contracts as slicing pre/post-conditions. We have checked the connections among TDD, contracts, and slices using a controlled experiment conducted in the industry.

Experiments are a commonly used method of research in software engineering (SE). Researchers report their experiments following detailed guidelines. However, researchers do not, in the field of test-driven development (TDD) at least, specify how they operationalized the response variables and the measurement process. This article has three aims: (i) identify the response variable operationalization components in TDD experiments that study external quality; (ii) study their influence on the experimental results;(ii) determine if the experiment reports describe the measurement process components that have an impact on the results. Sequential mixed method. The first part of the research adopts a quantitative approach applying a statistical an\'alisis (SA) of the impact of the operationalization components on the experimental results. The second part follows on with a qualitative approach applying a systematic mapping study (SMS). The test suites, intervention types and measurers have an influence on the measurements and results of the SA of TDD experiments in SE. The test suites have a major impact on both the measurements and the results of the experiments. The intervention type has less impact on the results than on the measurements. While the measurers have an impact on the measurements, this is not transferred to the experimental results. On the other hand, the results of our SMS confirm that TDD experiments do not usually report either the test suites, the test case generation method, or the details of how external quality was measured. A measurement protocol should be used to assure that the measurements made by different measurers are similar. It is necessary to report the test cases, the experimental task and the intervention type in order to be able to reproduce the measurements and SA, as well as to replicate experiments and build dependable families of experiments.

A recurring problem in software development is incorrect decision making on the techniques, methods and tools to be used. Mostly, these decisions are based on developers' perceptions about them. A factor influencing people's perceptions is past experience, but it is not the only one. In this research, we aim to discover how well the perceptions of the defect detection effectiveness of different techniques match their real effectiveness in the absence of prior experience. To do this, we conduct an empirical study plus a replication. During the original study, we conduct a controlled experiment with students applying two testing techniques and a code review technique. At the end of the experiment, they take a survey to find out which technique they perceive to be most effective. The results show that participants' perceptions are wrong and that this mismatch is costly in terms of quality. In order to gain further insight into the results, we replicate the controlled experiment and extend the survey to include questions about participants' opinions on the techniques and programs. The results of the replicated study confirm the findings of the original study and suggest that participants' perceptions might be based not on their opinions about complexity or preferences for techniques but on how well they think that they have applied the techniques.

Context: Experiment replications play a central role in the scientific method. Although software engineering experimentation has matured a great deal, the number of experiment replications is still relatively small. Software engineering experiments are composed of complex concepts, procedures and artefacts. Laboratory packages are a means of transfer-ring knowledge among researchers to facilitate experiment replications. Objective: This paper investigates the experiment replication process to find out what information is needed to successfully replicate an experiment. Our objective is to propose the content and structure of laboratory packages for software engineering experiments. Method: We evaluated seven replications of three different families of experiments. Each replication had a different experimenter who was, at the time, unfamiliar with the experi-ment. During the first iterations of the study, we identified experimental incidents and then proposed a laboratory package structure that addressed these incidents, including docu-ment usability improvements. We used the later iterations to validate and generalize the laboratory package structure for use in all software engineering experiments. We aimed to solve a specific problem, while at the same time looking at how to contribute to the body of knowledge on laboratory packages. Results: We generated a laboratory package for three different experiments. These packages eased the replication of the respective experiments. The evaluation that we conducted shows that the laboratory package proposal is acceptable and reduces the effort currently required to replicate experiments in software engineering. Conclusion: We think that the content and structure that we propose for laboratory pack-ages can be useful for other software engineering experiments.

The Next Generation Science Standards (NGSS) put forth an aspiring call for science educators to enact instructional experiences that enable all students to participate in the real-world practices of science. To achieve this, teachers need to create classroom environments that will attract all students to science learning and support and sustain their motivation and engagement. This report describes key insights from a collaboration among motivation experts, science education researchers, and middle school science teachers who used a research-based approach to develop a professional learning program with classroom-based resources to promote and sustain students' motivation and engagement for science learning.

The potential of digital twin technology is yet to be fully realized due to its diversity and untapped potential. Digital twins enable systems' analysis, design, optimization, and evolution to be performed digitally or in conjunction with a cyber-physical approach to improve speed, accuracy, and efficiency over traditional engineering methods. Industry 4.0, factories of the future, and digital twins continue to benefit from the technology and provide enhanced efficiency within existing systems. Due to the lack of information and security standards associated with the transition to cyber digitization, cybercriminals have been able to take advantage of the situation. Access to a digital twin of a product or service is equivalent to threatening the entire collection. There is a robust interaction between digital twins and artificial intelligence tools, which leads to strong interaction between these technologies, so it can be used to improve the cybersecurity of these digital platforms based on their integration with these technologies. This study aims to investigate the role of artificial intelligence in providing cybersecurity for digital twin versions of various industries, as well as the risks associated with these versions. In addition, this research serves as a road map for researchers and others interested in cybersecurity and digital security., Comment: 60 pages, 8 Figures, 15 Tables

[Context] Microservices enable the decomposition of applications into small and independent services connected together. The independence between services could positively affect the development velocity of a project, which is considered an important metric measuring the time taken to implement features and fix bugs. However, no studies have investigated the connection between microservices and development velocity. [Objective and Method] The goal of this study plan is to investigate the effect microservices have on development velocity. The study compares GitHub projects adopting microservices from the beginning and similar projects using monolithic architectures. We designed this study using a cohort study method, to enable obtaining a high level of evidence. [Results] The result of this work enables the confirmation of the effective improvement of the development velocity of microservices. Moreover, this study will contribute to the body of knowledge of empirical methods being among the first works adopting the cohort study methodology.

We present a ceramic material based on hematite (Fe$_2$O$_3$) and zirconia stabilized with yttria at 8% molar (YSZ), that exhibits stable electrical properties with transported charge and that can be tuned to the resistivities necessary to induce spark-quenching in gaseous detectors ($\rho = 10^9-10^{12}$ $\Omega \cdot$cm), from room temperature down to the liquid-vapor coexistence point of nitrogen (77 K). It, thus, allows covering the operating temperatures of most immediate interest to gaseous instrumentation. The ceramics have been produced in a region of mass concentrations far from what has been usually explored in literature: optimal characteristics are achieved for Fe$_2$O$_3$ concentrations of 75%wt (LAr boiling temperature), 35%wt (LXe boiling temperature), and 100%wt (room temperature). The nine order of magnitude enhancement observed for the electrical conductivity of the mixed phases relative to that of pure Fe$_2$O$_3$ is startling, however it can be qualitatively understood based on existing literature. Plates of 4 cm x 4 cm have been manufactured and, prior to this work, operated in-detector at the LXe boiling point (165 K), demonstrating spark-free operation. Preliminary results obtained for the first time on a spark-protected amplification structure (RP-WELL) at around the LAr boiling point (90 K) are now presented, too.

Software engineering techniques are increasingly relying on deep learning approaches to support many software engineering tasks, from bug triaging to code generation. To assess the efficacy of such techniques researchers typically perform controlled experiments. Conducting these experiments, however, is particularly challenging given the complexity of the space of variables involved, from specialized and intricate architectures and algorithms to a large number of training hyper-parameters and choices of evolving datasets, all compounded by how rapidly the machine learning technology is advancing, and the inherent sources of randomness in the training process. In this work we conduct a mapping study, examining 194 experiments with techniques that rely on deep neural networks appearing in 55 papers published in premier software engineering venues to provide a characterization of the state-of-the-practice, pinpointing experiments common trends and pitfalls. Our study reveals that most of the experiments, including those that have received ACM artifact badges, have fundamental limitations that raise doubts about the reliability of their findings. More specifically, we find: weak analyses to determine that there is a true relationship between independent and dependent variables (87% of the experiments); limited control over the space of DNN relevant variables, which can render a relationship between dependent variables and treatments that may not be causal but rather correlational (100% of the experiments); and lack of specificity in terms of what are the DNN variables and their values utilized in the experiments (86% of the experiments) to define the treatments being applied, which makes it unclear whether the techniques designed are the ones being assessed, or how the sources of extraneous variation are controlled. We provide some practical recommendations to address these limitations.

Cryogenic versions of Resistive WELL (RWELL) and Resistive Plate WELL (RPWELL) detectors have been developed, aimed at stable avalanche multiplication of ionization electrons in dual-phase TPCs. In the RWELL, a thin resistive layer deposited on top of an insulator is inserted in between the electron multiplier (THGEM) and the readout anode; in the RPWELL, a resistive plate is directly coupled to the THGEM. Radiation-induced ionization electrons in the liquid are extracted into the gaseous phase. They drift into the THGEM's holes where they undergo charge multiplication. Embedding resistive materials into the multiplier proved to enhance operation stability due to the mitigation of electrical discharges - thus allowing operation at higher charge gain compared to standard THGEM (a.k.a. LEM) multipliers. We present the detector concepts and report on the main preliminary results.

Microservice-based systems are often complex to understand, especially when their sizes grow. Abstracted views help practitioners with the system understanding from a certain perspective. Recent advancement in interactive data visualization begs the question of whether established software engineering models to visualize system design remain the most suited approach for the service-oriented design of microservices. Our recent work proposed presenting a 3D visualization for microservices in augmented reality. This paper analyzes whether such an approach brings any benefits to practitioners when dealing with selected architectural questions related to system design quality. For this purpose, we conducted a controlled experiment involving 20 participants investigating their performance in identifying service dependency, service cardinality, and bottlenecks. Results show that the 3D enables novices to perform as well as experts in the detection of service dependencies, especially in large systems, while no differences are reported for the identification of service cardinality and bottlenecks. We recommend industry and researchers to further investigate AR for microservice architectural analysis, especially to ease the onboarding of new developers in microservice~projects., Comment: The paper (10 pages) is accepted in ICPC 2023

This brief focuses on Colombia which, like most countries globally, closed its schools in March of 2020 due to the COVID-19 pandemic. As throughout most of Latin America, Colombian schools remained closed for over a year, and they only began to gradually reopen in July 2021. The brief explores the pandemic's impact on student learning by analyzing trends in student achievement in national assessments from 2015 to 2019 and comparing them with student achievement in the same national assessments carried out in 2020 and 2021. It also explores the extent to which students in subnational territories (ETCs)--the equivalent to U.S. states, except some are certified by the national government to have more autonomy in spending than others--with different lengths of school closure periods experienced varying levels of learning losses. Findings indicated that the effects of COVID-19 on student learning in Colombia were greatest for female students and students from wealthier backgrounds, a surprising finding given that studies from other countries have found that students from socioeconomically disadvantaged households suffered larger learning losses than their peers from wealthier households. A plausible explanation is that in Colombia students from wealthier backgrounds tend to attend better schools. Findings also showed that learning losses due to COVID-19 were lower in ETCs that provided access to in-person schooling sooner.