Search

Experimentation is an essential method for causal inference in any empirical discipline. Crossover-design experiments are common in Software Engineering (SE) research. In these, subjects apply more than one treatment in different orders. This design increases the amount of obtained data and deals with subject variability but introduces threats to internal validity like the learning and carryover effect. Vegas et al. reviewed the state of practice for crossover designs in SE research and provided guidelines on how to address its threats during data analysis while still harnessing its benefits. In this paper, we reflect on the impact of these guidelines and review the state of analysis of crossover design experiments in SE publications between 2015 and March 2024. To this end, by conducting a forward snowballing of the guidelines, we survey 136 publications reporting 67 crossover-design experiments and evaluate their data analysis against the provided guidelines. The results show that the validity of data analyses has improved compared to the original state of analysis. Still, despite the explicit guidelines, only 29.5% of all threats to validity were addressed properly. While the maturation and the optimal sequence threats are properly addressed in 35.8% and 38.8% of all studies in our sample respectively, the carryover threat is only modeled in about 3% of the observed cases. The lack of adherence to the analysis guidelines threatens the validity of the conclusions drawn from crossover design experiments

Test-driven development (TDD) is a widely used agile practice. However, very little is known with certainty about TDD's underlying foundations, i.e., the way TDD works. In this paper, we propose a theoretical framework for TDD, with the following characteristics: 1) Each TDD cycle represents a vertical slice of a (probably also small) user story, 2) vertical slices are captured using contracts, implicit in the developers' minds, and 3) the code created during a TDD cycle is a sliced-based specification of a code oracle, using the contracts as slicing pre/post-conditions. We have checked the connections among TDD, contracts, and slices using a controlled experiment conducted in the industry.

Experiments are a commonly used method of research in software engineering (SE). Researchers report their experiments following detailed guidelines. However, researchers do not, in the field of test-driven development (TDD) at least, specify how they operationalized the response variables and the measurement process. This article has three aims: (i) identify the response variable operationalization components in TDD experiments that study external quality; (ii) study their influence on the experimental results;(ii) determine if the experiment reports describe the measurement process components that have an impact on the results. Sequential mixed method. The first part of the research adopts a quantitative approach applying a statistical an\'alisis (SA) of the impact of the operationalization components on the experimental results. The second part follows on with a qualitative approach applying a systematic mapping study (SMS). The test suites, intervention types and measurers have an influence on the measurements and results of the SA of TDD experiments in SE. The test suites have a major impact on both the measurements and the results of the experiments. The intervention type has less impact on the results than on the measurements. While the measurers have an impact on the measurements, this is not transferred to the experimental results. On the other hand, the results of our SMS confirm that TDD experiments do not usually report either the test suites, the test case generation method, or the details of how external quality was measured. A measurement protocol should be used to assure that the measurements made by different measurers are similar. It is necessary to report the test cases, the experimental task and the intervention type in order to be able to reproduce the measurements and SA, as well as to replicate experiments and build dependable families of experiments.

A recurring problem in software development is incorrect decision making on the techniques, methods and tools to be used. Mostly, these decisions are based on developers' perceptions about them. A factor influencing people's perceptions is past experience, but it is not the only one. In this research, we aim to discover how well the perceptions of the defect detection effectiveness of different techniques match their real effectiveness in the absence of prior experience. To do this, we conduct an empirical study plus a replication. During the original study, we conduct a controlled experiment with students applying two testing techniques and a code review technique. At the end of the experiment, they take a survey to find out which technique they perceive to be most effective. The results show that participants' perceptions are wrong and that this mismatch is costly in terms of quality. In order to gain further insight into the results, we replicate the controlled experiment and extend the survey to include questions about participants' opinions on the techniques and programs. The results of the replicated study confirm the findings of the original study and suggest that participants' perceptions might be based not on their opinions about complexity or preferences for techniques but on how well they think that they have applied the techniques.

Context: Experiment replications play a central role in the scientific method. Although software engineering experimentation has matured a great deal, the number of experiment replications is still relatively small. Software engineering experiments are composed of complex concepts, procedures and artefacts. Laboratory packages are a means of transfer-ring knowledge among researchers to facilitate experiment replications. Objective: This paper investigates the experiment replication process to find out what information is needed to successfully replicate an experiment. Our objective is to propose the content and structure of laboratory packages for software engineering experiments. Method: We evaluated seven replications of three different families of experiments. Each replication had a different experimenter who was, at the time, unfamiliar with the experi-ment. During the first iterations of the study, we identified experimental incidents and then proposed a laboratory package structure that addressed these incidents, including docu-ment usability improvements. We used the later iterations to validate and generalize the laboratory package structure for use in all software engineering experiments. We aimed to solve a specific problem, while at the same time looking at how to contribute to the body of knowledge on laboratory packages. Results: We generated a laboratory package for three different experiments. These packages eased the replication of the respective experiments. The evaluation that we conducted shows that the laboratory package proposal is acceptable and reduces the effort currently required to replicate experiments in software engineering. Conclusion: We think that the content and structure that we propose for laboratory pack-ages can be useful for other software engineering experiments.

The Next Generation Science Standards (NGSS) put forth an aspiring call for science educators to enact instructional experiences that enable all students to participate in the real-world practices of science. To achieve this, teachers need to create classroom environments that will attract all students to science learning and support and sustain their motivation and engagement. This report describes key insights from a collaboration among motivation experts, science education researchers, and middle school science teachers who used a research-based approach to develop a professional learning program with classroom-based resources to promote and sustain students' motivation and engagement for science learning.

The potential of digital twin technology is yet to be fully realized due to its diversity and untapped potential. Digital twins enable systems' analysis, design, optimization, and evolution to be performed digitally or in conjunction with a cyber-physical approach to improve speed, accuracy, and efficiency over traditional engineering methods. Industry 4.0, factories of the future, and digital twins continue to benefit from the technology and provide enhanced efficiency within existing systems. Due to the lack of information and security standards associated with the transition to cyber digitization, cybercriminals have been able to take advantage of the situation. Access to a digital twin of a product or service is equivalent to threatening the entire collection. There is a robust interaction between digital twins and artificial intelligence tools, which leads to strong interaction between these technologies, so it can be used to improve the cybersecurity of these digital platforms based on their integration with these technologies. This study aims to investigate the role of artificial intelligence in providing cybersecurity for digital twin versions of various industries, as well as the risks associated with these versions. In addition, this research serves as a road map for researchers and others interested in cybersecurity and digital security., Comment: 60 pages, 8 Figures, 15 Tables

[Context] Microservices enable the decomposition of applications into small and independent services connected together. The independence between services could positively affect the development velocity of a project, which is considered an important metric measuring the time taken to implement features and fix bugs. However, no studies have investigated the connection between microservices and development velocity. [Objective and Method] The goal of this study plan is to investigate the effect microservices have on development velocity. The study compares GitHub projects adopting microservices from the beginning and similar projects using monolithic architectures. We designed this study using a cohort study method, to enable obtaining a high level of evidence. [Results] The result of this work enables the confirmation of the effective improvement of the development velocity of microservices. Moreover, this study will contribute to the body of knowledge of empirical methods being among the first works adopting the cohort study methodology.

We present a ceramic material based on hematite (Fe$_2$O$_3$) and zirconia stabilized with yttria at 8% molar (YSZ), that exhibits stable electrical properties with transported charge and that can be tuned to the resistivities necessary to induce spark-quenching in gaseous detectors ($\rho = 10^9-10^{12}$ $\Omega \cdot$cm), from room temperature down to the liquid-vapor coexistence point of nitrogen (77 K). It, thus, allows covering the operating temperatures of most immediate interest to gaseous instrumentation. The ceramics have been produced in a region of mass concentrations far from what has been usually explored in literature: optimal characteristics are achieved for Fe$_2$O$_3$ concentrations of 75%wt (LAr boiling temperature), 35%wt (LXe boiling temperature), and 100%wt (room temperature). The nine order of magnitude enhancement observed for the electrical conductivity of the mixed phases relative to that of pure Fe$_2$O$_3$ is startling, however it can be qualitatively understood based on existing literature. Plates of 4 cm x 4 cm have been manufactured and, prior to this work, operated in-detector at the LXe boiling point (165 K), demonstrating spark-free operation. Preliminary results obtained for the first time on a spark-protected amplification structure (RP-WELL) at around the LAr boiling point (90 K) are now presented, too.

Software engineering techniques are increasingly relying on deep learning approaches to support many software engineering tasks, from bug triaging to code generation. To assess the efficacy of such techniques researchers typically perform controlled experiments. Conducting these experiments, however, is particularly challenging given the complexity of the space of variables involved, from specialized and intricate architectures and algorithms to a large number of training hyper-parameters and choices of evolving datasets, all compounded by how rapidly the machine learning technology is advancing, and the inherent sources of randomness in the training process. In this work we conduct a mapping study, examining 194 experiments with techniques that rely on deep neural networks appearing in 55 papers published in premier software engineering venues to provide a characterization of the state-of-the-practice, pinpointing experiments common trends and pitfalls. Our study reveals that most of the experiments, including those that have received ACM artifact badges, have fundamental limitations that raise doubts about the reliability of their findings. More specifically, we find: weak analyses to determine that there is a true relationship between independent and dependent variables (87% of the experiments); limited control over the space of DNN relevant variables, which can render a relationship between dependent variables and treatments that may not be causal but rather correlational (100% of the experiments); and lack of specificity in terms of what are the DNN variables and their values utilized in the experiments (86% of the experiments) to define the treatments being applied, which makes it unclear whether the techniques designed are the ones being assessed, or how the sources of extraneous variation are controlled. We provide some practical recommendations to address these limitations.

Cryogenic versions of Resistive WELL (RWELL) and Resistive Plate WELL (RPWELL) detectors have been developed, aimed at stable avalanche multiplication of ionization electrons in dual-phase TPCs. In the RWELL, a thin resistive layer deposited on top of an insulator is inserted in between the electron multiplier (THGEM) and the readout anode; in the RPWELL, a resistive plate is directly coupled to the THGEM. Radiation-induced ionization electrons in the liquid are extracted into the gaseous phase. They drift into the THGEM's holes where they undergo charge multiplication. Embedding resistive materials into the multiplier proved to enhance operation stability due to the mitigation of electrical discharges - thus allowing operation at higher charge gain compared to standard THGEM (a.k.a. LEM) multipliers. We present the detector concepts and report on the main preliminary results.

This brief focuses on Colombia which, like most countries globally, closed its schools in March of 2020 due to the COVID-19 pandemic. As throughout most of Latin America, Colombian schools remained closed for over a year, and they only began to gradually reopen in July 2021. The brief explores the pandemic's impact on student learning by analyzing trends in student achievement in national assessments from 2015 to 2019 and comparing them with student achievement in the same national assessments carried out in 2020 and 2021. It also explores the extent to which students in subnational territories (ETCs)--the equivalent to U.S. states, except some are certified by the national government to have more autonomy in spending than others--with different lengths of school closure periods experienced varying levels of learning losses. Findings indicated that the effects of COVID-19 on student learning in Colombia were greatest for female students and students from wealthier backgrounds, a surprising finding given that studies from other countries have found that students from socioeconomically disadvantaged households suffered larger learning losses than their peers from wealthier households. A plausible explanation is that in Colombia students from wealthier backgrounds tend to attend better schools. Findings also showed that learning losses due to COVID-19 were lower in ETCs that provided access to in-person schooling sooner.

Microservice-based systems are often complex to understand, especially when their sizes grow. Abstracted views help practitioners with the system understanding from a certain perspective. Recent advancement in interactive data visualization begs the question of whether established software engineering models to visualize system design remain the most suited approach for the service-oriented design of microservices. Our recent work proposed presenting a 3D visualization for microservices in augmented reality. This paper analyzes whether such an approach brings any benefits to practitioners when dealing with selected architectural questions related to system design quality. For this purpose, we conducted a controlled experiment involving 20 participants investigating their performance in identifying service dependency, service cardinality, and bottlenecks. Results show that the 3D enables novices to perform as well as experts in the detection of service dependencies, especially in large systems, while no differences are reported for the identification of service cardinality and bottlenecks. We recommend industry and researchers to further investigate AR for microservice architectural analysis, especially to ease the onboarding of new developers in microservice~projects., Comment: The paper (10 pages) is accepted in ICPC 2023

Semantic segmentation is one of the most challenging tasks in computer vision. However, in many applications, a frequent obstacle is the lack of labeled images, due to the high cost of pixel-level labeling. In this scenario, it makes sense to approach the problem from a semi-supervised point of view, where both labeled and unlabeled images are exploited. In recent years this line of research has gained much interest and many approaches have been published in this direction. Therefore, the main objective of this study is to provide an overview of the current state of the art in semi-supervised semantic segmentation, offering an updated taxonomy of all existing methods to date. This is complemented by an experimentation with a variety of models representing all the categories of the taxonomy on the most widely used becnhmark datasets in the literature, and a final discussion on the results obtained, the challenges and the most promising lines of future research.

Computer science (CS) education helps students acquire skills such as computational thinking, problem-solving, and collaboration, among others. As these skills take preeminence in the rapidly changing 21st century, CS education promises to significantly enhance student preparedness for the future of work and active citizenship. Based on prior analysis and expert consultation, the authors selected 11 country, state, and provincial CS-education case studies with lessons that can apply broadly to other education systems. These cases come from diverse global regions and circumstances and have implemented CS education programs for various periods and to different levels of success. As such, the authors have examined information to extract lessons that can lead to successful implementation. This study will focus on the development of CS education in Arkansas. With regular funding, bipartisan political support, and strong executive leadership, Arkansas has made considerable progress at improving CS education since the beginning of the Hutchinson gubernatorial administration in 2015. To help guide key policy decisions, Governor Hutchinson appointed task force groups that included representatives from teacher associations, businesses, and government agencies. These stakeholders gave policymakers influential advice on teacher recruitment and training, student engagement activities, and curriculum standards.

Computer science (CS) education helps students acquire skills such as computational thinking, problem-solving, and collaboration. Since these skills take preeminence in the rapidly changing 21st century, CS education promises to significantly enhance student preparedness for the future of work and active citizenship. CS education can also reduce skills inequality if education systems make a concerted effort to ensure that all students have equitable access to curricula that provide them with the needed breadth of skills--regardless of their gender, ethnicity, or socioeconomic status. Based on prior analyses and expert consultations, the authors selected 11 CS-education country, state, and provincial case studies with lessons that can broadly apply to other education systems. This particular study examines how Chile is training a future workforce of creative problem-solvers to maintain its status as one of the higher-income countries in Latin America. To accomplish this goal, the Ministry of Education designed the National Plan for Digital Languages to prepare students to compete in an increasingly digital global economy. Now in the early stages of implementing this plan, the government seeks to increase the number of students learning about CS and computational thinking in the coming years.

The purpose of this study is to identify gaps and challenges in the use of education technology (ed tech) in Chennai, Tamil Nadu during COVID-19. Specifically, the authors investigated how use of ed tech differed by type of school (government or private), household socioeconomic status, and student gender--and how it changed during the COVID-19 school closures. Ultimately, the authors wanted to know how the use of ed tech may exacerbate or mitigate the unequal impact of school closures on student learning. The survey findings indicate that during the pandemic-related school closures, students in private schools and those from high-socioeconomic status households have more access to digital devices and are more engaged in regular educational activities during COVID-19 than their peers in government schools and from low-socioeconomic status households; findings also indicate that girls are more likely than boys to have access to digital devices for learning and to engage in more regular educational activities. Unsurprisingly, parents turned out to be a major source of educational activities of young children during the school closures. Altogether, these preliminary results shed light onto a likely growing inequality of educational opportunity in India and around the world, suggesting the need for policymakers to broaden access to continuous and equitable learning opportunities across the student population.

Using the data of the Pierre Auger Observatory, we report on a search for signatures that would be suggestive of super-heavy particles decaying in the Galactic halo. From the lack of signal, we present upper limits for different energy thresholds above ${\gtrsim}10^8$\,GeV on the secondary by-product fluxes expected from the decay of the particles. Assuming that the energy density of these super-heavy particles matches that of dark matter observed today, we translate the upper bounds on the particle fluxes into tight constraints on the couplings governing the decay process as a function of the particle mass. Instantons, which are non-perturbative solutions to Yang-Mills equations, can give rise to decay channels otherwise forbidden and transform stable particles into meta-stable ones. Assuming such instanton-induced decay processes, we derive a bound on the reduced coupling constant of gauge interactions in the dark sector: $\alpha_X \lesssim 0.09$, for $10^{9} \lesssim M_X/\text{GeV} < 10^{19}$. Conversely, we obtain that, for instance, a reduced coupling constant $\alpha_X = 0.09$ excludes masses $M_X \gtrsim 3\times 10^{13}~$GeV. In the context of dark matter production from gravitational interactions alone during the reheating epoch, we derive constraints on the parameter space that involves, in addition to $M_X$ and $\alpha_X$, the Hubble rate at the end of inflation, the reheating efficiency, and the non-minimal coupling of the Higgs with curvature., Comment: 15 pages, 8 figures, Accompanying paper of arXiv:2203.08854, accepted for publication in Phys. Rev. D

Computer science (CS) education helps students acquire skills such as computational thinking, problem-solving, and collaboration. Since these skills take preeminence in the rapidly changing 21st century, CS education promises to significantly enhance student preparedness for the future of work and active citizenship. Based on prior analyses and expert consultations, the authors selected 11-CS education country, state, and provincial case studies that may have lessons that can broadly apply to other education systems. These cases come from diverse global regions and circumstances and have implemented CS education programs for various periods and to different levels of success. As such, they have examined information to extract lessons that can lead to successful implementation. This study examines how New Brunswick seeks to improve and expand its CS education activities to train a future workforce that can thrive during economic transition and support the Canadian province's budding technology sector. The Department of Education and Early Childhood Development (DEECD) and various stakeholder organizations aim to give all students the opportunity to learn CS and apply their lessons in a creative and collaborative environment. In the early 2000s, the DEECD decided to refocus its curriculum on STEM (science, technology, engineering, and mathematics) subjects, including ICT (information and communications technology). This brought the necessary infrastructure and knowledge of digital technologies into schools that would later set the stage for mandatory CS courses in 2017. The DEECD also faced the challenge of rolling out CS education for students of two distinct language groups. Primary school teachers in the anglophone sector were encouraged to incorporate CS and computational thinking as interdisciplinary subjects, while the francophone sector had no requirement to offer either subject in primary schools. All lower-secondary school students, whether English- or French-speaking, take CS courses that emphasize programming skills. Further, both language systems offer more advanced CS courses in upper secondary school as electives.

Computer Science (CS) education helps students acquire skills such as computational thinking, problem-solving, and collaboration. Since these skills take preeminence in the rapidly changing 21st century, CS education promises to significantly enhance student preparedness for the future of work and active citizenship. Based on prior analysis and expert consultation, the authors selected 11 country, state, and provincial CS education case studies that may apply broadly to other education systems. These cases have come from diverse global regions and circumstances and have implemented CS education programs for various periods and with different levels of success. As such, they have examined information to extract lessons that can lead to successful implementation in other parts of the world. This study will focus on how the Canadian province of British Columbia (B.C.) developed its computer science program. B.C. is home to a growing technology industry and is one of the leading Canadian provinces for K-12 CS and computational thinking education (Gannon & Buteau, 2018). Its education system creates opportunities for students to pursue CS education based on their interests, including theoretical and quantitative concepts, practical applications, and problem-solving. Further, the province's approach to teacher professional development and student engagement merits consideration.