Search

For future regulatory compliance, organizations must assess and report on the state of sustainability in terms of its impacts over time. Sustainability, being a multidimensional concern, is complex to quantify. This complexity further increases with the interdependencies of the quality concerns across different sustainability dimensions. The research literature lacks a holistic way to evaluate sustainability at the software architecture level. With this study, our aim is to identify quality attribute (QA) trade-offs at the software architecture level and quantify the related sustainability impact. To this aim we present an improved version of the Sustainability Impact Score (SIS), building on our previous work. The SIS facilitates the identification and quantification of trade-offs in terms of their sustainability impact, leveraging a risk- and importance-based prioritization mechanism. To evaluate our approach, we apply it to an industrial case study involving a multi-model framework for integrated decision-making in the energy sector. Our study reveals that technical quality concerns have significant, often unrecognized impacts across sustainability dimensions. The SIS coupled with QA trade-offs can help practitioners make informed decisions that align with their sustainability goals. Early evaluations can help organizations mitigate sustainability risks by taking preventive actions.

The ability to distribute heralded entanglement between distant matter nodes is a primitive for the implementation of large-scale quantum networks. Some of the most crucial requirements for future applications include high heralding rates at telecom wavelengths, multiplexed operation and on-demand retrieval of stored excitations for synchronization of separate quantum links. Despite tremendous progress in various physical systems, the demonstration of telecom-heralded entanglement between quantum nodes featuring both multiplexed operation and on-demand retrieval remains elusive. In this work, we combine narrowband parametric photon-pair sources and solid-state quantum memories based on rare-earth doped crystals to demonstrate telecom heralded entanglement between spatially separated spin-wave quantum memories with fully adjustable recall time and temporal multiplexing of 15 modes. In a first experiment, the storage in the spin-state is conditioned on the entanglement heralding. We take advantage of the control over readout pulse phase to achieve feed-forward conditional phase-shifts on the stored photons depending on which heralding detector clicked. We exploit this effect to double the entanglement heralding rate for a given quantum state up to \SI{500}{\cps}, with a measured positive concurrence by up to 6 standard deviations. In a second experiment, we simulate the communication time of a long-distance link by implementing an unconditional storage scheme with a dead-time of \SI{100}{\micro\second}, leading to an entanglement rate of \SI{22}{\cps} per heralding detector, increased by a factor of 15 thanks to temporal multiplexing. These results establish our architecture as a prime candidate for the implementation of scalable high-rate quantum network links., Comment: 14 pages, 10 figures

We propose a methodology that combines several advanced techniques in Large Language Model (LLM) retrieval to support the development of robust, multi-source question-answer systems. This methodology is designed to integrate information from diverse data sources, including unstructured documents (PDFs) and structured databases, through a coordinated multi-agent orchestration and dynamic retrieval approach. Our methodology leverages specialized agents-such as SQL agents, Retrieval-Augmented Generation (RAG) agents, and router agents - that dynamically select the most appropriate retrieval strategy based on the nature of each query. To further improve accuracy and contextual relevance, we employ dynamic prompt engineering, which adapts in real time to query-specific contexts. The methodology's effectiveness is demonstrated within the domain of Contract Management, where complex queries often require seamless interaction between unstructured and structured data. Our results indicate that this approach enhances response accuracy and relevance, offering a versatile and scalable framework for developing question-answer systems that can operate across various domains and data sources., Comment: International Conference on NLP, AI, Computer Science & Engineering (NLAICSE 2024)

We present a question-and-answer (Q\&A) application designed to support the contract management process by leveraging combined information from contract documents (PDFs) and data retrieved from contract management systems (database). This data is processed by a large language model (LLM) to provide precise and relevant answers. The accuracy of these responses is further enhanced through the use of Retrieval-Augmented Generation (RAG), text-to-SQL techniques, and agents that dynamically orchestrate the workflow. These techniques eliminate the need to retrain the language model. Additionally, we employed Prompt Engineering to fine-tune the focus of responses. Our findings demonstrate that this multi-agent orchestration and combination of techniques significantly improve the relevance and accuracy of the answers, offering a promising direction for future information systems., Comment: KDIR 2024 - Knowledge Discovery and Information Retrieval

Background. The rapid growth of Language Models (LMs), particularly in code generation, requires substantial computational resources, raising concerns about energy consumption and environmental impact. Optimizing LMs inference for energy efficiency is crucial, and Small Language Models (SLMs) offer a promising solution to reduce resource demands. Aim. Our goal is to analyze the impact of deep learning runtime engines and execution providers on energy consumption, execution time, and computing-resource utilization from the point of view of software engineers conducting inference in the context of code SLMs. Method. We conducted a technology-oriented, multi-stage experimental pipeline using twelve code generation SLMs to investigate energy consumption, execution time, and computing-resource utilization across the configurations. Results. Significant differences emerged across configurations. CUDA execution provider configurations outperformed CPU execution provider configurations in both energy consumption and execution time. Among the configurations, TORCH paired with CUDA demonstrated the greatest energy efficiency, achieving energy savings from 37.99% up to 89.16% compared to other serving configurations. Similarly, optimized runtime engines like ONNX with the CPU execution provider achieved from 8.98% up to 72.04% energy savings within CPU-based configurations. Also, TORCH paired with CUDA exhibited efficient computing-resource utilization. Conclusions. Serving configuration choice significantly impacts energy efficiency. While further research is needed, we recommend the above configurations best suited to software engineers' requirements for enhancing serving efficiency in energy and performance., Comment: 26 pages, submitted to journal

Accurate and timely regional weather prediction is vital for sectors dependent on weather-related decisions. Traditional prediction methods, based on atmospheric equations, often struggle with coarse temporal resolutions and inaccuracies. This paper presents a novel machine learning (ML) model, called MiMa (short for Micro-Macro), that integrates both near-surface observational data from Kentucky Mesonet stations (collected every five minutes, known as Micro data) and hourly atmospheric numerical outputs (termed as Macro data) for fine-resolution weather forecasting. The MiMa model employs an encoder-decoder transformer structure, with two encoders for processing multivariate data from both datasets and a decoder for forecasting weather variables over short time horizons. Each instance of the MiMa model, called a modelet, predicts the values of a specific weather parameter at an individual Mesonet station. The approach is extended with Re-MiMa modelets, which are designed to predict weather variables at ungauged locations by training on multivariate data from a few representative stations in a region, tagged with their elevations. Re-MiMa (short for Regional-MiMa) can provide highly accurate predictions across an entire region, even in areas without observational stations. Experimental results show that MiMa significantly outperforms current models, with Re-MiMa offering precise short-term forecasts for ungauged locations, marking a significant advancement in weather forecasting accuracy and applicability.

Since their appearance in the 1950s, computational models capable of performing probabilistic choices have received wide attention and are nowadays pervasive in almost every areas of computer science. Their development was also inextricably linked with inquiries about computation power and resource issues. Although most crucial notions in the field are well-known, the related terminology is sometimes imprecise or misleading. The present work aims to clarify the core features and main differences between machines and classes developed in relation to randomized computation. To do so, we compare the modern definitions with original ones, recalling the context in which they first appeared, and investigate the relations linking probabilistic and counting models.

Multiparty Session Types (MPSTs) offer a structured way of specifying communication protocols and guarantee relevant communication properties, such as deadlock-freedom. In this paper, we extend a minimal MPST system with quantum data and operations, enabling the specification of quantum protocols. Quantum MPSTs (QMPSTs) provide a formal notation to describe quantum protocols, both at the abstract level of global types, describing which communications can take place in the system and their dependencies, and at the concrete level of local types and quantum processes, describing the expected behavior of each participant in the protocol. Type-checking relates these two levels formally, ensuring that processes behave as prescribed by the global type. Beyond usual communication properties, QMPSTs also allow us to prove that qubits are owned by a single process at any time, capturing the quantum no-cloning and no-deleting theorems. We use our approach to verify four quantum protocols from the literature, respectively Teleportation, Secret Sharing, Bit-Commitment, and Key Distribution., Comment: To appear at SEFM 2024

This is an index of the contributions by the Giant Radio Array for Neutrino Detection (GRAND) Collaboration to the 10th International Workshop on Acoustic and Radio EeV Neutrino Detection Activities (ARENA 2024, University of Chicago, June 11-14, 2024). The contributions include an overview of GRAND in its present and future incarnations, methods of radio-detection that are being developed for them, and ongoing joint work between the GRAND and BEACON experiments., Comment: Note: To access the list of contributions, please follow the "HTML" link that can be found on the arXiv page

A $\lambda$-calculus is introduced in which all programs can be evaluated in probabilistic polynomial time and in which there is sufficient structure to represent sequential cryptographic constructions and adversaries for them, even when the latter are oracle-based. A notion of observational equivalence capturing computational indistinguishability and a class of approximate logical relations are then presented, showing that the latter represent a sound proof technique for the former. The work concludes with the presentation of an example of a security proof in which the encryption scheme induced by a pseudorandom function is proven secure against active adversaries in a purely equational style., Comment: APLAS 2024 conference

This paper presents the global analysis of two extended decreases of the galactic cosmic ray intensity observed by world-wide networks of ground-based detectors in 2012. This analysis is capable of separately deriving the cosmic ray density (or omnidirectional intensity) and anisotropy each as a function of time and rigidity. A simple diffusion model along the spiral field line between Earth and a cosmic-ray barrier indicates the long duration of these events resulting from about 190$^\circ$ eastern extension of a barrier such as an IP-shock followed by the sheath region and/or the corotating interaction region (CIR). It is suggested that the coronal mass ejection merging and compressing the preexisting CIR at its flank can produce such the extended barrier. The derived rigidity spectra of the density and anisotropy both vary in time during each event period. In particular we find that the temporal feature of the ``phantom Forbush decrease'' reported in an analyzed period is dependent on rigidity, looking quite different at different rigidities. From these rigidity spectra of the density and anisotropy, we derive the rigidity spectrum of the average parallel mean-free-path of pitch angle scattering along the spiral field line and infer the power spectrum of the magnetic fluctuation and its temporal variation. Possible physical cause of the strong rigidity dependence of the ``phantom Forbush decrease'' is also discussed. These results demonstrate the high-energy cosmic rays observed at Earth responding to remote space weather., Comment: Accepted for publication in ApJ

The operation of upcoming ultra-high-energy cosmic-ray, gamma-ray, and neutrino radio-detection experiments, like the Giant Radio Array for Neutrino Detection (GRAND), poses significant computational challenges involving the production of numerous simulations of particle showers and their detection, and a high data throughput. GRANDlib is an open-source software tool designed to meet these challenges. Its primary goal is to perform end-to-end simulations of the detector operation, from the interaction of ultra-high-energy particles, through -- by interfacing with external air-shower simulations -- the ensuing particle shower development and its radio emission, to its detection by antenna arrays and its processing by data-acquisition systems. Additionally, GRANDlib manages the visualization, storage, and retrieval of experimental and simulated data. We present an overview of GRANDlib to serve as the basis of future GRAND analyses., Comment: 11 pages, 9 figures, plus appendices; Matches published version

Results are presented for the measurement of large-scale anisotropies in the arrival directions of ultra-high-energy cosmic rays detected at the Pierre Auger Observatory during 19 years of operation, prior to AugerPrime, the upgrade of the Observatory. The 3D dipole amplitude and direction are reconstructed above $4\,$EeV in four energy bins. Besides the established dipolar anisotropy in right ascension above $8\,$EeV, the Fourier amplitude of the $8$ to $16\,$EeV energy bin is now also above the $5\sigma$ discovery level. No time variation of the dipole moment above $8\,$EeV is found, setting an upper limit to the rate of change of such variations of $0.3\%$ per year at the $95\%$ confidence level. Additionally, the results for the angular power spectrum are shown, demonstrating no other statistically significant multipoles. The results for the equatorial dipole component down to $0.03\,$EeV are presented, using for the first time a data set obtained with a trigger that has been optimized for lower energies. Finally, model predictions are discussed and compared with observations, based on two source emission scenarios obtained in the combined fit of spectrum and composition above $0.6\,$EeV., Comment: Minor typo in table 4 corrected

We introduce a type system for the Quipper language designed to derive upper bounds on the size of the circuits produced by the typed program. This size can be measured according to various metrics, including width, depth and gate count, but also variations thereof obtained by considering only some wire types or some gate kinds. The key ingredients for achieving this level of flexibility are effects and refinement types, both relying on indices, that is, generic arithmetic expressions whose operators are interpreted differently depending on the target metric. The approach is shown to be correct through logical predicates, under reasonable assumptions about the chosen resource metric. This approach is empirically evaluated through the QuRA tool, showing that, in many cases, inferring tight bounds is possible in a fully automatic way., Comment: 26 pages, 13 figures

Development of artificial intelligence (AI) techniques in medical imaging requires access to large-scale and diverse datasets for training and evaluation. In dermatology, obtaining such datasets remains challenging due to significant variations in patient populations, illumination conditions, and acquisition system characteristics. In this work, we propose S-SYNTH, the first knowledge-based, adaptable open-source skin simulation framework to rapidly generate synthetic skin, 3D models and digitally rendered images, using an anatomically inspired multi-layer, multi-component skin and growing lesion model. The skin model allows for controlled variation in skin appearance, such as skin color, presence of hair, lesion shape, and blood fraction among other parameters. We use this framework to study the effect of possible variations on the development and evaluation of AI models for skin lesion segmentation, and show that results obtained using synthetic data follow similar comparative trends as real dermatologic images, while mitigating biases and limitations from existing datasets including small dataset size, lack of diversity, and underrepresentation., Comment: Accepted to the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2024

In this work, we introduce Gemma 2, a new addition to the Gemma family of lightweight, state-of-the-art open models, ranging in scale from 2 billion to 27 billion parameters. In this new version, we apply several known technical modifications to the Transformer architecture, such as interleaving local-global attentions (Beltagy et al., 2020a) and group-query attention (Ainslie et al., 2023). We also train the 2B and 9B models with knowledge distillation (Hinton et al., 2015) instead of next token prediction. The resulting models deliver the best performance for their size, and even offer competitive alternatives to models that are 2-3 times bigger. We release all our models to the community.

Wearable devices are increasingly used as tools for biomedical research, as the continuous stream of behavioral and physiological data they collect can provide insights about our health in everyday contexts. Long-term tracking, defined in the timescale of months of year, can provide insights of patterns and changes as indicators of health changes. These insights can make medicine and healthcare more predictive, preventive, personalized, and participative (The 4P's). However, the challenges in modeling, understanding and processing longitudinal data are a significant barrier to their adoption in research studies and clinical settings. In this paper, we review and discuss three models used to make sense of longitudinal data: routines, rhythms and stability metrics. We present the challenges associated with the processing and analysis of longitudinal wearable sensor data, with a special focus on how to handle the different temporal dynamics at various granularities. We then discuss current limitations and identify directions for future work. This review is essential to the advancement of computational modeling and analysis of longitudinal sensor data for pervasive healthcare.

Methyl cyanide is one of the simplest interstellar complex organic molecules, widely detected in young solar analogues, shocked regions, protoplanetary disks and comets. CH3CN can be considered a key species to explore the chemical connections between planet forming disks and comets. For such comparison to be meaningful kinetics data for the reactions leading to CH3CN formation and destruction must be updated. We focus on the destruction of methyl cyanide through collisions with He+. A combined experimental and theoretical methodology is employed to obtain cross sections (CSs) and branching ratios (BRs) as a function of collision energy, from which reaction rate coefficients $k(T)$ are calculated in the temperature range from 10 to 300 K. CSs and BRs are measured using a guided ion beam set-up. A theoretical treatment based on an analytical formulation of the potential energy surfaces (PESs) for the charge exchange process is developed. The method employs a Landau Zener model to obtain reaction probabilities at crossings between the entrance and exit PESs, and an adiabatic centrifugal sudden approximation to calculate CSs and k(T). Rates and BRs differ from those predicted from widely-used capture models. In particular, the rate coefficient at 10 K is estimated to be almost one order of magnitude smaller than what reported in the KIDA database. As for BRs, the charge exchange is completely dissociative and the most abundant fragments are HCCN+/CCNH+, HCNH+ and CH2+. Our results, combined with a revised chemical network for formation of CH3CN, support the hypothesis that methyl cyanide in protoplanetary disks could be mostly the product of gas-phase processes rather than grain chemistry, as currently proposed. These findings are expected to have implications in the comparison of the abundance ratios of N-bearing molecules observed in disks with cometary abundance ratios, Comment: 19 pages, 7 figures, full article

Ultra-high-energy cosmic rays are known to be mainly of extragalactic origin, and their propagation is limited by energy losses, so their arrival directions are expected to correlate with the large-scale structure of the local Universe. In this work, we investigate the possible presence of intermediate-scale excesses in the flux of the most energetic cosmic rays from the direction of the supergalactic plane region using events with energies above 20 EeV recorded with the surface detector array of the Pierre Auger Observatory up to 31 December 2022, with a total exposure of 135,000 km^2 sr yr. The strongest indication for an excess that we find, with a post-trial significance of 3.1{\sigma}, is in the Centaurus region, as in our previous reports, and it extends down to lower energies than previously studied. We do not find any strong hints of excesses from any other region of the supergalactic plane at the same angular scale. In particular, our results do not confirm the reports by the Telescope Array collaboration of excesses from two regions in the Northern Hemisphere at the edge of the field of view of the Pierre Auger Observatory. With a comparable exposure, our results in those regions are in good agreement with the expectations from an isotropic distribution., Comment: submitted to ApJ

In this chapter we share our experience in running ten editions of the Green Lab course at the Vrije Universiteit Amsterdam, the Netherlands. The course is given in the Software Engineering and Green IT track of the Computer Science Master program of the VU. The course takes place every year over a 2-month period and teaches Computer Science students the fundamentals of Empirical Software Engineering in the context of energy-efficient software. The peculiarity of the course is its research orientation: at the beginning of the course the instructor presents a catalog of scientifically relevant goals, and each team of students signs up for one of them and works together for 2 months on their own experiment for achieving the goal. Each team goes over the classic steps of an empirical study, starting from a precise formulation of the goal and research questions to context definition, selection of experimental subjects and objects, definition of experimental variables, experiment execution, data analysis, and reporting. Over the years, the course became well-known within the Software Engineering community since it led to several scientific studies that have been published at various scientific conferences and journals. Also, students execute their experiments using \textit{open-source tools}, which are developed and maintained by researchers and other students within the program, thus creating a virtuous community of learners where students exchange ideas, help each other, and learn how to collaboratively contribute to open-source projects in a safe environment.

Background: Machine learning (ML) model composition is a popular technique to mitigate shortcomings of a single ML model and to design more effective ML-enabled systems. While ensemble learning, i.e., forwarding the same request to several models and fusing their predictions, has been studied extensively for accuracy, we have insufficient knowledge about how to design energy-efficient ensembles. Objective: We therefore analyzed three types of design decisions for ensemble learning regarding a potential trade-off between accuracy and energy consumption: a) ensemble size, i.e., the number of models in the ensemble, b) fusion methods (majority voting vs. a meta-model), and c) partitioning methods (whole-dataset vs. subset-based training). Methods: By combining four popular ML algorithms for classification in different ensembles, we conducted a full factorial experiment with 11 ensembles x 4 datasets x 2 fusion methods x 2 partitioning methods (176 combinations). For each combination, we measured accuracy (F1-score) and energy consumption in J (for both training and inference). Results: While a larger ensemble size significantly increased energy consumption (size 2 ensembles consumed 37.49% less energy than size 3 ensembles, which in turn consumed 26.96% less energy than the size 4 ensembles), it did not significantly increase accuracy. Furthermore, majority voting outperformed meta-model fusion both in terms of accuracy (Cohen's d of 0.38) and energy consumption (Cohen's d of 0.92). Lastly, subset-based training led to significantly lower energy consumption (Cohen's d of 0.91), while training on the whole dataset did not increase accuracy significantly. Conclusions: From a Green AI perspective, we recommend designing ensembles of small size (2 or maximum 3 models), using subset-based training, majority voting, and energy-efficient ML algorithms like decision trees, Naive Bayes, or KNN., Comment: Currently under review at a journal

Context. Despite the increasing adoption of Machine Learning Operations (MLOps), teams still encounter challenges in effectively applying this paradigm to their specific projects. While there is a large variety of available tools usable for MLOps, there is simultaneously a lack of consolidated architecture knowledge that can inform the architecture design. Objective. Our primary objective is to provide a comprehensive overview of (i) how MLOps architectures are defined across the literature and (ii) which tools are mentioned to support the implementation of each architecture component. Method. We apply the Systematic Mapping Study method and select 43 primary studies via automatic, manual, and snowballing-based search and selection procedures. Subsequently, we use card sorting to synthesize the results. Results. We contribute (i) a categorization of 35 MLOps architecture components, (ii) a description of several MLOps architecture variants, and (iii) a systematic map between the identified components and the existing MLOps tools. Conclusion. This study provides an overview of the state of the art in MLOps from an architectural perspective. Researchers and practitioners can use our findings to inform the architecture design of their MLOps systems.

Education for sustainable development has evolved to include more constructive approaches and a better understanding of what is needed to align education with the cultural, societal, and pedagogical changes required to avoid the risks posed by an unsustainable society. This evolution aims to lead us toward viable, equitable, and sustainable futures. However, computing education, including software engineering, is not fully aligned with the current understanding of what is needed for transformational learning in light of our current challenges. This is partly because computing is primarily seen as a technical field, focused on industry needs. Until recently, sustainability was not a high priority for most businesses, including the digital sector, nor was it a prominent focus for higher education institutions and society. Given these challenges, we aim to propose a research roadmap to integrate sustainability principles and essential skills into the crowded computing curriculum, nurturing future software engineering professionals with a sustainability mindset. We conducted two extensive studies: a systematic review of academic literature on sustainability in computing education and a survey of industry professionals on their interest in sustainability and desired skills for graduates. Using insights from these studies, we identified key topics for teaching sustainability, including core sustainability principles, values and ethics, systems thinking, impact measurement, soft skills, business value, legal standards, and advocacy. Based on these findings, we will develop recommendations for future computing education programs that emphasise sustainability. The paper is accepted at the 2030 Software Engineering workshop, which is co-located with the FSE'24 conference.

In this work, we present a method which determines optimal multi-step dynamic mode decomposition (DMD) models via entropic regression, which is a nonlinear information flow detection algorithm. Motivated by the higher-order DMD (HODMD) method of \cite{clainche}, and the entropic regression (ER) technique for network detection and model construction found in \cite{bollt, bollt2}, we develop a method that we call ERDMD that produces high fidelity time-delay DMD models that allow for nonuniform time space, and the time spacing is discovered by consider most informativity based on ER. These models are shown to be highly efficient and robust. We test our method over several data sets generated by chaotic attractors and show that we are able to build excellent reconstructions using relatively minimal models. We likewise are able to better identify multiscale features via our models which enhances the utility of dynamic mode decomposition.

The Pierre Auger Observatory is the most sensitive instrument to detect photons with energies above $10^{17}$ eV. It measures extensive air showers generated by ultra high energy cosmic rays using a hybrid technique that exploits the combination of a fluorescence detector with a ground array of particle detectors. The signatures of a photon-induced air shower are a larger atmospheric depth of the shower maximum ($X_{max}$) and a steeper lateral distribution function, along with a lower number of muons with respect to the bulk of hadron-induced cascades. In this work, a new analysis technique in the energy interval between 1 and 30 EeV (1 EeV = $10^{18}$ eV) has been developed by combining the fluorescence detector-based measurement of $X_{max}$ with the specific features of the surface detector signal through a parameter related to the air shower muon content, derived from the universality of the air shower development. No evidence of a statistically significant signal due to photon primaries was found using data collected in about 12 years of operation. Thus, upper bounds to the integral photon flux have been set using a detailed calculation of the detector exposure, in combination with a data-driven background estimation. The derived 95% confidence level upper limits are 0.0403, 0.01113, 0.0035, 0.0023, and 0.0021 km$^{-2}$ sr$^{-1}$ yr$^{-1}$ above 1, 2, 3, 5, and 10 EeV, respectively, leading to the most stringent upper limits on the photon flux in the EeV range. Compared with past results, the upper limits were improved by about 40% for the lowest energy threshold and by a factor 3 above 3 EeV, where no candidates were found and the expected background is negligible. The presented limits can be used to probe the assumptions on chemical composition of ultra-high energy cosmic rays and allow for the constraint of the mass and lifetime phase space of super-heavy dark matter particles., Comment: 19 pages, 22 figures

The integration of Smart Surfaces in 6G communication networks, also dubbed as Reconfigurable Intelligent Surfaces (RISs), is a promising paradigm change gaining significant attention given its disruptive features. RISs are a key enabler in the realm of 6G Integrated Sensing and Communication (ISAC) systems where novel services can be offered together with the future mobile networks communication capabilities. This paper addresses the critical challenge of precisely localizing users within a communication network by leveraging the controlled-reflective properties of RIS elements without relying on more power-hungry traditional methods, e.g., GPS, adverting the need of deploying additional infrastructure and even avoiding interfering with communication efforts. Moreover, we go one step beyond: we build COLoRIS, an Opportunistic ISAC approach that leverages localization agnostic RIS configurations to accurately position mobile users via trained learning models. Extensive experimental validation and simulations in large-scale synthetic scenarios show 5% positioning errors (with respect to field size) under different conditions. Further, we show that a low-complexity version running in a limited off-the-shelf (embedded, low-power) system achieves positioning errors in the 11% range at a negligible +2% energy expense with respect to the classical RIS.

We report an investigation of the mass composition of cosmic rays with energies from 3 to 100 EeV (1 EeV=$10^{18}$ eV) using the distributions of the depth of shower maximum $X_\mathrm{max}$. The analysis relies on ${\sim}50,000$ events recorded by the Surface Detector of the Pierre Auger Observatory and a deep-learning-based reconstruction algorithm. Above energies of 5 EeV, the data set offers a 10-fold increase in statistics with respect to fluorescence measurements at the Observatory. After cross-calibration using the Fluorescence Detector, this enables the first measurement of the evolution of the mean and the standard deviation of the $X_\mathrm{max}$ distributions up to 100 EeV. Our findings are threefold: (1.) The evolution of the mean logarithmic mass towards a heavier composition with increasing energy can be confirmed and is extended to 100 EeV. (2.) The evolution of the fluctuations of $X_\mathrm{max}$ towards a heavier and purer composition with increasing energy can be confirmed with high statistics. We report a rather heavy composition and small fluctuations in $X_\mathrm{max}$ at the highest energies. (3.) We find indications for a characteristic structure beyond a constant change in the mean logarithmic mass, featuring three breaks that are observed in proximity to the ankle, instep, and suppression features in the energy spectrum., Comment: Version accepted for publication in Phys. Rev. D, 29 pages, 19 figures, 5 tables

We present measurements of the atmospheric depth of the shower maximum $X_\mathrm{max}$, inferred for the first time on an event-by-event level using the Surface Detector of the Pierre Auger Observatory. Using deep learning, we were able to extend measurements of the $X_\mathrm{max}$ distributions up to energies of 100 EeV ($10^{20}$ eV), not yet revealed by current measurements, providing new insights into the mass composition of cosmic rays at extreme energies. Gaining a 10-fold increase in statistics compared to the Fluorescence Detector data, we find evidence that the rate of change of the average $X_\mathrm{max}$ with the logarithm of energy features three breaks at $6.5\pm0.6~(\mathrm{stat})\pm1~(\mathrm{sys})$ EeV, $11\pm 2~(\mathrm{stat})\pm1~(\mathrm{sys})$ EeV, and $31\pm5~(\mathrm{stat})\pm3~(\mathrm{sys})$ EeV, in the vicinity to the three prominent features (ankle, instep, suppression) of the cosmic-ray flux. The energy evolution of the mean and standard deviation of the measured $X_\mathrm{max}$ distributions indicates that the mass composition becomes increasingly heavier and purer, thus being incompatible with a large fraction of light nuclei between 50 EeV and 100 EeV., Comment: Version accepted for publication in Phys. Rev. Lett., 9 pages, 3 figures, 1 table

Separation logic is a substructural logic which has proved to have numerous and fruitful applications to the verification of programs working on dynamic data structures. Recently, Barthe, Hsu and Liao have proposed a new way of giving semantics to separation logic formulas in which separating conjunction is interpreted in terms of probabilistic independence. The latter is taken in its exact form, i.e., two events are independent if and only if the joint probability is the product of the probabilities of the two events. There is indeed a literature on weaker notions of independence which are computational in nature, i.e. independence holds only against efficient adversaries and modulo a negligible probability of success. The aim of this work is to explore the nature of computational independence in a cryptographic scenario, in view of the aforementioned advances in separation logic. We show on the one hand that the semantics of separation logic can be adapted so as to account for complexity bounded adversaries, and on the other hand that the obtained logical system is useful for writing simple and compact proofs of standard cryptographic results in which the adversary remains hidden. Remarkably, this allows for a fruitful interplay between independence and pseudorandomness, itself a crucial notion in cryptography., Comment: to be published in CSF'24

A key challenge for the development and deployment of artificial intelligence (AI) solutions in radiology is solving the associated data limitations. Obtaining sufficient and representative patient datasets with appropriate annotations may be burdensome due to high acquisition cost, safety limitations, patient privacy restrictions or low disease prevalence rates. In silico data offers a number of potential advantages to patient data, such as diminished patient harm, reduced cost, simplified data acquisition, scalability, improved quality assurance testing, and a mitigation approach to data imbalances. We summarize key research trends and practical uses for synthetically generated data for radiological applications of AI. Specifically, we discuss different types of techniques for generating synthetic examples, their main application areas, and related quality control assessment issues. We also discuss current approaches for evaluating synthetic imaging data. Overall, synthetic data holds great promise in addressing current data availability gaps, but additional work is needed before its full potential is realized., Comment: Provisionally accepted to the British Journal of Radiology (BJR) Artificial Intelligence

Software intensive systems play a crucial role in most, if not all, aspects of modern society. As such, both their sustainability and their role in supporting sustainable processes, must be realized by design. To this aim, the architecture of software intensive systems should be designed to support sustainability goals; and measured to understand how effectively they do so. In this paper, we present the Sustainability Assessment Framework (SAF) Toolkit -- a set of instruments we developed to support software architects and design decision makers in modeling sustainability as a software quality property. The SAF Toolkit is the result of our experience gained in over a decade of cases in collaboration with industrial partners. We illustrate the toolkit with examples stemming from various cases. We extract our lessons learned, and our current research and future plans to extend the SAF Toolkit for further architecture modeling and measurement.

ML-enabled systems that are deployed in a production environment typically suffer from decaying model prediction quality through concept drift, i.e., a gradual change in the statistical characteristics of a certain real-world domain. To combat this, a simple solution is to periodically retrain ML models, which unfortunately can consume a lot of energy. One recommended tactic to improve energy efficiency is therefore to systematically monitor the level of concept drift and only retrain when it becomes unavoidable. Different methods are available to do this, but we know very little about their concrete impact on the tradeoff between accuracy and energy efficiency, as these methods also consume energy themselves. To address this, we therefore conducted a controlled experiment to study the accuracy vs. energy efficiency tradeoff of seven common methods for concept drift detection. We used five synthetic datasets, each in a version with abrupt and one with gradual drift, and trained six different ML models as base classifiers. Based on a full factorial design, we tested 420 combinations (7 drift detectors * 5 datasets * 2 types of drift * 6 base classifiers) and compared energy consumption and drift detection accuracy. Our results indicate that there are three types of detectors: a) detectors that sacrifice energy efficiency for detection accuracy (KSWIN), b) balanced detectors that consume low to medium energy with good accuracy (HDDM_W, ADWIN), and c) detectors that consume very little energy but are unusable in practice due to very poor accuracy (HDDM_A, PageHinkley, DDM, EDDM). By providing rich evidence for this energy efficiency tactic, our findings support ML practitioners in choosing the best suited method of concept drift detection for their ML-enabled systems., Comment: Accepted for publication at the International Conference on Information and Communications Technology for Sustainability 2024 (ICT4S'24)

Machine learning algorithms are fundamental components of novel data-informed Artificial Intelligence architecture. In this domain, the imperative role of representative datasets is a cornerstone in shaping the trajectory of artificial intelligence (AI) development. Representative datasets are needed to train machine learning components properly. Proper training has multiple impacts: it reduces the final model's complexity, power, and uncertainties. In this paper, we investigate the reliability of the $\varepsilon$-representativeness method to assess the dataset similarity from a theoretical perspective for decision trees. We decided to focus on the family of decision trees because it includes a wide variety of models known to be explainable. Thus, in this paper, we provide a result guaranteeing that if two datasets are related by $\varepsilon$-representativeness, i.e., both of them have points closer than $\varepsilon$, then the predictions by the classic decision tree are similar. Experimentally, we have also tested that $\varepsilon$-representativeness presents a significant correlation with the ordering of the feature importance. Moreover, we extend the results experimentally in the context of unseen vehicle collision data for XGboost, a machine-learning component widely adopted for dealing with tabular data.

Recent advances in artificial intelligence (AI) capabilities have increased the eagerness of companies to integrate AI into software systems. While AI can be used to have a positive impact on several dimensions of sustainability, this is often overshadowed by its potential negative influence. While many studies have explored sustainability factors in isolation, there is insufficient holistic coverage of potential sustainability benefits or costs that practitioners need to consider during decision-making for AI adoption. We therefore aim to synthesize trade-offs related to sustainability in the context of integrating AI into software systems. We want to make the sustainability benefits and costs of integrating AI more transparent and accessible for practitioners. The study was conducted in collaboration with a Dutch financial organization. We first performed a rapid review that led to the inclusion of 151 research papers. Afterward, we conducted six semi-structured interviews to enrich the data with industry perspectives. The combined results showcase the potential sustainability benefits and costs of integrating AI. The labels synthesized from the review regarding potential sustainability benefits were clustered into 16 themes, with "energy management" being the most frequently mentioned one. 11 themes were identified in the interviews, with the top mentioned theme being "employee wellbeing". Regarding sustainability costs, the review discovered seven themes, with "deployment issues" being the most popular one, followed by "ethics & society". "Environmental issues" was the top theme from the interviews. Our results provide valuable insights to organizations and practitioners for understanding the potential sustainability implications of adopting AI., Comment: Accepted for publication at the 8th International Workshop on Green and Sustainable Software (GREENS'24), collocated with ICSA'24

The flux of ultra-high energy cosmic rays reaching Earth above the ankle energy (5 EeV) can be described as a mixture of nuclei injected by extragalactic sources with very hard spectra and a low rigidity cutoff. Extragalactic magnetic fields existing between the Earth and the closest sources can affect the observed CR spectrum by reducing the flux of low-rigidity particles reaching Earth. We perform a combined fit of the spectrum and distributions of depth of shower maximum measured with the Pierre Auger Observatory including the effect of this magnetic horizon in the propagation of UHECRs in the intergalactic space. We find that, within a specific range of the various experimental and phenomenological systematics, the magnetic horizon effect can be relevant for turbulent magnetic field strengths in the local neighbourhood of order $B_{\rm rms}\simeq (50-100)\,{\rm nG}\,(20\rm{Mpc}/{d_{\rm s})( 100\,\rm{kpc}/L_{\rm coh}})^{1/2}$, with $d_{\rm s}$ the typical intersource separation and $L_{\rm coh}$ the magnetic field coherence length. When this is the case, the inferred slope of the source spectrum becomes softer and can be closer to the expectations of diffusive shock acceleration, i.e., $\propto E^{-2}$. An additional cosmic-ray population with higher source density and softer spectra, presumably also extragalactic and dominating the cosmic-ray flux at EeV energies, is also required to reproduce the overall spectrum and composition results for all energies down to 0.6~EeV., Comment: Matches published version

In recent years, Deep Learning has gained popularity for its ability to solve complex classification tasks, increasingly delivering better results thanks to the development of more accurate models, the availability of huge volumes of data and the improved computational capabilities of modern computers. However, these improvements in performance also bring efficiency problems, related to the storage of datasets and models, and to the waste of energy and time involved in both the training and inference processes. In this context, data reduction can help reduce energy consumption when training a deep learning model. In this paper, we present up to eight different methods to reduce the size of a tabular training dataset, and we develop a Python package to apply them. We also introduce a representativeness metric based on topology to measure how similar are the reduced datasets and the full training dataset. Additionally, we develop a methodology to apply these data reduction methods to image datasets for object detection tasks. Finally, we experimentally compare how these data reduction methods affect the representativeness of the reduced dataset, the energy consumption and the predictive performance of the model.

The growing use of large machine learning models highlights concerns about their increasing computational demands. While the energy consumption of their training phase has received attention, fewer works have considered the inference phase. For ML inference, the binding of ML models to the ML system for user access, known as ML serving, is a critical yet understudied step for achieving efficiency in ML applications. We examine the literature in ML architectural design decisions and Green AI, with a special focus on ML serving. The aim is to analyze ML serving architectural design decisions for the purpose of understanding and identifying them with respect to quality characteristics from the point of view of researchers and practitioners in the context of ML serving literature. Our results (i) identify ML serving architectural design decisions along with their corresponding components and associated technological stack, and (ii) provide an overview of the quality characteristics studied in the literature, including energy efficiency. This preliminary study is the first step in our goal to achieve green ML serving. Our analysis may aid ML researchers and practitioners in making green-aware architecture design decisions when serving their models., Comment: Accepted for publication as short paper in Conference on AI Engineering Software Engineering for AI (CAIN 2024)

White light spectral interferometry is applied to measure the refractive index in absorbing liquids in the spectral range from 400 to 1000 nm. We analyze the influence of absorption on the visibility of interferometric fringes and, accordingly, on the measurement of the refractive index. Further, we show that the refractive index in the absorption band can be retrieved by a two step process. The procedure requires the use of two samples of different thickness, the thicker one to retrieve the refractive index in the transparent region and the thinnest to obtain the data in the absorption region. First, the refractive index values are retrieved with good accuracy in the transparent region of the material for 1 mm thick samples. Second, these refractive index values serve also to precisely calculate the thickness of a thinner sample (about 150 micrometers) since the accuracy of the method depends strongly on the thickness of the sample. Finally, the refractive index is recovered for the entire spectral range. Copyright 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement. See corresponding link in the pdf file., Comment: 9 pages, 6 figures, 2 tables

We design a spectrally resolved interferometer to measure the refractive index of transparent samples over a broad spectral range between 400 and 550 nm. The measuring device consists of a Michelson interferometer whose output is analysed by means of three fibre spectrometers: a homemade prism spectrometer which obtains the interferogram generated by the sample between 400 and 1050 nm, a homemade transmission grating spectrometer that measures the interferogram in between 950 and 1550 nm, and a commercial Czerny Turner spectrometer used to make high precision measurements of the displacement between the Michelson mirrors. The whole system is illuminated by a white light source with an emission spectrum similar to that of a black body. We test the instrument on solid and liquid samples achieving accuracy of up to 0.0001 in the refractive index after fitting it with the Cauchy formula., Comment: 18 pages, 11 figures, 3 tables

In this work the role that charge delocalization plays in the non-linear optical response of ionic liquids is evaluated. The first hyperpolarizability for the non-linear process of second harmonic generation (SHG) and second hyperpolarizability for the non-linear process of electro-optical Kerr-Effect (EOKE) of a large number of ionic liquid forming ions were estimated by means of density functional theory calculations. The results point to that both charge delocalization and molecular geometry are the key features that govern their hyperpolarizabilities. Our findings show that some of the most commonly used anions in ionic liquids are expected to present strong non-linear responses while common cations present a much more limited performance. However, this limitation can be overcome by a proper tailoring of cations to present charge delocalization over large molecular regions. The hypothesis of additivity of hyperpolarizabilities in ionic liquids is tested and exploited to obtain a map of second and third order non-linear susceptibilities of 1496 ion combinations. This map is intended to be a guide for future works on the hyperpolarizability of ILs, Comment: 12 pages, 6 figures, 4 tables, supplementary data associated with this article can be found, in the online version, at https://doi.org/10.1016/j.molliq.2021.118153

A thorough analysis of the refractive index of eleven 1-alkyl-3-methylimidazolium-based ionic liquids with three different anions, tetrafluoroborate bis(trifluoromethylsulfonyl)imide, and trifluoromethanesulfonate, is reported. Refractive indices were estimated, in the temperature interval from 298.15 to 323.15 K, using an Abbe refractometer to determine the value at the sodium D line and white light spectral interferometry to obtain dispersion in the range of wavelengths from 400 to 1000 nm. The first part of the manuscript is focused on the dependence of refractive index with wavelength, temperature, cation alkyl chain length, and anion nature. Once the main features are detailed, and in order to explain the experimental trends, a model for the refractive index is considered where its square is expressed by a single resonance Sellmeier dispersion formula. This formula has two coefficients: the first one identifies the position of the resonance in the spectral axis, and the second one specifies its strength. It was found that, for a given compound, the resonances position is independent of temperature, while the strength varies linearly with it. This model reproduces successfully the experimental data within the refractive index uncertainty. Furthermore, the model allows calculating the thermo-optic coefficient and its wavelength dependence., Comment: Accepted version, 28 pages, 9 figures, 1 table, the Supporting Information is available free of charge on the ACS Publications website at DOI: 10.1021/acs.jpcc.8b08971

Electronic charge delocalization on the molecular backbones of ionic liquid-forming ions substantially impacts their molecular polarizabilities. Density functional theory calculations of polarizabilities and volumes of many cations and anions are reported and applied to yield refractive indices of 1216 ionic liquids. A novel expression for the precise estimation of the molecular volumes of the ionic liquids from simulation data is also introduced, adding quadratic corrections to the usual sum of atomic volumes. Our significant findings include i) that the usual assumption of uniform, additive atomic polarizabilities is challenged when highly mobile electrons in conjugated systems are present, and ii) that cations with conjugated large carbon chains can be used together with anions for the design of ionic liquids with very high refractive indices. A novel relation for the polarizability volume is reported together with a refractive index map made up of the studied ionic liquids, Comment: 12 pages, 7 figures, 6 tables, Supplementary data associated with this article can be found, in the online version, at https://doi.org/10.1016/j.molliq.2021. 117099

We propose Expert Monitoring, an approach that leverages domain expertise to enhance the detection and mitigation of concept drift in machine learning (ML) models. Our approach supports practitioners by consolidating domain expertise related to concept drift-inducing events, making this expertise accessible to on-call personnel, and enabling automatic adaptability with expert oversight.

The class of type-two basic feasible functionals ($\mathtt{BFF}_2$) is the analogue of $\mathtt{FP}$ (polynomial time functions) for type-2 functionals, that is, functionals that can take (first-order) functions as arguments. $\mathtt{BFF}_2$ can be defined through Oracle Turing machines with running time bounded by second-order polynomials. On the other hand, higher-order term rewriting provides an elegant formalism for expressing higher-order computation. We address the problem of characterizing $\mathtt{BFF}_2$ by higher-order term rewriting. Various kinds of interpretations for first-order term rewriting have been introduced in the literature for proving termination and characterizing first-order complexity classes. In this paper, we consider a recently introduced notion of cost-size interpretations for higher-order term rewriting and see second order rewriting as ways of computing type-2 functionals. We then prove that the class of functionals represented by higher-order terms admitting polynomially bounded cost-size interpretations exactly corresponds to $\mathtt{BFF}_2$., Comment: This version subsumes the older conference version. It is submitted (to LMCS) as an special issue paper in honour of FoSSaCs 2024

We test the predictions of hadronic interaction models regarding the depth of maximum of air-shower profiles, $X_{max}$, and ground-particle signals in water-Cherenkov detectors at 1000 m from the shower core, $S(1000)$, using the data from the fluorescence and surface detectors of the Pierre Auger Observatory. The test consists in fitting the measured two-dimensional ($S(1000)$, $X_{max}$) distributions using templates for simulated air showers produced with hadronic interaction models EPOS-LHC, QGSJet II-04, Sibyll 2.3d and leaving the scales of predicted $X_{max}$ and the signals from hadronic component at ground as free fit parameters. The method relies on the assumption that the mass composition remains the same at all zenith angles, while the longitudinal shower development and attenuation of ground signal depend on the mass composition in a correlated way. The analysis was applied to 2239 events detected by both the fluorescence and surface detectors of the Pierre Auger Observatory with energies between $10^{18.5}$ to $10^{19.0}$ eV and zenith angles below $60^\circ$. We found, that within the assumptions of the method, the best description of the data is achieved if the predictions of the hadronic interaction models are shifted to deeper $X_{max}$ values and larger hadronic signals at all zenith angles. Given the magnitude of the shifts and the data sample size, the statistical significance of the improvement of data description using the modifications considered in the paper is larger than $5\sigma$ even for any linear combination of experimental systematic uncertainties., Comment: Published version

Reconfigurable Intelligent Surfaces (RIS) have emerged as a disruptive technology with the potential to revolutionize wireless communication systems. In this paper, we present RIShield, a novel application of RIS technology specifically designed for radiation-sensitive environments. The aim of RIShield is to enable electromagnetic blackouts, preventing radiation leakage from target areas. We propose a comprehensive framework for RIShield deployment, considering the unique challenges and requirements of radiation-sensitive environments. By strategically positioning RIS panels, we create an intelligent shielding mechanism that selectively absorbs and reflects electromagnetic waves, effectively blocking radiation transmission. To achieve optimal performance, we model the corresponding channel and design a dynamic control that adjusts the RIS configuration based on real-time radiation monitoring. By leveraging the principles of reconfiguration and intelligent control, RIShield ensures adaptive and efficient protection while minimizing signal degradation. Through full-wave and ray-tracing simulations, we demonstrate the effectiveness of RIShield in achieving significant electromagnetic attenuation. Our results highlight the potential of RIS technology to address critical concerns in radiation-sensitive environments, paving the way for safer and more secure operations in industries such as healthcare, nuclear facilities, and defense.

The rapid adoption of artificial intelligence (AI) and machine learning (ML) has generated growing interest in understanding their environmental impact and the challenges associated with designing environmentally friendly ML-enabled systems. While Green AI research, i.e., research that tries to minimize the energy footprint of AI, is receiving increasing attention, very few concrete guidelines are available on how ML-enabled systems can be designed to be more environmentally sustainable. In this paper, we provide a catalog of 30 green architectural tactics for ML-enabled systems to fill this gap. An architectural tactic is a high-level design technique to improve software quality, in our case environmental sustainability. We derived the tactics from the analysis of 51 peer-reviewed publications that primarily explore Green AI, and validated them using a focus group approach with three experts. The 30 tactics we identified are aimed to serve as an initial reference guide for further exploration into Green AI from a software engineering perspective, and assist in designing sustainable ML-enabled systems. To enhance transparency and facilitate their widespread use and extension, we make the tactics available online in easily consumable formats. Wide-spread adoption of these tactics has the potential to substantially reduce the societal impact of ML-enabled systems regarding their energy and carbon footprint., Comment: Accepted for publication at the 2024 International Conference on Software Engineering - Software Engineering in Society (ICSE-SEIS'2024); *Caveat:* the published version (https://doi.org/10.1145/3639475.3640111) contains a mistake in Section 4.2 for tactic T6 (incorrect comparison of energy consumption of kNN and Random Forest); this mistake is corrected in this postprint

The solar physics community is entering a golden era that is ripe with next-generation ground- and space-based facilities. With ever-increasing resolving power stemming from the newest observational telescopes, it becomes more challenging to obtain (near-)simultaneous measurements at high spatial, temporal and spectral resolutions, while operating at the diffraction limit of these new facilities. Hence, in recent years there has been increased interest in the capabilities integral field units (IFUs) offer towards obtaining the trifecta of high spatial, temporal and spectral resolutions contemporaneously. To date, IFUs developed for solar physics research have focused on mid-optical and infrared measurements. Here, we present an IFU prototype that has been designed for operation within the near-ultraviolet to mid-optical wavelength range, hence providing additional spectral coverage to the instrument suites developed to date. The IFU was constructed as a low-budget proof-of-concept for the upcoming 2m class Indian National Large Solar Telescope and employs circular cross-section fibres to guide light into a Czerny-Turner configuration spectrograph, with the resulting spectra captured using a high quantum efficiency scientific CMOS camera. Mapping of each input fibre allows for the reconstruction of two-dimensional spectral images, with frame rates exceeding 20 per second possible while operating in a non-polarimetric configuration. The science verification data presented here highlights the suitability of fibre-fed IFUs operating at near-ultraviolet wavelengths for solar physics research. Importantly, the successful demonstration of this type of instrument paves the way for further technological developments to make a future variant suitable for upcoming ground-based and space-borne telescope facilities., Comment: 53 pages, 16 figures, accepted for publication by Solar Physics

Dark matter particles could be superheavy, provided their lifetime is much longer than the age of the universe. Using the sensitivity of the Pierre Auger Observatory to ultra-high energy neutrinos and photons, we constrain a specific extension of the Standard Model of particle physics that meets the lifetime requirement for a superheavy particle by coupling it to a sector of ultra-light sterile neutrinos. Our results show that, for a typical dark coupling constant of 0.1, the mixing angle $\theta_m$ between active and sterile neutrinos must satisfy, roughly, $\theta_m \lesssim 1.5\times 10^{-6}(M_X/10^9~\mathrm{GeV})^{-2}$ for a mass $M_X$ of the dark-matter particle between $10^8$ and $10^{11}~$GeV., Comment: Version accepted for publication in PRD Letters, including Supplemental Material

This paper presents the real-world smart-meter dataset and offers an analysis of solutions derived from the Energy Prediction Technical Challenges, focusing primarily on two key competitions: the IEEE Computational Intelligence Society (IEEE-CIS) Technical Challenge on Energy Prediction from Smart Meter data in 2020 (named EP) and its follow-up challenge at the IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) in 2021 (named as XEP). These competitions focus on accurate energy consumption forecasting and the importance of interpretability in understanding the underlying factors. The challenge aims to predict monthly and yearly estimated consumption for households, addressing the accurate billing problem with limited historical smart meter data. The dataset comprises 3,248 smart meters, with varying data availability ranging from a minimum of one month to a year. This paper delves into the challenges, solutions and analysing issues related to the provided real-world smart meter data, developing accurate predictions at the household level, and introducing evaluation criteria for assessing interpretability. Additionally, this paper discusses aspects beyond the competitions: opportunities for energy disaggregation and pattern detection applications at the household level, significance of communicating energy-driven factors for optimised billing, and emphasising the importance of responsible AI and data privacy considerations. These aspects provide insights into the broader implications and potential advancements in energy consumption prediction. Overall, these competitions provide a dataset for residential energy research and serve as a catalyst for exploring accurate forecasting, enhancing interpretability, and driving progress towards the discussion of various aspects such as energy disaggregation, demand response programs or behavioural interventions.