Search

The BABAR experiment collected electron-positron collisions at the SLAC National Accelerator Laboratory (SLAC) from 1999-2008. Although data taking has stopped 15 years ago, the collaboration is still actively doing data analyses, publishing results, and giving presentations at international conferences. Special considerations were needed to do analyses using a computing environment that was developed decades ago. A framework is required that preserves the data, data access, and the capability of doing analyses using a well defined and preserved environment. Also, BABAR's support by SLAC ended at the beginning of 2021. Fortunately, the High Energy Physics Research Computing group at the University of Victoria (UVic), Canada, offered to provide the new home for the main BABAR computing infrastructure, the Grid Computing Centre Karlsruhe offered to host all data for access by analyses running at UVic, and CERN and the IN2P3 Computing Centre offered to store a backup of all data. This paper presents what was done at BABAR to preserve the data and analysis capabilities and what was needed to move the whole computing infrastructure, including collaboration tools and data files, away from SLAC. It will be shown how BABAR preserved the ability to continue to do data analyses and also have a working collaboration tools infrastructure. This paper will describe on BABAR's experience with such a big change in its infrastructure and what was learned from it, which may be useful to other experiments which are interested in long term analysis support and data preservation in general. [ABSTRACT FROM AUTHOR]

Apptainer (formerly known as Singularity) since its beginning implemented many of its container features with the assistance of a setuidroot program. It still supports that mode, but as of version 1.1.0 it no longer uses setuid by default. This is feasible because it now can mount squashfs filesystems, ext3 filesystems, and overlay filesystems using unprivileged user namespaces and FUSE. It also now enables unprivileged users to build containers, even without requiring system administrators to configure /etc/subuid and /etc/subgid unlike other "rootless" container systems. As a result, all the unprivileged functions can be used nested inside of another container, even if the container runtime prevents any elevated privileges. As of version 1.2.0 Apptainer also supports completely unprivileged encryption of Singularity Image Format (SIF) container files. Performance with a particularly challenging HEP benchmark using the FUSE-based mounts both with and without encryption is essentially identical to the previous methods that required elevated privileges to use the Linux kernel-based counterparts. [ABSTRACT FROM AUTHOR]

The RNTuple I/O subsystem is ROOT's future event data file format and access API. It is driven by the expected data volume increase at upcoming HEP experiments, e.g. at the HL-LHC, and recent opportunities in the storage hardware and software landscape such as NVMe drives and distributed object stores. RNTuple is a redesign of the TTree binary format and API and has shown to deliver substantially faster data throughput and better data compression both compared to TTree and to industry standard formats. In order to let HENP computing workflows benefit from RNTuple's superior performance, however, the I/O stack needs to connect efficiently to the rest of the ecosystem, from grid storage to (distributed) analysis frameworks to (multithreaded) experiment frameworks for reconstruction and ntuple derivation. With the RNTuple binary format soon arriving at its first production release, we present RNTuple's feature set, integration efforts, and its performance impact on the time-to-solution. We show the latest performance figures of RDataFrame analysis code of realistic complexity, comparing RNTuple and TTree as data sources. We discuss RNTuple's approach to functionality critical to the HENP I/O (such as multithreaded writes, fast data merging, schema evolution) and we provide an outlook on the road to its use in production. [ABSTRACT FROM AUTHOR]

ALICE has upgraded many of its detectors for LHC Run 3 to operate in continuous readout mode recording Pb–Pb collisions at 50 kHz interaction rate without trigger. This results in the need to process data in real time at rates 100 times higher than during Run 2. In order to tackle such a challenge we introduced O2, a new computing system and the associated infrastructure. Designed and implemented during the LHC long shutdown 2, O2 is now in production taking care of all the data processing needs of the experiment. O2 is designed around the message passing paradigm, enabling resilient, parallel data processing for both the synchronous (to LHC beam) and asynchronous data taking and processing phases. The main purpose of the synchronous online reconstruction is detector calibration and raw data compression. This synchronous processing is dominated by the TPC detector, which produces by far the largest data volume, and TPC reconstruction runs fully on GPUs. When there is no beam in the LHC, the powerful GPU-equipped online computing farm of ALICE is used for the asynchronous reconstruction, which creates the final reconstructed output for analysis from the compressed raw data. Since the majority of the compute performance of the online farm is in the GPUs, and since the asynchronous processing is not dominated by the TPC in the way the synchronous processing is, there is an ongoing effort to offload a significant amount of compute load from other detectors to the GPU as well. [ABSTRACT FROM AUTHOR]

The ATLAS experiment is a particle physics experiment situated at the Large Hadron Collider (LHC) at CERN. It involves almost 6000 members from approximately 300 institutes spread all over the globe and more than 100 papers published every year. This dynamic environment brings some challenges such as how to ensure publication deadlines, communication between the groups involved, and the continuity of workflows. The solution found for those challenges was automation, which was achieved through the Glance project, more specifically through the Glance Analysis systems, developed to support the analysis and publications life cycle in 2011. Now, after twelve years, in order to satisfy the experiments' most recent needs, the systems need code refactoring and database remodelling. The goal is to have only one system to accommodate all the analysis and publications workflows, the so-called ATLAS Publication Tracking system, an evolution of the current Analysis systems. This project includes a database remodelling that reflects the hierarchical relation between analyses and publications; a code base that supports non-standard workflows; the expansion of the current API so all the authorized ATLAS members can access ATLAS publication data programmatically; a service-oriented architecture for integration with external software, such as GitLab; the creation of an automatic test environment, which assures the quality of the systems on each update. The ATLAS Publication Tracking system is a long-term project being developed with an iterative and incremental approach, which ensures that the most valuable tools are implemented with priority while allowing a smooth transition between the old systems and the new one. [ABSTRACT FROM AUTHOR]

The LHCb experiment is one of the 4 large LHC experiments at CERN. With more than 1500 members and tens of thousands of assets, the Collaboration requires systems that allow the extraction of data from many databases according to some very specific criteria. In LHCb there are 4 production web applications responsible for managing members and institutes, tracking assets and their current status, presenting radiological information about the cavern, and supporting the management of cables. A common requirement shared across all these systems is to allow searching information based on logical expressions. Therefore, in order to avoid rework, the Glance Search Library was created with the goal of providing components for applications to deploy frontend search interfaces capable of generating standardized queries based on users' input, and backend utility functions that compile such queries into a SQL clause. The Glance Search Library is split into 2 smaller libraries maintained in different GitLab repositories. The first one only contains Vue components and JavaScript modules and, in LHCb, it is included as a dependency of the SPAs (Single Page Applications). The second is a PHP Object-Oriented library, mainly used by REST APIs that are required to expose large amounts of data stored in their relational databases. This separation provides greater flexibility and more agile deployments. It also enables lighter applications with no graphical interface to build command line tools solely on top of the backend classes and predefined queries. [ABSTRACT FROM AUTHOR]

The Cherenkov Telescope Array Observatory (CTAO) is the next generation ground-based observatory for gamma-ray astronomy at very high energies. It is expected to produce about 2 PB of raw data each year and to manage a global data volume which will grow through the years to reach about 100 PB in 2030. In addition, CTAO will require a high computing capacity for data processing and Monte Carlo simulations, of the order of hundreds of millions of CPU HS06 hours per year. To meet these requirements, CTAO will adopt a distributed computing model using 4 academic data centers, and will use the DIRAC framework as its workload management system. In the past ten years, to optimize the instrument design and study its performances, CTAO has used the European Grid Infrastructure (EGI) to run massive Monte Carlo campaigns. In order to handle these campaigns and to automatize simulation and data processing workflows, we have developed a production system prototype based on DIRAC. Recently, we have also developed a user interface allowing for the configuration and submission of complex workflows. In this contribution we present the production system prototype, its user interface for workflow management as well as its application to CTAO workflows. [ABSTRACT FROM AUTHOR]

The ATLAS experiment has 18+ years of experience using workload management systems to deploy and develop workflows to process and to simulate data on the distributed computing infrastructure. Simulation, processing and analysis of LHC experiment data require the coordinated work of heterogeneous computing resources. In particular, the ATLAS experiment utilizes the resources of 250 computing centers worldwide, the power of supercomputing centres, and national, academic and commercial cloud computing resources. In this contribution, we present new techniques for cost-effectively improving efficiency introduced in workflow management system software. The evolution from a mesh framework to new types of computing facilities such as cloud and HPCs is described, as well as new types of production and analysis workflows. [ABSTRACT FROM AUTHOR]

In preparation for LHC Run 3 and 4 the ALICE Collaboration has moved to a new Grid middleware, JAliEn, and workflow management system. The migration was dictated by the substantially higher requirements on the Grid infrastructure in terms of payload complexity, increased number of jobs and managed data volume, all of which required a complete rewrite of the middleware using modern software languages and technologies. Through containerisation, self-contained binaries, managed by the JAliEn middleware, we provide a uniform execution environment across sites and various architectures, including accelerators. The model and implementation have proven their scalability and can be easily deployed across sites with minimal intervention. This contribution outlines the architecture of the new Grid workflow as deployed in production and the workflow process. Specifically shown is how core components are moved and bootstrapped through CVMFS, enabling the middleware to run anywhere fully independent of the host system. Furthermore, we examine how new middleware releases, containers and their runtimes are centrally maintained and easily deployed across the Grid, also by the means of a common build system. [ABSTRACT FROM AUTHOR]

The ATLAS EventIndex system consists of the catalogue of all events collected, processed or generated by the ATLAS experiment at the CERN LHC accelerator, and all associated software tools. The new system, developed for LHC Run 3, makes use of Apache HBase—the Hadoop database—and Apache Phoenix—an SQL/relational database layer for HBase—to store and access all the event metadata. The Query Service allows searches for and retrieval of information of interest. Its design is based on a server-client architecture with the sides communicating via a RESTful API. The OpenAPI Specification is used to describe the API and generate parts of the source code of the server and client. Based on selection criteria received from the client, the server executes SQL queries against the Phoenix tables, retrieves the data, and returns it to the client. The client—a command line utility following the UNIX/POSIXconforming program design style—can output results in various formats including plain text and JSON. The server and the client are developed in Java 8, with the use of the Jersey 2 JAX-RS API implementation and the Spring Framework. The service performance is suitable for interactive use with queries for up to 1000 events at a time. [ABSTRACT FROM AUTHOR]

Effective metadata management is a consistent challenge faced by many scientific experiments. These challenges are magnified by the evolving needs of the experiment, the intricacies of seamlessly integrating a new system with existing analytical frameworks, and the crucial mandate to maintain database integrity. In this work we present the various challenges faced by experiments that produce a large amount of metadata and describe the solution used by the XENON experiment for metadata management. [ABSTRACT FROM AUTHOR]

Providing high performance and reliable tape storage system is GridKa's top priority. The GridKa tape storage system was recently migrated from IBM SP to High Performance Storage System (HPSS) for LHC and non-LHC HEP experiments. These are two different tape backends and each has its own design and specifics that need to be studied thoroughly. Taking into account the features and characteristics of HPSS, a new approach has been developed for flushing and staging files to and from tape storage system. This new approach allows better optimized and efficient flush and stage operations and leads to a substantial improvement in the overall performance of the GridKa tape storage system. The efficient interface that was developed to use IBM SP is now adapted to the HPSS use-case to connect the access point from experiments to the tape storage system. This contribution provides details on these changes and the results of the Tape Challenge 2022 within the new HPSS tape storage configuration. [ABSTRACT FROM AUTHOR]

The ATLAS experiment is preparing a major change in the conditions data infrastructure in view of LHC Run 4. In this paper we describe the ongoing changes in the database architecture which have been implemented for Run 3, and describe the motivations and the on-going developments for the deployment of a new system (called CREST for Conditions Representational State Transfer, as a reference to REST architectures). The main goal is to set up a parallel infrastructure for full scale testing before the end of Run 3. [ABSTRACT FROM AUTHOR]

The Hapke model has been widely used in the field of soil remote sensing. However, the latest development of the Hapke model (i.e., Hapke-HSR model) adopted a simple hypothesis to consider the influence of the soil moisture content (SMC), which brought great difficulties to SMC parameter inversion. This paper presents a method to improve the Hapke model using the improved multilayer radiative transfer model of soil reflectance (MARMIT-2), which can effectively improve the ability of the Hapke-HSR model to characterize the variation in the SMC. Finally, we used the soil database to comprehensively verify the ability of the improved Hapke model. The results show that the improved Hapke can effectively characterize the spectral characteristics of soil and show a higher fitting accuracy (RMSE = 0.009) compared with the Hapke-HSR model (RMSE = 0.031), especially at a high SMC (≥30%). Therefore, the improved Hapke model can better understand soil physical properties and improve the inversion accuracy of soil–vegetation physical parameters, which can be used to enhance agricultural water use efficiency. [ABSTRACT FROM AUTHOR]

Digital content is any content that exists in digital data. Digital media, also known as digital content, is stored in specific formats in digital or analog storage. Digital content forms contain information that is digitally transmitted, streamed, or stored in computer files. Narrowly defined, digital content includes a broader range of media, whereas digital content is defined as any type of digital information. Digital content has grown in popularity as more households have access to the Internet. While studies such as Digital Future are not a substitute for your market research in Focus 2010, they might also be a wonderful start to establish the backdrop where you will finally develop your own research and plan People can now watch news and television online thanks to increased access. Traditional platforms' popularity is under threat. Increased internet access has resulted in the widespread distribution of digital content in the form of eBooks, blog posts, and even Facebook posts. [ABSTRACT FROM AUTHOR]

Data security steganography (images) is very important in today's modern world. This is further strengthened by the business need to maintain the confidentiality of data which is very important to be kept confidential. Many image manipulations occur in a negative form and make the original image owner feel disadvantaged because the image has been manipulated and shows a bad image. Confidential data is made into a storage and delivery system so that it is not read or changed by irresponsible people, both when the data is stored as an image file on a computer or when the data is sent via the internet such as email, and other online storage media. Therefore, Least Significant Bit and End Of File algorithms are used to increase the security of an image. The system that will be built in this research will create an application that can insert images into images. Confidential images will be inserted into the image so that the image remains safe from people who intend to manipulate the image. [ABSTRACT FROM AUTHOR]

Selecting a suitable file format for data exchange in additive manufacturing is fundamental when designing these digital process chains. Within the scope of this investigation, alternatives to the de-facto industry standard STL are to be found to overcome the disadvantages of the STL-based digital process chain. Therefore, suitable file formats are identified by conducting literature and market research and evaluated regarding their suitability to support a continuous digital process chain. In addition, typical use cases in additive manufacturing are defined, and their requirements for a file format for data exchange are derived. Finally, for each use case defined, recommended and suitable file formats are proposed. [ABSTRACT FROM AUTHOR]