Search

StreamDevice, as a general-purpose string interface device���s Epics driver, has been widely used in the control of devices with network and serial ports in CAFe equipment. For example, the remote control of magnet power supply, vacuum gauges, and various vacuum valves or pumps, as well as the information reading and control of Gauss meter equipment used in magnetic field measurement. In the process of on-site software development, we found that various errors are caused during the deployment of StreamDevice about the dependence on software environment and library functions, which because of different operating system environments and EPICS tool software versions. This makes StreamDevice deployment very time-consuming and labor-intensive. To ensure that StreamDevice works in a unified environment and can be deployed and migrated efficiently, the Docker container technology is used to encapsulate its software and its application environment. Images will be uploaded to an Aliyun private library to facilitate software developers to download and use., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

Programmable Logic Controllers (PLC) are widely used for industrial automation including safety systems at CERN. The incorrect behaviour of the PLC control system logic can cause significant financial losses by damage of property or the environment or even injuries in some cases, therefore ensuring their correct behaviour is essential. While testing has been for many years the traditional way of validating the PLC control system logic, CERN developed a model checking platform to go one step further and formally verify PLC logic. This platform, called PLCverif, first released internally for CERN usage in 2019, is now available to anyone since September 2020 via an open source licence. In this paper, we will first give an overview of the PLCverif platform capabilities before focusing on the improvements done since 2019 such as the larger support coverage of the Siemens PLC programming languages, the better support of the C Bounded Model Checker backend (CBMC) and the process of releasing PLCverif as an open-source software., 18th International Conference on Accelerator and Large Experimental Physics Control Systems (ICALEPCS2021)

In the field of Experimental Physics and Industrial Control Systems (EPICS)*, the traditional tool to implement high level procedures is the Sequencer*. While this is a mature, fast, and well-proven software, it comes with some drawbacks. For example, it���s based on a custom C-like programming language which may be unfamiliar to new users and it often results in complex, hard to read code. This paper presents pysmlib, a free and open source Python library developed as a simpler alternative to the EPICS Sequencer. The library exposes a simple interface to develop event-driven Finite State Machines (FSM), where the inputs are connected to Channel Access Process Variables (PV) thanks to the PyEpics** integration. Other features include parallel FSM with multi-threading support and input sharing, timers, and an integrated watchdog logic. The library offers a lower barrier to enter and greater extensibility thanks to the large ecosystem of scientific and engineering python libraries, making it a perfect fit for modern control system requirements. Pysmlib has been deployed in multiple projects at INFN Legnaro National Laboratories (LNL), proving its robustness and flexibility., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

Building C++ software for low-level computers requires carefully tested frameworks and libraries. The major difficulties in building C++ software are to ensure that the artifacts are compatible with the target system���s (OS, Application Binary Interface), and to ensure that transitive dependent libraries are compatible when linked together. Thus developers/maintainers must be provided with efficient tooling for friction-less workflows: standardisation of the project description and build, automatic CI, flexible development environment. The open-source community with services like Github and Gitlab have set high expectations with regards to developer user experience. This paper describes how we leveraged Conan and CMake to standardise the build of C++ projects, avoid the "dependency hell" and provide an easy way to distribute C++ packages. A CI system orchestrated by Jenkins and based on automatic job definition and in-source, versioned, configuration has been implemented. The developer experience is further enhanced by wrapping the common flows (compile, test, release) into a command line tool, which also helps transitioning from the legacy build system (legacy makefiles, SVN)., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

The control system of large scientific facilities is composed of several heterogeneous control systems. As time goes by, the facilities need to be continuously upgraded and the control system also needs to be upgraded. This is a challenge for the integration of complex and large-scale heterogeneous systems. This article describes the design of a data integration model based on component technology, software middleware(The Apache Thrift*) and real-time database. The realization of this model shields the relevant details of the software middleware, encapsulates the remote data acquisition as a local function operation, realizes the combination of data and complex calculations through scripts, and can be assembled into new components., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

The Controls Configuration Service (CCS) is a core component of the data-driven Control System at CERN. Built around a central database, the CCS provides a range of client APIs and graphical user interfaces (GUI) to enable efficient and user-friendly configuration of Controls. As the entry point for all the modifications to Controls system configurations, the CCS provides the means to ensure global data coherency and propagation of changes across the distributed Controls sub-systems and services. With the aim of achieving global data coherency in the most efficient manner, the need for an advanced data integrator emerged. The Controls Configuration Data Lifecycle manager (CCDL) is the core integration bridge between the distributed Controls sub-systems. It aims to ensure consistent, reliable, and efficient exchange of information and triggering of workflow actions based on events representing Controls configuration changes. The CCDL implements and incorporates cutting-edge technologies used successfully in the IT industry. This paper describes the CCDL architecture, design and technology choices made, as well as the tangible benefits that have been realised since its introduction., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

OPC Unified Architecture (OPC UA) is an open platform independent communication architecture for industrial automation developed by the OPC Foundation. Its key characteristics include a rich service-oriented architecture, enhanced security functionality and an integral information model, allowing to map complex data into an OPC UA namespace. With its increasing popularity in the industrial world, OPC UA is an excellent strategic choice for integrating a wealth of different COTS devices and controllers into an existing control system infrastructure. The security functions extend its application to larger networks and across firewalls, while the support of user-defined data structures and fully symbolic addressing ensure flexibility, separation of concerns and robustness in the user interfaces. In an international collaboration, a generic OPC UA support for the EPICS control system toolkit has been developed. It is used in operation at several facilities, integrating a variety of commercial controllers and systems. We describe design and implementation approach, discuss use cases and software quality aspects, report performance and present a roadmap of the next development steps., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

The ITER Real-Time Framework (RTF) is a middleware providing common services and capabilities to build real-time control applications in ITER such as the Plasma Control System (PCS) and plasma diagnostics. The RTF dynamically constructs applications at runtime from the configuration. The principal building blocks that compose an application process are called Function Blocks (FB), which follow a modular structure pattern. The application configuration defines the information that can influence control behavior, such as the connections among FBs, their corresponding parameters, and event handlers. The consecutive pipeline process in a busy-waiting mode and a data-driven pattern minimizes jitter and hardens the deterministic system behavior. In contrast, infrastructural capabilities are managed differently in the service layer using non-real-time threads. The deployment configuration covers the final placement of a program instance and thread allocation to the appropriate computing infrastructure. In this paper, we will introduce the architecture and design patterns of the framework as well as the real-life examples used to benchmark the RTF., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

Obsolescence is one of the challenges facing all long-term projects. It not only affects hardware platforms, but also software. Python 2.x reached official End Of Life status on 1 January 2020. In this paper we review our efforts to port to the replacement, Python 3.x. While the two versions are very similar, there are important differences which can lead to incompatibility or changes in behaviour. We discuss our motivation and strategy for porting our code base of approximately 200 k source lines of code over 20 Python packages. This includes aspects such as internal and external dependencies, legacy and proprietary software that cannot be easily ported, testing and verification, and why we selected a phased approach rather than "big bang". We also report on the challenges and lessons learnt - notably why good test coverage is so important for software maintenance. Our application is the 64-antenna MeerKAT radio telescope in South Africa - a precursor to the Square Kilometre Array, Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

In order to find out the problem of the device remotely, we aimed to develop a new alarm system. The main functions of the alarm system are real-time monitoring of EPICS PV data, data storage, and data storage when an alarm occurs. In addition, an alarm is transmitted in real time through an app on the smartphone to communicate the situation to machine engineers of PLS-II. This system uses InfluxDB to store data. In addition, a new program written in Java language was developed so that data acquisition, analysis, and beam dump conditions can be known. furthermore Vue.js is used to develop together with node.js and web-based android and iOS-based smart phone applications, and user interface is serviced. Eventually, using this system, we were able to check the cause analysis and data in real time when an alarm occurs. In this paper, we introduce the design of an alarm system and the transmission of alarms to an application., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

In 2019, the Tango Controls Collaboration decided to write down a formal specification of the existing Tango Controls protocol as Requests For Comments (RFC). The work resulted in a Markdown-formatted specification rendered in HTML and PDF on Readthedocs.io. The specification is already used as a reference during Tango Controls source code maintenance and for prototyping a new implementation. All collaborating institutes and several companies were involved in the work. In addition to providing the reference, the effort brought the Community more value: review and clarification of concepts and their implementation in the core libraries in C++, Java and Python. This paper summarizes the results, provides technical and organizational details about writing the RFCs for the existing protocol and presents the impact and benefits on future maintenance and development of Tango Controls., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

The SKA telescope control system comprises several devices working on different hierarchies on different sites to provide a running observatory. The importance of logs, whether in its simplest form or correlated, in this system as well as any other distributed system is critical to fault finding and bug tracing. The SKA logging system will collect logs produced by numerous networked kubernetes deployments of devices and processes running a combination off-the-shelf, derived and bespoke software. The many moving parts of this complex system are delivered and maintained by different agile teams on multiple SKA Agile Release Trains. To facilitate an orderly and correlated generation of events in the running telescope, we implement a logging architecture which enforces consistently formatted logs with event tracing capability. We discuss the details of the architecture design and implementation, ending off with the limitations of the tracing solution in the context of a multiprocessing environment., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

The photon beam lines and instruments at the European XFEL (EuXFEL) are operated using the Karabo* control system that has been developed in house since 2011. Monitoring and incident analysis requires quick access to historic values of control data. While Karabo���s original custom-built text-file-based data logging system suits well for small systems, a time series data base offers in general a faster data access, as well as advanced data filtering, aggregation and reduction options. EuXFEL has chosen InfluxDB** as backend that is operated since summer 2020. Historic data can be displayed as before via the Karabo GUI or now also via the powerful Grafana*** web interface. The latter is e.g. used heavily in the new Data Operation Center of the EuXFEL. This contribution describes the InfluxDB setup, its transparent integration into Karabo and the experiences gained since it is in operation., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

Alarms are one of the most important aspects of control systems. Each control system can face unexpected issues, which demand fast and precise resolution. As the control system starts to grow, it requires the involvement of more engineers to access the alarm���s list and focus on the most important ones. Our objective was to allow users to access the alarms fast, remotely and without special software. According to current trends in the IT community, creating a web application turned out to be a perfect solution. Our application is the extension and web equivalent to the current Panic GUI application. It allows constant remote access using just a web browser which is currently present on every machine including mobile phones and tablets. The access to the different functionalities can be restricted to the users provided just with appropriate roles. Alarms can be easily added and managed from the web browser as well as adding new data sources is possible. From each data source, an attribute can be extracted, and multiple attributes can be combined into composer being the base for further analysis or alarms creation., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

LabVIEW software is often chosen for developing small scale control systems, especially for novice software developers. However, because of its ease of use, many functional LabVIEW applications suffer from limits to extensibility and scalability. Developing highly extensible and scalable applications requires significant skill and time investment. To close this gap between new and experienced developers we present an object-oriented application framework that offloads complex architecture tasks from the developer. The framework provides native functionality for data acquisition, logging, and publishing over HTTP and WebSocket with extensibility for adding further capabilities. The system is scalable and supports both single process applications and small to medium sized distributed systems. By leveraging the framework, developers can produce robust applications that are easily integrated into a unified architecture for simple and distributed systems. This allows for decreased system development time, improved onboarding for new developers, and simple framework extension for new capabilities., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

The institute of modern physics (IMP) has two sets of linear accelerator facilities, they are CAFe (China ADS front-end demo linac) and LEAF (Low Energy Accelerator Facility). The Main equipment of LEAF facility consists of ion source, LEBT (Low Energy Beam Transport), RFQ (Radio Frequency Quadrupole) and some experiment terminals. Compare with LEAF, CAFe equipment has more and adds MEBT (Middle Energy Beam Transport) and four sets of superconducting cavity strings at the back end of RFQ. In the process of commissioning and running linac equipment, The EPICS Archiver application and Alarm system are used. According to the refined control requirements of the facility sites, we have completed the software upgrade and deployment of the archiver and alarm systems. The upgraded software systems have made the operation of linac machines more effective in term of monitoring, fault-diagnostic and system recovery, and becomes more user-friendly as well., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

Web-based applications have seen a huge increase in popularity in recent years, replacing standalone applications. GraphQL provides a complete and understandable description of the data exchange between client browsers and back-end servers. GraphQL is a powerful query language allowing API to evolve easily and to query only what is needed. GraphQL also offers a WebSocket based protocol which perfectly fit to the Tango event system. Lots of popular tools around GraphQL offer very convenient way to browse and query data. TangoGraphQL is a pure C++ http(s) server which exports a GraphQL binding for the Tango C++ API. TangoGraphQL also exports a GraphiQL web application which allows to have a nice interactive description of the API and to test queries. TangoGraphQL* has been designed with the aim to maximize performances of JSON data serialization, a binary transfer mode is also foreseen., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

The start of the Advanced Proton Driven Plasma Wakefield Acceleration Experiment (AWAKE) facility at CERN in 2016 came with the need for a continuous image acquisition system. The international scientific collaboration responsible for this project requested low and high resolution acquisition at a capture rate of 10Hz and 1 Hz respectively. To match these requirements, GigE digital cameras were connected to a PXI system running PharLap, a real-time operating system, using dual port 1GB/s network cards. With new requirements for a faster acquisition with higher resolution, it was decided to add 10GB/s network cards and a Network Attached Storage (NAS) directly connected to the PXI system to avoid saturating the network. There was also a request to acquire high-resolution images on several cameras during a limited duration, typically 30 seconds, in a burst acquisition mode. To comply with these new requirements PharLap had to be abandoned and replaced with Linux RT. This paper describes the limitation of the PharLap system and the lessons learned during the transition to Linux RT. We will show the improvement of CPU stability and data throughput reached., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

Python provides broad design freedom to programmers and a low barrier of entry for new software developers. These aspects have proven that unless standardized, a Python codebase will tend to diverge from a common style and architecture, becoming unmaintainable across the scope of a large controls system. Mitigating these effects requires a set of tools, standards, and procedures developed to assert boundaries on certain aspects of Python development ��� namely project organization, version management, and deployment procedures. Common tools like Git, GitLab, and virtual environments form a basis for development, with in-house utilities presenting their capabilities in a clear, developer-focused way. This paper describes the necessary constraints needed for development and deployment of large-scale Python applications, the function of the tools which comprise the development environment, and how these tools are leveraged to create simple and effective procedures to guide development., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

At the turn of 2020/2021, the Tango community faced the challenge of a massive migration of all Tango software repositories from GitHub to GitLab. The motivation has been a change in the pricing model of the Travis CI provider and the shutdown of the JFrog Bintray service used for artifact hosting. GitLab has been chosen as a FOSS-friendly platform for storing both the code and build artifacts and for providing CI/CD services. The migration process faced several challenges, both technical, like redesign and rewrite of CI pipelines, and non-technical, like coordination of actions impacting multiple interdependent repositories. This paper explains the strategies adopted for migration, the outcomes, and the impact on the Tango Controls collaboration., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

ALBA, a 3rd Generation Synchroton Light Source located near Barcelona in Spain, is in operation since 2012. During the last 10 years, the updates of ALBA���s Control System were severely limited in order to prevent disruptions of production equipment, at the cost of having to deal with hardware and software obsolescence, elevating the effort of maintenance and enhancements. The construction of the second phase new beamlines accelerated the renewal of the software stack. In order to limit the number of supported platforms we also gradually upgraded the already operational subsystems. We are in the process of switching to the Debian OS, upgrading to the Tango 9 Control System framework including the Tango Archiving System to HDB++, migrating our code to Python 3, and migrating our GUIs to PyQt5 and PyQtGraph, etc. In order to ensure the project quality and to facilitate future upgrades, we try to automate testing, packaging, and configuration management with CI/CD pipelines using, among others, the following tools: pytest, Docker, GitLab-CI and Salt. In this paper, we present our strategy in this project, the current status of different upgrades and we share the lessons learnt., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

As with all experimental physics facilities, NIF has software applications that must persist on a multi-decade timescale. They must be kept up to date for viability, sustainability and security. We present the steps and challenges involved in a major application upgrade project from Oracle APEX v5 to Oracle APEX v19.2. This upgrade involved jumping over 2 major versions and a total of 5 releases of Oracle APEX. Some applications that depended on now legacy Oracle APEX constructs required redesigning, while others that broke due to custom JavaScript needed to be updated for compatibility. This upgrade project, undertaken by the NIF Shot Data Systems team at LLNL, involved reverse-engineering functional requirements for applications that were then redesigned using the latest APEX out-of-the-box functionality, as well as identifying changes made in the new Oracle APEX built-in ���plumbing��� to update custom-built features for compatibility with the new Oracle APEX version. As NIF enters into its second decade of operations, this upgrade allows these aging applications to function in a more sustainable way, while enhancing user experience with a modernized GUI for Oracle APEX web-pages., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

The Square Kilometre Array (SKA) is an international effort to build two radio interferometers in South Africa and Australia forming one Observatory monitored and controlled from global headquarters (GHQ) based in the United Kingdom at Jodrell Bank. SKA is highly focused on adopting CI/CD practices for its software development. CI/CD stands for Continuous Integration \ Delivery and/or Deployment. Continuous Integration is the practice of merging all developers��� local copies into the mainline frequently. Continuous Delivery is the approach of developing software in short cycles ensuring it can be released anytime, and Continuous Deployment is the approach of delivering the software into operational use frequently and automatically. This paper analyses the decisions taken by the Systems Team (a specialized agile team devoted to developing and maintaining the tools that allow continuous practices) to promote the CI/CD practices with the TANGO-controls framework., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

Supervision, Control and Data Acquisition (SCADA) systems generate large amounts of data over time. Analyzing collected data is essential to discover useful information, prevent failures, and generate reports. Facilitating access to data is of utmost importance to exploit the information generated by SCADA systems. CERN���s occupational Health Safety and Environmental protection (HSE) Unit operates a web infrastructure allowing users of the Radiation and Environment Monitoring Unified Supervision (REMUS) to visualize and extract near-real-time and historical data from desktop and mobile devices. This application, REMUS Web, collects and combines data from multiple sources and presents it to the users in a format suitable for analysis. The web application and the SCADA system can operate independently thanks to a data-centric, loosely coupled architecture. They are connected through common data sources such as the open-source streaming platform Apache Kafka and Oracle Rdb. This paper describes the benefits of providing a feature-rich web application as a complement to control systems. Moreover, it details the underlying architecture of the solution and its capabilities., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

This paper presents the Continuous Integration and Continuous Deployment (CI/CD) tool chain for WinCC Open Architecture applications and frameworks developed at CERN, enabling a DevOps oriented approach of working. By identifying common patterns and time consuming procedures, and by agreeing on standard repository structures, naming conventions and tooling, we have gained a turnkey solution which automates the compilation of binaries and generation of documentation, thus guaranteeing they are up to date and match the source code in the repository. The pipelines generate deployment-ready software releases, which pass through both static code analysis and unit tests before automatically being deployed to short and long-term repositories. The tool chain leverages industry standard technologies, such as GitLab, Docker and Nexus. The technologies chosen for the tool chain are well understood and have a long, solid track record, reducing the effort in maintenance and potential long term risk. The setup has reduced the expert time needed for testing and releases, while generally improving the release quality., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

The Unified Controls Acquisition and Processing (UCAP) framework provides a means to facilitate and streamline data processing in the CERN Accelerator Control System. UCAP���s generic structure is capable of tackling classic "Acquisition - Transformation - Publishing/Presentation" use cases, ranging from simple aggregations to complex machine reports and pre-processing of software interlock conditions. In addition to enabling end-users to develop data transformations in Java or Python and maximising integration with other controls sub-systems, UCAP puts an emphasis on offering self-service capabilities for deployment, operation and monitoring. This ensures that accelerator operators and equipment experts can focus on developing domain-specific transformation algorithms, without having to pay attention to typical IT tasks, such as process management and system monitoring. UCAP is already used by Linac4, PSB and SPS operations and will be used by most CERN accelerators, including LHC by the end of 2021. This contribution presents the UCAP framework and gives an insight into how we have productively combined modern agile development with conservative technical choices., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

vscode-epics is a Visual Studio Code module developed by CEA Irfu that aims to enlight your EPICS code. This module makes developer life easier, improves code quality and helps standardizing EPICS code. It provides syntax highlighting, snippets and header template for EPICS file and provides snippets for WeTest*. This VSCode module is based on Visual Studio Code language Extension and it uses basic JSON files that make feature addition easy. The number of downloads increases version after version and the different feedback motivates us to strongly maintain it for the EPICS community. Since 2019, some laboratories of the EPICS community have participated in the improvement of the module and it seems to have a nice future (linter, snippet improvements, specific language support, etc.). The module is available on Visual Studio Code marketplace** and on EPICS extension GitHub***. CEA Irfu is open to bug notifications, enhancement suggestions and merge requests to continuously improve vscode-epics., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

The CERN Control and Monitoring platform (C2MON) is an open-source platform for industrial controls data acquisition, monitoring, control and data publishing. Its high availability, fault tolerance and redundancy make it a perfect fit to handle the complex and critical systems present at CERN. C2MON must cope with the ever-increasing flows of data produced by the CERN technical infrastructure, such as cooling and ventilation or electrical distribution alarms, while maintaining integrity and availability. Distributed caching is a common technique to dramatically increase the availability and fault tolerance of redundant systems. For C2MON we have replaced the existing legacy Terracotta caching framework with Apache Ignite. Ignite is an enterprise grade, distributed caching platform, with advanced cloud-native capabilities. It enables C2MON to handle high volumes of data with full transaction support and makes C2MON ready to run in the cloud. This article first explains the challenges we met when integrating Apache Ignite into the C2MON framework, and then demonstrates how Ignite enhances the capabilities of a monitor and control system in an industrial controls environment., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

Proceedings of ICALEPCS2021 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai (virtual), Peoples R China, 14 Oct 2021 - 21 Oct 2021; JACoW Publishing, Geneva, Switzerland 188-192 (2021). doi:10.18429/JACoW-ICALEPCS2021-MOPV027, This contribution traces the development of DESY's control system DOOCS from its origins in 1992 to its current state as the backbone of the European XFEL and FLASH accelerators and of the future Petra IV light source. Some details of the continual modernization and refactoring efforts on the 1.5 million line C++ code base are highlighted., Published by JACoW Publishing, Geneva, Switzerland

As part of the MACH-B (Multipole Accelerator Codes for Hadron Beams) project, we have developed a Fast Multipole Method (FMM**)-based tool for higher fidelity modeling of particle accelerators for high-energy physics within Fermilab���s Synergia* simulation package. We present results from our implementations with a focus on studying the difference between tune depression estimates obtained using PIC codes for computing the particle interactions and those obtained using FMM-based algorithms integrated within Synergia. In simulating the self-interactions and macroparticle actions necessary for accurate simulations, we present a newly-developed kernel inside of a kernel-independent FMM in which near-field kernels are modified to incorporate smoothing while still maintaining consistency at the boundary of the far-field regime. Each simulation relies on Synergia with one major difference: the way in which particles interactions were computed. Specifically, following our integration of the FMM into Synergia, changes between PIC-based computations and FMM-based computations are made by changing only the method for near-field (and self) particle interactions., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

In 2019, Python was adopted as an officially supported language for interacting with CERN���s accelerator controls. In practice, this change of status was as much pragmatic as it was progressive - Python has been available as part of the underlying operating system for over a decade and unofficial Python interfaces to controls have existed since at least 2015. So one might ask: what really changed when Python���s adoption became official? This paper will discuss what it takes to officially support Python in a controls environment and will focus on the cultural and technological shifts involved in running Python operationally. It will highlight some of the infrastructure that has been put in place at CERN to facilitate a stable and user-friendly Python platform, as well as some of the key decisions that have led to Python thriving in CERN���s accelerator controls domain. Given its general nature, it is hoped that the approach presented in this paper can serve as a reference for other scientific organisations from a broad range of fields who are considering the adoption of Python in an operational context., Proceedings of the 18th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2021, Shanghai, China

There are a variety of beam current transformers that are used at all accelerator facilities for current and bunch charge measurements. Transformer signals are traditionally measured using integrator electronics followed by a digitizer. However, integrator circuits have a limited bandwidth and are susceptible to noise. By directly digitizing the output of the transformer, the signal bandwidth is limited only by the transformer characteristics and the digitizing platform. Digital integration and filtering can then easily be applied to reduce noise resulting in an overall improvement of the beam parameter measurements. This paper describes a Python-based application that performs the filtering and integration of a current transformer pulse that has been directly digitized by an oscilloscope., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

The MeerKAT Control and Monitoring(CAM) solution is a mature software system that has undergone multiple phases of construction and expansion. It is a distributed system with a run-time environment of 15 logical nodes featuring dozens of interdependent, short-lived processes that interact with a number of long-running services. This presents a challenge for the development team to balance operational goals with continued discovery and development of useful enhancements for its users (astronomers, telescope operators). Continuous Delivery is a set of practices designed to always keep software in a releasable state. It employs the discipline of release engineering to optimise the process of taking changes from source control to production. In this paper, we review the current path to production (build, test and release) of CAM, identify shortcomings and introduce approaches to support further incremental development of the system. By implementing patterns such as deployment pipelines and immutable release candidates we hope to simplify the release process and demonstrate increased throughput of changes, quality and stability in the future, Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

To achieve a tight integration of instrument control and (online) data analysis, the European XFEL decided in 2011 to develop Karabo*, a custom control and data processing system. Karabo provides control via event-driven communication. Signal/slot and request/reply patterns are implemented via a central message broker. Data pipelines for e.g. scientific workflows or detector calibration are implemented as direct TCP/IP connections. The core entities of Karabo are self-describing devices written in C++ or Python. They represent hardware, orchestrate other devices, or provide system services like data logging and configuration storage. To operate Karabo, a Python command line interface and a generic GUI written in PyQt are provided. Control and data widgets compose Karabo scenes that are provided by devices or are manually customized and stored together with device configurations in a central database. Since 2016, Karabo is used to commission and operate the currently three photon beam lines and six scientific instruments at the European XFEL. This contribution summarizes the status of Karabo, highlights achievements and lessons learned, and gives an outlook for future directions., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

EPICS is mature in accelerator community. However, there are endeavors to improve existing control system software like Tango and EPICS 7 mainly driven by the needs of flexibility of the control system and the development of computer technology. This paper presents a new way of building a large experimental device control system using web technology instead of EPICS toolkit. The goal is to improve the interoperability of the control system allowing different component in the control system to talk to each other effortlessly. An abstraction of the control system is made. The control system components are abstracted into resources. The accessing of the resources is done via standard HTTP RESTful web API. HMI is based on HTML and JavaScript in browsers. Web Socket is used for event distribution. The main feature of this design is that all interfaces in the system are based on open web standards, which are interoperable among almost all kinds of devices. The paper also presents a software toolkit to build this kind of control system. A control system for a diagnostic on J-TEXT tokamak built using this toolkit will be presented., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

The Italian National Hadrontherapy Center (CNAO) is a particle treatment and research center equipped with a synchrotron accelerator. The configuration and support environment of CNAO’s control system, originally designed in 2003, is currently being upgraded to incorporate mobile devices. As part of the technological upgrade, a product line architecture has been designed with intent to define application scope, reusability of core assets, and specification of variation points. Implementation and compliance with the product line architecture aims at reducing application’s development time, improving reliability, and aiding medical certification procedures. However, definition and compliance with the architecture comes with considerable overhead development costs. In order to assist the development of new environment applications, a visual wizard has been developed to create customized base applications. This paper presents the challenges encountered and description of the product line architecture for the upgraded configuration and support environment. Alongside, we also describe the Wizard Generator, currently implemented applications, and planned application validation., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

The Vacuum Controls Configurator (vacCC) is an application developed at CERN for the management of large-scale vacuum control systems. The application was developed to facilitate the management of the configuration of the vacuum control system at CERN, the largest vacuum system in operation in the world, with over 15,000 vacuum devices spread over 128 km of vacuum chambers. It allows non-experts in software to easily integrate or modify vacuum devices within the control system via a web browser. It automatically generates configuration data that enables the communication between vacuum devices and the supervision system, the generation of SCADA synoptics, long and short term archiving, and the publishing of vacuum data to external systems. VacCC is a web application built for the cloud, dockerized, and based on a microservice architecture. In this paper, we unveil the application’s main aspects concerning its architecture, data flow, data validation, and generation of configuration for SCADA/PLC., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

Quasar (Quick OPC-UA Server Generation Framework) enables efficient development of OPC-UA servers. The project evolved into a software ecosystem providing complete OPC-UA support for Detector Control Systems. OPC-UA servers can be modeled and generated and profit from tooling to aid development, deployment and maintenance. OPC-UA client libraries can be generated and published to users. Client-server chaining is supported. quasar was used to build OPC-UA servers for different computing platforms including server machines, credit-card computers as well as System-on-a-chip solutions. Quasar generated servers can be integrated as slave modules into other software projects written in higher-level programming languages (such as Python) to provide OPC-UA information exchange. quasar supports quick and efficient integration of OPC-UA servers into a control system based on the WinCC OA SCADA platform. The ecosystem can work with different OPC-UA stacks including 100% free and open-source ones. Thus it’s not restricted by licensing constraints. The contribution will present an overview and the evolution of the ecosystem along with example applications from ATLAS DCS and beyond., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

Control systems require several, core services for work coordination and everyday operation. One such example is Directory Service, which is a central registry of all access points and their physical location in the network. Another example is Authentication Service, which verifies callers identity and issues a signed token, which represents the caller in the distributed communication. Both cases are real life examples of middleware services, which have to be always available and scalable. The paper discusses design decisions and technical background behind these two central services used at CERN. Both services were designed using latest technology standards, namely Spring Boot and REST. Moreover, they had to comply with demanding requirements for fault tolerance and scalability. Therefore, additional extensions were necessary, as distributed in-memory cache (using Apache Infinispan), or Oracle database local mirroring using H₂ database. Additionally, the paper will explain the tradeoffs of different approaches providing high-availability features and lessons learnt from operational usage., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

Since its integration into the CERN industrial controls environment, the SCADA Statistics project has become a valuable asset for controls engineers and hardware experts in their daily monitoring and maintenance tasks. The adoption of the tool outside of the Industrial Controls and Safety Systems group scope is currently being evaluated by ALICE, since they have similar requirements for alarms and value changes monitoring in their experiment. The increasing interest in scaling up the SCADA Statistics project with new customers has motivated the review of the infrastructure deployment, configuration management and service maintenance policies. In this paper we present the modifications we have integrated in order to improve its configuration flexibility, maintainability and reliability. With this improved solution we believe we can propose our solution to a wider scope of customers., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

Conda is an open source package, dependency and environment management system. It runs on Windows, macOS and Linux and can package and distribute software for any language (Python, R, Ruby, C/C++…). It allows one to build a software in a clean and repeatable way. EPICS is made of many different modules that need to be compiled together. Conda makes it easy to define and track dependencies between EPICS base and the different modules (and their versions). Anaconda’s new compilers allow conda to build binaries that can run on any modern linux distribution (x86₆4). Not relying on any specific OS packages removes issues that can arise when upgrading the OS. At ESS, conda packages are built using gitlab-ci and pushed to a local channel on our Artifactory server. Using conda makes it easy for the users to install the EPICS modules they want, where they want (locally on a machine, in a docker container for testing…). All dependencies and requirements are handled by conda. Conda environments make it possible to work on different versions on the same machine without any conflict., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

The Italian National Hadrontherapy Center (CNAO) is a cancer treatment center employing a synchrotron to accelerate charged particle beams. The configuration and support environment of CNAO’s control system is responsible for managing the repository, configuring the control system, as well as performing non-real time support operations. Applications in this environment interface with the relational repository, remote file systems, as well as lower level control system components. As part of the technological upgrade of the configuration and support environment, CNAO plans to integrate mobile applications into the control system. In order to lay the groundwork for the new generation of applications, new communication interfaces had to be designed. To achieve this, a web services approach was taken, with the objective of standardizing access to these resources. In this paper we describe in detail the update of the communication channels. Additionally, several solutions to challenges encountered, such as access management, logging, and interoperability, are presented., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

The Tango HDB++ project is a high performance event-driven archiving system which stores data with micro-second resolution timestamps, using archivers written in C++. HDB++ supports MySQL/MariaDB and Apache Cassandra backends and has been recently extended to support PostgreSQL and TimescaleDB*, a time-series PostgreSQL extension. The PostgreSQL backend has enabled efficient multi-dimensional data storage in a relational database. Time series databases are ideal for archiving and can take advantage of the fact that data inserted do not change. TimescaleDB has pushed the performance of HDB++ to new limits. The paper will present the benchmarking tools that have been developed to compare the performance of different backends and the extension of HDB++ to support TimescaleDB for insertion and extraction. A comparison of the different supported back-ends will be presented., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

Programmable Logic Controllers (PLC) are widely used for industrial automation in industry and at CERN. The reliability of PLC software is crucial, but typically only testing is used to validate it. Our work targets the use of formal verification in practical ways for many years, which showed that it can be beneficial and practically applicable to various PLC programs. In this paper, we present PLCverif, our platform for formal analysis of PLC programs which has largely enhanced the quality of the deployed PLC software. By re-engineering the previous internal prototype tool, we built PLCverif to be an open, extensible platform that can be used not only for CERN’s specific PLC programs. PLCverif is licensed under an open source license, allowing the interested parties to use and extend it., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

A previous paper described an upgrade to EPICS enabling client side tools at LANSCE to receive subscription updates filtered selectively to match a logical configuration of LANSCE beam gates, as specified dynamically by control room application programs. This update paper will examine evolving enhancements enabling Lua-language based data acquisition processing subscription update filters, specified by snippets of Lua-language source-code embedded within the EPICS channel-name’s postfix. We will discuss the generalized utility of this approach across a wide range of data acquisition applications, projects, and platforms; the performance and robustness of our production implementation; and our operational experience with the software at LANSCE., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

A nuclear plant, for energy or for nuclear physics, is a complex facility where high level security is mandatory, both for machines and people. But sometimes the status of danger is not correctly felt, inducing workers to misinterpret situations and, as consequence, not act in the best way. At the same time problems related to area accessibility can occur during normal machine operations, limiting actions related to local maintenance and environment supervision. It would be suitable to have the opportunity to perform these tasks in an independently from environment limitations and machine operations. In order to overcome these limits, we applied Virtual Technology to the nuclear physics context. As consequence, this new tool has given us the chance to reinterpret concepts like training or maintenance planning. In this paper the main proof of concept implemented are described and additional information related to different VR technology usages are exposed., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

The European Spallation Source (ESS) is a neutron research facility based in Sweden that will be in operation in 2023. It is expected to have around 1500 IOCs controlling both the machine and end-station instruments. To manage the IOCs, an application called IOC Factory was developed at ESS. It provides a consistent and centralized approach on how IOCs are configured, generated, browsed and audited. The configuration allows users to select EPICS module versions of interest, and set EPICS environment variables and macros for IOCs. The generation automatically creates IOCs according to configurations. Browsing retrieves information on when, how and why IOCs were generated and by whom. Finally, auditing tracks changes of generated IOCs deployed locally. To achieve these functionalities, the IOC Factory relies on two other applications: the Controls Configuration Database (CCDB) and the ESS EPICS Environment (E3). The first stores information about IOCs, devices controlled by these, and required EPICS modules and snippets, while the second stores snippets needed to generate IOCs (st.cmd files). Combined, these applications enable ESS to successfully manage IOCs with minimum effort., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

During the past decades, a variety of particle optics programs have been applied for accelerator studies at the storage ring facility DELTA. Depending on the application, most programs were used offline without dynamic machine synchronisation. In order to centralize and standardize storage ring modeling capabilities, a dedicated online model server was developed and integrated into the EPICS-based control system. The core server is based on Python/EPICS service modules using OCELOT and COBEA as simulation tools. All data, actual machine readings/settings, conversion coefficients, results of simulation calculations as well as manual parameter settings, are handled via EPICS process variables. Thus, the data are transparently available in the entire control system for further processing or visualisation. To improve maintainability and adaptability, the remote presentation model controller concept was realized in the implementation. The paper explains the setup of the model server and discusses first use cases., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

This paper presents the ELT M1 Local Control Software. M1 is the 39 m primary mirror of the Extremely Large Telescope composed of 798 hexagonal segments. Each segment can be controlled in piston, tip, and tilt, and provides several types of sensor data, totaling 24000 I/O points. The control algorithm, used to dynamically maintain the alignment and the shape of the mirror, is based on three pipelined stages dedicated to collect the sensors’ measurements, compute new references, and apply them to the actuators. Each stage runs at 500 Hz and the network traffic produced by devices and servers is close to 1.2 million UDP packets/s. The reliability of this large number of devices is improved by the introduction of a failure detection isolation and recovery SW component. The paper summarizes the main SW requirements, presents the architecture based on a variation of the estimator/controller/adapter design pattern, and provides details on the implementation technologies, including the SW platform and the application framework. The lessons learned from deploying the SW on CPUs with different NUMA architectures and from the adoption of different testing strategies are also described., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA

The control system of the scientific experimental facility requires heterogeneous control access, domain algorithm, sequence control, monitoring, log, alarm and archiving. We must extract common requirements such as monitoring, control, and data acquisition. Based on the Tango framework, we build typical device components, algorithms, sequence engines, graphical models and data models for scientific experimental facility control systems developed to meet common needs, and are named the Intelligent integrated Control Software Framework of Facilities for Scientific Experiments (iCOFFEE). As a development platform for integrated control system software, iCOFFEE provides a highly flexible architecture, standardized templates, basic functional components and services for control systems that increase flexibility, robustness, scalability and maintainability. This article focuses on the design of the framework, especially the monitoring configuration and control flow design., Proceedings of the 17th International Conference on Accelerator and Large Experimental Physics Control Systems, ICALEPCS2019, New York, NY, USA