Search

Onboard computers are expected to perform more and more processor intensive tasks autonomously. Examples can be found in a broad number of applications: efficient data compression for Earth Observation data, system anomaly and fault detection, data encryption, sensor data fusion, etc. More over, the increasing interest in Artificial intelligence (AI) from the Space Industry has only exacerbated the problem. Vision based navigation in Space, Earth Observation data validation in Space, signal to noise ratio (SNR) reduction are only some examples of the need of AI in Space. The Space Industry, as many other industries, has focused exclusively in the use of standard parallelisation techniques for the acceleration of processor intensive algorithms. Parallelisation tools like OpenMP are used to break down a specific operation within the algorithm, e.g. matrix multiplication, into smaller pieces that are processed using several parallel threads. This technique ensures a low processing time of the algorithm. However, it has limited control of CPU usage and moreover it lacks control of the data throughput. In this work the authors propose a well known paradigm to data processing that has recently been rising renewed interest in Academia: pipelining. Pipelining, as opposed to parallelisation, breaks down an algorithm in steps or operation and assign threads to each of these steps. Pipelining is somehow an an assembly line, where each thread is in charge of one specific piece of the product. Academia research shows significant improvements in the overall performance of the algorithm. Moreover, the authors of this paper made use of a lock-free data processing framework to implement a novel pipeline framework that can enhance onboard processing performance even more. This lead to ground-breaking results in terms of CPU usage and data throughput: more than 20%-60% CPU reduction and 2x - 8x data processing rate increase. Two validation experiments are presented by the authors. First, an algorithms consisting on a number of matrix multiplications to be performed sequentially. And secondly, an AI network for Asteroid pose estimation using data from past Space mission. Both experiments where benchmarked and compared with OpenMP, for the matrix multiplication example, and with TensorFlowLite for the AI algorithm, In both instances, the here presented lock-free pipelined approach substantially outperformed both OpenMP and TensorFlowLite both in terms of CPU and data throughput.

During the OBPDC-2020 conference held last year, we presented the publication “Onboard image processing using AI to reduce data transmission: example of OPS-SAT cloud segmentation”. In this paper, we explained how we implemented three Artificial Neural Networks (ANNs) on OPS-SAT FPGA to perform cloud segmentation based on: - A classical LeNet-5 architecture, - A fully convolutional architecture, - A hybrid convolutional / spiking architecture. Cloud segmentation is a useful onboard service to filter unnecessary data and to preserve the limited storage and bandwidth of nanosats. This service is also compatible with OPS-SAT spatial resolution and the number of logic cells within its Cyclone V FPGA. In the OBPDC-2020 paper, we detailed several challenges we had to tackle to achieve OPS-SAT implementation, specifically: - Dataset engineering, which was made difficult by the fact that no actual OPS-SAT images were available at the time of ANN trainings, - ANN architectures selection, which was almost completely driven by the execution target capability and required to come up with tiny designs, - Hardware acceleration of the trained ANNs, using a VHDL based solution specifically developed to target OPS-SAT FPGA on Cyclone-V System on Chip. In the continuity of the OBPDC-2020 paper, we propose for the OBDP-2021 conference to report the in-flight inferences of our ANNs on FPGA that is probably a world first. We will discuss the different parameters affecting the overall performance measured onboard OPS-SAT, while presenting, at least for one reference ANN, the impact of the different deployment steps on the inference metrics (in full precision on CPU, quantified on the validation board, and in-flight). Then we will propose relevant improvements. We will especially analyze the generalization capability of the trained ANNs on real OPS-SAT images. Since these images display a wide variety of solar irradiance and geometry, we tested different kinds of pre-processing to handle this sensor behavior. We will therefore explain how we used the first images of OPS-SAT to create the new learning dataset. We will also discuss the challenge of ANN quantization to perform inference on FPGA, with potential over or under flow depending on the selected arithmetic, applied not only to weights and biases but also on all the feature maps calculation. We will report the results of the solutions we tested, and we will propose further improvements. The resulting pre and post processing time on HPS, and the inference time on FPGA, will be measured in-flight and compared to the on ground results. Finally, some interesting information will be provided on the process that allowed, in coordination with ESOC team, uploading the full experiment onboard OPS-SAT: codes for the Hard Processor System (HPS) part, and bitstreams for the FPGA. Depending on OPS-SAT availability before the conference, we could also present some improvements we intend to implement. In particular, we want to test an evolutionary algorithm as a complementary or challenging solution of the tiny Artificial Neural Networks.

The use of commercial-off-the-shelf (COTS), cutting-edge processing systems in space applications has received much attention due to an increasingly competitive commercial space sector. Such components would increment the processing capabilities on orbit to unprecedented levels, bringing a great competitive advantage, but assuring reliability under harsh space conditions is a challenge. Single-event effects (SEEs) are a major concern in processors. When using COTS components, available SEE protections are limited and the knowledge about the behavior of the device under radiation is poor. Typically, limited actions can be performed on the hardware to enhance radiation hardness. For that reason, COTS processors usually introduce software-level hardening, by modifying the code to increment robustness, but paying significant performance penalties. Moreover, software hardening can only protect software-accessible resources, but other processor resources may be left unprotected. To achieve fault tolerance, processing systems based on COTS must be designed to implement error detection and recovery capabilities. However, few details are typically available about the internal architecture or implementation of COTS components, and the observability of the processor internal state is usually low. In addition, complex processing systems may present varied failure modes that need to be tackled in different ways, especially when considering different criticality levels. We are presenting a solution to tackle both radiation hardening and testability challenges regarding COTS microprocessors. We have developed a lightweight IP core in HDL that leverages the information available at the trace interface to detect and diagnose errors in microprocessors. The trace interface is a resource commonly found in modern microprocessors to support application development. It provides relevant data about execution with low latency in a non-intrusive manner. When development finishes, trace circuit is no longer in use and could be reused for other purposes. The presented IP can detect errors online with processor execution and obtain error evidence and traceability with low impact on system design and no performance penalty, supporting several development phases: -Design: providing error detection and diagnosis capabilities during development to identify flaws in the system and enhance a given application to meet dependability requirements. -Device evaluation: detecting and classifying errors in different devices, allowing severity evaluation to provide objective criteria on component selection. Not only for COTS but also for RadHard devices, it could help to understand and mitigate complex failure modes. -Operation: working side by side with a microprocessor to check the integrity of the executed application in real time, raise an alert upon error, and provide diagnosis information to perform the necessary corrective action with low latency, achieving fault tolerance. This solution is currently available at ARQUIMEA as an IP core compatible with ARM Cortex-A9 processor. It has been functionally validated in Xilinx Zynq device under radiation testing (TRL3-4) obtaining high error detection rate (up to 99.9%) and useful diagnosis information. The IP features low pin count and parametric design ready to be implemented in any FPGA with low footprint. Currently, efforts are ongoing to enhance IP compatibility with a wider range of technologies and processor cores, including microchip and NANOXPLORE FPGAs.

Computational benchmarking of on-board processing performance for space applications has often been done in a case-to-case basis, taking into account only a small subset of devices and specific, often proprietary, applications, limiting domain coverage and reproducibility. While commercial benchmarks exists for embedded systems, they are usually limited to CPUs and are based on synthetic algorithms non-relevant for space. Consequently, they are not generally suitable for assessing highly parallel processors (GPUs, DSPs, etc.) and/or hardware implementations (i.e. ASICs and FPGAs) which are commonplace in space systems. For on-board processing, there are a number of application types which reoccur over multiple missions. These applications and algorithms are often driving the overall computational requirements of the mission, e.g. in the case of image and radar processing, RF signal processing and compression. In each case, there are certain performance metrics – such as the number of pixels processed per second – which are well-known and easily understandable by designers and users. Finally, with the rise of machine learning applications in on-board space applications, tasks such as image classification and object detection using SVMs and CNNs are becoming commonly used. OBPMark (On-Board Processing Benchmarks) defines a set of benchmarks covering the typical classes of applications commonly found on-board spacecraft. The benchmark suite is publicly available to enable easy comparison of different systems and to quickly down-select possible processing solutions for a mission. It is open source and includes multiple implementations, while it is easily extensible allowing porting and optimization to target platforms, including heterogeneous ones, for fair comparison. Currently, implementations in standard C, OpenMP, OpenCL and CUDA are included. A technical note, defining the algorithms used is also provided to allow implementers to provide additional dedicated versions, including reference inputs and outputs for correctness verification as well as an optional automated launching framework for reproducibility. This also allows the benchmarks to be implemented in FPGAs, while ensuring equivalence with the reference implementations. Five categories of benchmarks are defined 1) Image Processing Pipelines; 2) Standard Compression Algorithms; 3) Standard Encryption Algorithms; 4) Processing Building Blocks; and 5) Machine Learning Inference. In each category, specific benchmarks are included, e.g. both image and radar image compression. Recommended parameters for the CCSDS compression standards 121.0, 122.0 and 123.0 are provided. The processing building blocks include e.g. FIR filters and FFT processing. Two ML applications have been chosen: cloud screening and ship detection. Both will be provided as standard pre-trained machine learning models, both floating point and quantized integer models – to allow support for multiple microarchitectures. The specification of OBPMark has been initiated by ESA together with BSC as an open source project to allow transparent and open performance comparison of devices and systems. The project will also maintain a list of available benchmark results on its open repository. The work has been carried out both internally at ESA, and at BSC through the on-going ESA-funded GPU4S activity, whose optimised versions of algorithmic building blocks implemented in the open source GPU4S Bench benchmarking suite were used as a basis.

Machine Learning applications are finding their ways in demonstration missions like ESA's Φ-sat, which were enabled by the use of COTS solutions and the improvement of tools for ML deployment on radiation-tolerant processing units. The satellite industry has been looking at these developments with great interest. The challenge of implementing such ML applications lies mainly on three points: 1) There are limited processing capabilities on spacecraft hardware, meaning that algorithms need to be optimized for their embedded application. 2) This poses challenges also in terms of tools to be used in the development flow, as classical GPU inference is not possible, and the integration in the industry workflow is complex. 3) The available datasets in terms of openly accessibility and reusability for space missions are limited, as data is either proprietary or poorly labeled. To address these challenges, a benchmark for ML inference applications in space is proposed. Such a method would simplify the comparison of algorithms in early development phases, enabling engineers to define necessary processing power for the desired applications. Moreover, appropriate benchmarking suites will enable the investigation of the software tools, various custom reconfigurable IP designs, and COTS solutions for ML inference for on-board data processing. In the frame of the MLAB project, Airbus, TU Munich, and OroraTech are working on developing an ML inference benchmark based on the commercial MLPerf method. In this work, we specifically focus on the description of this benchmark as the main part of MLAB project, and discuss initial findings and directions with respect to the datasets and tools. The benchmark intends to cover diverse set of algorithms including feature extraction, object detection, classification, tracking, and change detection of different complexity. This ensures that various space use-cases and different computational complexities are represented in benchmarks. The benchmarking suite relies on publicly available large-scale standardized datasets to ensure the reproducibility of results. Specifically, the datasets that are published in recent years, including BigEarth, Kaggle Ship Dataset and EuroSAT, satisfy this requirement and are promising for the benchmarks. In addition, the benchmark is intended to cover various ML inference development and deployment tools. Due to the fact that optimization plays an important part in the final inference performance, tool choice is crucial to meet the requirements with the least amount of design effort. For this reason, inference tools, like FINN, Vitis AI, hls4ml, and more, featuring different levels of HW design, have to be part of the benchmarking procedure. Additionally, this benchmark covers the general performance metrics associated with the payload applications that use ML inference, including accuracy rates, throughput, model complexity, computational complexity, resource utilization, and memory footprint. On the other hand, the special requirements of on-board computation require more careful considerations and implementation of space-specific merits such as energy efficiency, tail-latency bounds, which will also be addressed in this benchmark. The described benchmark will represent a crucial tool for the adoption and deployment of ML-based applications in next-generation on-board data processing for future spacecraft missions.

The interdisciplinary process of space systems engineering poses challenges on the development of its on-board software. On-board software integrates components from different domains and organizations and must fulfill requirements, such as robustness, reliability, real-time capability and memory management. Model-based methods do not only help to simplify the system and give a comprehensive overview, but they also improve productivity by allowing artifacts to be generated from the model automatically. However, general-purpose modeling languages, such as Matlab Simulink, SysML, or UML, are not always adequate because of their ambiguity through their generic nature. Furthermore, sensor data handling, analysis, and processing of data in on-board software requires focus on the system’s data flow and event mechanism. To achieve this, we developed the Tasking Modeling Language (TML), a model-driven software development platform, planned to be released open source. TML incorporates a set of textual and graphical domain-specific languages (DSLs), which allow system engineers to model complex event-driven software systems in a simple way and to generate software from it. The textual DSLs in TML provide a strong-typed syntax to define data types and system components, whereas a graphical DSL enables to design the composition and data flow of the system. Type and consistency checks on this formal level help to reduce errors early on in the engineering process. The TML environment facilitates model-driven software development (MDSD) by incorporating a code generator which is capable of generating executable C++ source-code, unit tests, its build configuration and documentation. This way, the model acts as a single point of truth, and changes on the data flow or integration of new components become trivial. By default, the code generator utilizes the Tasking Framework, an open-source C++ software execution platform developed by DLR. This allows software modules to run concurrently in separate tasks, exchange data between them via channels, and to schedule the task-execution based on events, such as input data availability. TML and the Tasking Framework are based on the task channel paradigm, where tasks are bare executors of algorithms whereas channels are data containers. While TML is focused on data-driven systems, its models are designed to be extended and customized to specific mission requirements. It provides three levels of extension mechanism: firstly, new components, channel types, and parameters can be added to the model dynamically. Secondly, the code generator can be customized by extending or implementing new code templates. This enables to support new execution environments or even new programming languages. Lastly, the generated code is designed to be extended by manually written code. This combination of a modeling language, customized to data-flow oriented systems with a highly extensible artifact generation, enables effective support for development of on-board software. This paper describes the architecture of the TML in detail, explains the base technology, the methodology, and the DSLs. It evaluates the design approach of the software via case study and presents the advantages as well as the faced challenges.

Satellite Earth Observation (EO) is nowadays receiving significant attention. In this regard, the latency of EO products provision to the ground segment is undoubtedly among the first key performance indicators for these systems. Traditionally, small EO satellites rely on the flight segment for raw data acquisition and compression, while the image processing tasks are performed at the ground segment. The latency of raw data transmission prevents such systems from achieving better than Near Real-Time (NRT) delivery of EO products, which are typically available to the end-user after 1h to 3h from acquisition time. The European Union Horizon 2020 EO-ALERT project aims at significantly reducing this latency by moving all the critical processing tasks on the flight segment and accelerating them using high-performance commercial off-the-shelf (COTS) devices. The resulting architecture minimizes the amount of transmitted data and eliminates ground-based data processing from the EO data chain, hence achieving actual real-time product delivery in less than 5min with optical and Synthetic Aperture Radar (SAR) data. The centerpiece of the proposed architecture is the embedded CPU Scheduling, Compression, Encryption, and Data Handling (CS-CEDH) Subsystem, essentially fulfilling two roles: 1) acquire and move images and products among the image processing and communications subsystems, therefore also coordinating their tasks; 2) compress and encrypt the input and output data with different settings depending on the mission requirements. From an optimal design and resource allocation perspective, these aspects are complementary: while the former is software-focused, aiming at maximizing modularity, flexibility, and dynamic scalability, required by the inherent system-level real-time event-driven nature of the CPU Scheduling processes, the latter represents an intrinsically highly-specialized, computationally expensive data-processing function, better suited for hardware implementation. In order to achieve the overall goal of minimizing the system-level latency, it is therefore mandatory to effectively co-design a mixed hardware/software solution, leveraging the performance of state-of-the-art COTS Multi-Processor System-on-Chip (MPSoC) devices featuring a Processing System (PS) directly interfaced to a Programmable Logic (PL) unit. Such platform enables, thanks to state-of-the-art Electronic Design Automation (EDA) tools, to obtain a Register-Transfer Level (RTL)model (to be deployed on the PL unit) directly from a high-level hardware-friendly software model through High-Level Synthesis (HLS), thereby effectively shifting from a software-only design to a more efficient and high-performance hybrid hardware/software one, without sacrificing run-time tuning of compression and encryption parameters and still meeting all the system-level requirements. The PS hosts the scheduling and data handling software and seamlessly off-loads data-intensive tasks to the PL, allowing to dedicate most of the CPU resources to compress, encrypt and transmit small, high-priority EO products (alerts) with very low latency while also processing larger data (e.g., generated images) with high-throughput. Here we present the first promising results of this design methodology applied to the CS-CEDH Subsystem of the EO-ALERT architecture. In particular, its contribution to the alert provision latency is under 1s in every foreseen application scenario. Simultaneously, the hardware compression/encryption accelerator enables a 6 to 7-fold speed-up compared to a software-optimized implementation when also compressing, encrypting, and transmitting the processed images.

Processing requirements are exponentially increasing to keep pace with the data volumes generated by increasingly “Big Data” sensors. These requirements are compounded when factoring in the data movements planned for future spacecraft constellation mesh networks, i.e. connected spacecraft infrastructures for on-orbit fleet management, autonomous sensor fusion, data storage, very low latency actionable information generation, and real-time communication. Unibap AB and Troxel Aerospace Industries, Inc. have worked together to develop a heterogeneous radiation-tolerant onboard cloud computing hardware platform bringing terrestrial Internet-of-Things edge processing to space, e.g. Infrastructure as a Service, Big Data analytics and Artificial Intelligence. This platform is part of Unibap’s SpaceCloud ecosystem, which makes virtual servers and other resources dynamically available to customers. Leveraging its powerful heterogeneous hardware platform, the SpaceCloud framework has been developed with support by the European Space Agency (ESA) to enable rapid and flexible application development using containerized and isolated virtualization either for execution locally or on networked spacecraft. SpaceCloud allows exchange of information that is transparent between local or networked nodes to facilitate cooperation using a distributed mesh network communication architecture. A node is any entity or subfunction, such as an application, or a vehicle, ground control station, sensor read-out module, cloud detection application, data base indexing etc., that is connected to the mesh network. Data exchanges may include intra-data processing between different software apps in a pipeline, or telemetry from on-orbit robotics-based nodes, commands from ground control nodes, science data fusion, etc. By exchanging pertinent data, nodes can act together to perform a task autonomously without requiring direct control or intervention by a central control node. The focus of SpaceCloud is to enable commercial software to be reused onboard to decrease the overall cost and development time to deploy new capabilities on compatible space assets. As an example, SaraniaSat and Unibap have worked with L3Harris Geospatial to enable the geospatial intelligence software suite ENVI®/IDL® on SpaceCloud. A very low-latency onboard SpaceCloud application for detecting aircraft using ENVI®/IDL® and machine learning within 100 sq. km multispectral satellite imagery has been successfully developed and demonstrated. The SpaceCloud framework executes on the iX5 and iX10 families of x86 radiation tolerant computer solutions featuring AMD multi-core CPU, GPU, Microsemi FPGA, and Intel Movidius Myriad X VPU accelerator and local high-speed solid-state storage. Radiation testing in the US has shown very promising results on both 28 nm and 14 nm processor nodes with high tolerance for single event latch-up (SEL) and total ionizing dose (TID). To improve radiation tolerance, the SpaceCloud framework performs real-time software monitoring and FDIR through the SafetyChip feature working in tandem on the x86 software stack and an RTOS in a fault-tolerant configuration in FPGA. The concept has been developed with funding support from ESA. Radiation tolerance can be further increased by use of a single event upset (SEU) mitigating middleware that protects CPU and GPU processing. This paper presents the SpaceCloud In-orbit Demonstration compute architecture and framework configuration as implemented in D-Orbit’s Wild Ride ION SCV mission due for launch in Q2 2021.

The increasing demand for onboard calculation capability produces the necessity for high performance processing systems onboard nano to small satellites. Modern NewSpace applications especially signal processing, image processing, artificial intelligence require an amount of processing which is not feasible with currently available space grade solutions. In this paper, Thales Alenia Space in Germany is presenting its answer to this need: the multiMIND processing system - a highly flexible, multi-mission solution with modular software framework. The development and demonstration are supported by the European Space Agency in the frame ofthe ARTES Competitiveness & Growth activity “multiMIND on EIVE”. Using latest COTS Xilinx Zynq Ultrascale+ MPSoC (multi-processor system on chip) effectively combines large FPGA with multi-core ARM processors and delivers the required performance power, whereas robustness against radiation effects is ensured by a specific radhard island circuitry. This radhard circuit serves as a supervisor for the radiation sensitive COTS parts as well as reliable interface to other satellite subsystems. This design is completed by using other elements like Linux, software components and COTS FPGA IP cores allowing for fast development in demonstration mission as well as cost reduction in next generation missions. The generic core is interfaced with mission specific data acquisition boards, such as RF frontend and digitizers, RF agile processors or optical sensors using standard, high speed lanes exceeding a total data rate of 100 Gbit/s. Internal high speed communication allows integration of plug-in accelerators and dedicated companion boards as well. Dedicated interface boards even allow to fly COTS FMC boards with minimum adaptations. Since the supervisor section manages all space-specific challenges, the user can widely use terrestrial and COTS proven routines, accelerating the design cycle and providing a fast “highway to space” for his designs. Radiation data and characterization – especially for latest edge COTS chips – are not available and these components require active SEE mitigation. The supervisor performs autonomous latch-up monitoring and clearing, including smart current analysis to catch lower current-step or microlatching events. Relevant telemetry is monitored, analysed and provided to the satellite platform. The supervisor also handles controlled initialization, reboot or reset if necessary and informs the platform accordingly. It buffers and manages the TM/TC flow and can update the MPSoC firmware and software autonomously. The first application of the multiMIND framework is the E/W-band demonstration mission EIVE (Exploratory In-Orbit Verification of an E/W-Band Satellite Communication Link), where camera data handling and high speed data downlink will be demonstrated; it is scheduled to fly in 2022. This paper will also show how the multiMIND framework can be adapted for modern RF SDR, signal / spectrum intelligence, radar or optical applications.

Efficient data and signal processing is a key enabler in several space-related application areas. Within the last years complex applications like machine-learning-based earth observation, satellite-based quantum communications or 5G non-terrestrial networks have become very popular, but require a significant increase of the currently available processing resources. On the other hand, also a strong demand for an effective reduction of the production costs is recognizable. A promising idea to achieve these challenging goals is to go for a distinctive standardization of space components and interfaces. By this measure the scalability and re-usability of existing designs can be significantly improved, which are key parameters for the cost-efficient production and deployment of novel space hardware. To compete with these challenging goals, we present DSI's high-performance data processing unit (HPDPU): a novel powerful PDHU for the efficient data processing of next generation space applications, designed as a compact (form factor 6U) but also lightweight (~ 800g) board. Core of this platform is the novel Xilinx Kintex Ultrascale FPGA device that is available as a commercial (XCKU060) and a radiation hard (XQRCU060) variant and provides an extremely high number of device resources, e.g. logic cells, DSP slices, block RAM, etc. at reasonable energy costs. For high-speed serial data transmission, 32 gigabit transceiver channels (GTH) are available. In order to achieve the desired re-usability, the cPCI Serial Space (cPCI-SS) backplane connector is considered which enables the deployment of a modular computer system. Typically, cPCI-SS is designed to manage two system boards, seven peripheral boards, a power supply and a shelf controller board via the backplane. For high-performance data transfer and configuration, the HPDPU provides 2 PCIe, 2 Ethernet and 2 SpaceWire interfaces. For further functionality and modularity increase, the HPDPU also possesses FMC-based mezzanine connectors. By this measure, the same basic platform can be adapted to numerous different applications simply by adding an application-specific extension card to the HPDPU motherboard. For supervision of the power supply and FPGA behavior, a radiation tolerant supervisory system monitor IC is considered, providing essential reset, watchdog as well as power-up/power-down functionality. In order to guarantee correct boot loading and (re-)configuration, a sophisticated radiation hardened system controller is implemented. In addition, a 256 MiByte NOR-Flash-based configuration memory for periodic scrubbing is taken into account, running in TMR. For the storage of user data or payload, 8 GiBit DDR3 SDRAM with a 16 bit data bus is provided. To achieve high reliability, a Reed-Solomon Code RS(12,8) ECC block is considered that is capable to detect and correct symbol errors on the data bus.

In this paper we present an ad-hoc architecture for on-board Deep Learning (DL) network implemented onto rad-tol FPGA, creating building blocks re-usable for different AI solutions or networks to be developed in the future. The problem analysed is based on autonomous descent and landing scenario, trying to compare traditional techniques. The implementation of the Deep Learning algorithm is focused on extraction of features in navigation camera images. The solution in FPGA allows a reduced power consumption while maximizing the execution performance, opposite to some, or many, on-ground solutions. A space representative breadboard is prepared to demonstrate the solution. For training/testing/validating of the Deep Learning (DL) it has been selected the North Pole of the Moon surface, where one trajectory is used to train the DL and a second one to validate the DL. The architecture of the neural network is divided in numerous layers and it is based on Processing Units (PU), where on layer can have multiple PU. Each PU has the possibility to perform operations such as Convolution, MaxPooling and Upsampling. The implementation is compose by a set of PUs, three DSPs and a controller. The controller is responsible of the coordination of reading and writing commands into the external memory, as well as deciding which operation to execute. Whereas lower address of the memory is allocated for storage the parameters necessary for the operations (weights and biases), once the FPGA is initialized, the following memory addresses is reserved for the input images. Moreover, each layer has a fraction of the RAM reserved for its. Autonomous Navigation Based is a complex DL implementation and presents a lot of challenges, but two of them are of outmost importance: FPGA resources and timing performances. Both terms are completely correlated and the design is tailored in order to balance the bottleneck, the output performance based on timing requirements and the number of accesses to external memory. This architecture requires of a huge amount of arithmetical resources and there are not enough resources available, such as DSPs and BRAMs that help performing all operations in a row, not even in the biggest space grade FPGA on the market. The liaison between resources and timing is to find the balance between both. The architecture based on PUs tries to minimize the latency of the operations, especially for the Convolution module, while at the same time minimizing resources. The more PUs, less accesses to external memory are needed. Overall, this paper presents an analysis on the performance of a DL, where visual based navigation algorithms are implemented on a FPGA hardware. The main goal of this activity is to reduce the computational load of on-board processors, starting from architecture, resources and timing analysis between SW and HW implementation.

CubeSat missions, in most cases, utilize commercial off-the-shelf (COTS) components. The COTS components are vulnerable to radiation effects such as single event effects (SEE) or total ionizing dose damage (TID). Those effects decrease overall system reliability and can lead to permanent damage to components. One of the methods of mitigating the risk is system-level hardening. The most commonly used hardening techniques are fault detection, isolation, and recovery (FDIR) mechanisms implemented in a ruggedized controller that controls state-of-the-art systems on a chip (SoCs), error correction coded (ECC) or triple modular redundant (TMR) memories, and Configuration RAM (CRAM) scrubbing in the SoCs. In some cases, partial or full redundancy of selected components is implemented. These techniques were reviewed then selected techniques were implemented in the KP Labs' Leopard data processing unit (DPU). The proposed solutions may be re-used in other missions to fulfill mission reliability, availability, and safety levels. The Leopard data processing unit (DPU) is a part of KP Labs' Intuition-1 mission scheduled to be launched in 2023. Intuition-1 will be a 6U-class CubeSat, and it will utilize a specialized hyperspectral camera with spectral resolution in the range of 470-900 nm with 150 spectral bands. The primary purpose of this mission is to technologically demonstrate the reduction of the spatial resolution of hyperspectral images (HSI), hyperspectral band selection, and segmentation of HSI with a neural network-based in-orbit processing hardware that is the Leopard DPU. Implementation of algorithms on-board the satellite will allow to quickly decimate data, reducing the amount of radio air time required to download all the data to Earth. The Leopard DPU consists of two redundant processing nodes controlled by a shared supervisor. Both elements are built using COTS components. Each processing node utilizes a state-of-the-art Xilinx Zynq Ultrascale+ MPSoC, 16 GB of DDR4 memory with ECC, 4 GB of NAND flash memory, and two 256 GB solid-state drives (SSD). Leopard DPU utilizes Xilinx's Vitis AI development environment platform to support mainstream AI networks such as TensorFlow and Caffe. Zynq's bootloaders and basic Linux image are located on a TMRed QSPI NOR flash memory, placed on a supervisor board. Moreover, the supervisor's board implements basic safety features for the two processing nodes, selects Linux images to be loaded, and multiplexes platform interfaces. DPUs are powered by highly integrated power management integrated circuits (PMICs) surrounded by multiple sensors. The supervisor monitors currents, voltages, and temperatures to implement FDIR techniques for detecting single event latch-ups and high current events. Single event upsets (SEUs) are corrected on many levels. Zynq's configuration RAM (CRAM) is scrubbed by a soft error mitigation module. DDR4 utilizes an ECC mechanism that detects and corrects SEU errors that occurred in memory. The supervisor board is composed of a radiation-hardened Vorago's Cortex-M0 microcontroller and accompanying ProASIC3 FPGA. The main tasks of the supervisor are: controlling DPUs, multiplexing platform interfaces such as high-speed X-Band and S-Band links, and routing CubeSat space protocol (CSP) packets.

EO-ALERT (http://eo-alert-h2020.eu/) is a European Commission H2020 project coordinated by DEIMOS Space, whose main objective is to obtain Earth Observation products with very low latency ( In order to program this COTS device from a global perspective in high level languages, the platform concept has been used. It is based on the block design of the FPGA and the custom Linux created with the Petalinux tool. Once the platform is ready, the new tools for programming Xilinx® COTS allow using all board resources such as IP cores in the FPGA for hardware acceleration or the use of parallel processing frameworks like OpenMP to fork the data between the 4 available cores to increase the performance. To process the sensor images and obtain the target products, artificial intelligence (AI) and machine learning (ML) algorithms developed in the project are implemented by migrating them to the target Hardware, taking into account the most efficient implementation on the system multi-core (4 A53 ARM® cores) or FPGA. The development includes the use of rapid prototyping tools to produce optimized hardware IP blocks and SW libraries. In the implementation and verification phase, a PIL (Processor-In-the-Loop) testbench is created. The objective of the test bench is to have a multi-board breadboard representative of the final architecture, in which to validate the design and its performance. The execution times are measured in the PIL platform, obtaining at the present moment results within the requirements ( For that, the HW architecture allows a master-slave configuration, to be used for this on-board processing, thus reducing the total latency of the product generation or cover more area. On each board the processing takes advantage of the board’s resources (multi-core, FPGA) to obtain the processed products and alerts, which are centralised in the master processing board. In addition, the results are highly significative due to the evaluation of the resulting processing system is performed experimentally using real Earth Observation data from a reference-image database, corresponding to the DEIMOS-2 VHR optical satellite and the multispectral SEVIRI instrument on the MSG satellite. Ground truth information from multiple sources is used in the verification phase. Furthermore, the performance of the initial algorithms is compared with those obtained in the Hardware, thus obtaining an evaluation of the effect of the Hardware implementation. © 2021 EO-ALERT All Rights Reserved

The open-source real-time OS RTEMS has become quite popular in the Space Domain. Particularly with its Symmetric Multi Processing (SMP) option it may provide a very efficient use of multicore processors. In order to permit its application in more critical space missions, attempts have been made for a qualification according to ECSS standards. As an innovative approach the qualification is organised in form of an open-source approach as well. Funded by ESA a consortium with different contractors is preparing a qualification toolkit that shall be made available to the public. This qualification toolkit will provide tool-chain and test-suite for a mostly automated generation of qualification documents on level C and D, together with a basic set of interfaces for the most common system on chips used in the space domain. A later independent verification and validation is anticipated to obtain a level B qualification. Compared to proprietary software this approach shall lower the required investments and reduce the dependency to sole software providers. The presentation will provide an overview of the available software resources as well as the test equipment. It will further explain how to use those resources in order to prepare the qualification of a set application software and provide an outlook on expected future extensions and additional resources. A very special aspect is the open-source approach for qualification. After the initial steps have been supported by ESA, a continuation and extension of the database has to be organised in an open, self-organised process. Such a process will need to be established in order to sustain the use of the qualification toolkit in future environments without extensive re-developments.

The growing interest in on-board data processing for Earth observation satellites has been driven by both the needs of ground-based applications of satellite data and the increasing challenges borne by the latest on-board instrument technologies. On the application side, end users of satellite data are seeking methods of acquiring more timely data and lower latencies, supplied in a form that is immediately useful to them. On the technology side, instrument manufacturers are developing payloads with higher spatial and spectral resolutions and greatly increased acquisition rates. Such rates far exceed typical current and near-term downlink bandwidths, resulting in severe data bottlenecks. Addressing the application needs leads to solutions which also address the challenges of adopting these new instruments. On-board processing activities which can extract information from data, reduce the data and prioritise useful information can not only deliver data to end users faster and in a more useful form, they can also filter and intelligently compress data to reduce or eliminate the downlink bottleneck. These on-board activities are many and variable, depending on the on-board functionality desired and the end user requirements to meet. In this paper, generic on-board applications are presented which may be configured to target groupings of use cases and provide measurable end benefits to mission stakeholders. These use cases have been solicited from a comprehensive survey and a workshop and one-to-one engagements with remote sensing end users. These applications are underpinned by enabling technologies which leverage the state-of-the-art in embedded processing and AI. These technologies include machine learning algorithms (both traditional and deep learning), training datasets, processing architectures and embedded computing devices. The paper summarises the algorithms which can be used to implement the on-board applications and the datasets which can be used to train and test these algorithms. Finally, a summary of the requirements and further implications of adopting these technologies to implement the proposed applications is presented. This leads to several impact areas and corresponding needs that must be addressed in the design of future on-board processing systems.

The design of a device to support data processing on a satellite must provide good radiation tolerance, low power consumption, the ability to interface with a range of different data sources, and high throughput digital signal and neural network processing. It should also be compatible with the main software development platforms, in particular those used in artificial intelligence. The new hisaor system-on-chip from O.C.E. Technology addresses these requirements, balancing them in a way that offers an effective solution across a wide range of potential applications, in particular those that use neural network processing. Our paper will describe and explain the choices made in arriving at the hisaor design and provide an analysis of its performance. The hisaor design includes an Artificial Intelligence unit with 8 GPUs and 8 Neural Network processors sharing a common cache. The main system processor is a quad core ARM Cortex-A9, each core with 32 KiB L1 instruction and data caches, a 512 KiB Level 2 cache is shared by all four cores. High band-width on-chip memory includes 1 MiB SRAM and 2 x 32 KiB boot ROMs, and a range of external memory types can be connected. ECC is supported. At its 1 GHz maximum clock frequency hisaor can provide 64 GFLOPS and for fixed point calculations 12 TOPS. Maximum power consumption is 6 Watts. As it is expected that many applications will involve processing images, hisaor includes a range of media processors, including H264/H265/JPEG2000 encoders/decoders. Camera Link and MIPI interfaces are provided, other interfaces include 1553B, CAN, BT1120, SPI, USB, Gigabit Ethernet, and others. hisaor is compatible with OpenCV, OpenVX and OpenCL. Its neural network processing units support convolutional and other approaches to neural network processing and are compatible with the standard approaches to machine learning and neural network processing, including the Caffe framework and TensorFlow. As well as describing the reasoning behind the design of hisaor our paper will present performance metrics based on a number of applications and outline possible future developments in the design.

Cobham Gaisler and RUAG Space are developing a high-performance Single-Board Computer (SBC). The SBC provides substantial processing capability along with an extensive set of memories and redundant interfaces to support the needs of current and future OBC, data handling platforms, and payload data processing. The SBC is developed following the CompactPCI Serial Space backplane standard (CPCI-S.1 R1.0). The processing capability for the SBC is provided by the GR740 Quad-Core 32-bit LEON4FT SPARC V8 processor along with the Microsemi RTG4 radiation tolerant FPGA. The RTG4 brings the DDR2 memory and high-speed serial link (HSSL) capability to the SBC. The glue-logic required to be compatible with the CPCI-S.1 R1.0 standard are also implemented in the FPGA. More than 70 % of the LUT/Flip-flops and 100 % of the math blocks are still available in the FPGA for application specific developments and implementation of accelerator functions. With the available processing capability, the following applications are identified as potential use cases for the SBC: • Image processing and selection of valuable data for earth observation or similar missions • Visual navigation for critical docking and lander missions, e.g., debris removal, satellite life-extension and lunar and mars missions • Object identification tracking for surveillance missions • Centralized payload control and data handling for multiple instruments The SBC is accompanied by boot software (GRBOOT) and device drivers. The GRBOOT boot software is responsible for taking the GR740 from system reset state to the execution of multi-processor mission application software. The boot SW consists of three parts, multi-processor initialization, processor self-tests, standby mode and application loader. The implementation of the GRBOOT represents a tailoring of the ESA SAVOIR-GS-002 specification "Flight computer initialization sequence". A high-quality implementation of drivers for: GR740 CAN (GRCAN), PCI host (GRPCI2) and SPI (SPICTRL) controllers are available for the GR740 SBC application developer. The drivers are compatible with RTEMS-5 SMP.

Satellite-based applications produce ever-increasing quantities of data, challenging the capabilities of existing telemetry and on-board processing systems, especially when results must be transmitted quickly to ground. The Scalable On-Board Computing for Space Avionics (ScOSA) platform contributes the processing capability necessary to perform such computationally intensive analysis on-board. This platform offers a high-performance on-board computer by combining multiple commercial off-the-shelf processors and space-grade processors into a distributed computer. Middleware ensures reliability by detecting and mitigating faults, while allowing applications to effectively use multiple, distributed processors. The current work aims to demonstrate the use and advantages of utilizing the data-flow programming paradigm supported by the ScOSA platform to provide high-throughput on-board analysis. This enables rapid analysis even for applications requiring high frame rates, high resolutions, multi-spectral imaging or in-depth processing. The On-Board Data Analysis and Real-Time Information System (ODARIS) is used to demonstrate this method. ODARIS is a system for providing low-latency access to satellite-based observations, even when large quantities of sensor data are involved. By performing on-board processing of the data from the satellite-borne instruments, the amount of data which must be sent to ground is drastically reduced. This allows the use of low-latency telecommunication-satellite constellations for communicating with ground to achieve query-response times of only a few minutes. The current application combines an Earth-observation camera with AI-based image processing to provide real-time object detection. In the data-flow driven implementation of ODARIS on the ScOSA platform, images are captured by a camera and sent to any of several processors for the computationally intensive image processing. This allows multiple images to be processed in parallel by as many processors as are available, while avoiding the need to divide each image across several processors. The results are transferred to an on-board database from which queries can be served asynchronously. The system will be tested in configurations with one, two and three processors and the resulting image throughput presented. Testing is performed on a ground-based prototype system using pre-recorded images. This paper presents the necessary details of the underlying ScOSA and ODARIS systems as well as the implementation of the objection-detection algorithm using a parallelized, data-flow model. The results of executing the system using a variable number of processors are presented to demonstrate the improvement in image throughput and its potential application to other computationally-intensive tasks.

The proliferation of demanding workloads in on-board systems for space applications, such as Vision-Based Navigation (VBN), has led to a new era of embedded on-board processing. To enable new applications and reconfigurable high-performance computing within a restricted power envelope, the space industry is examining alternative platforms/technologies. Among the existing embedded platforms, the FPGAs have gained increased popularity due to their attractive performance per power ratio, and thus, they are already being considered for future space missions either as main accelerators or framing processors. In this context, the new European space-grade family of FPGAs, named BRAVE and provided by NanoXplore, is expected to play a key role owing to its radiation-hardness, high density, and reconfiguration features, as well as its software tools providing end-to-end FPGA development and seamless chip configuration. The BRAVE family of FPGAs constitutes an additional promising solution in the current limited pool of space-grade FPGAs. Most of these FPGAs are inferior to their Commercial Off-The-Shelf (COTS) counterparts either in terms of performance or resources availability. NanoXplore provides various BRAVE FPGAs ranging from low-end to high-end, i.e., NG-MEDIUM (65nm), NG-LARGE (65nm), and NG-ULTRA (28nm), which are Radiation-Hardened By Design (RHBD) and incorporate the traditional FPGA programmable logic resources (LUTs, DFF, DSPs, RAMBs, etc.). NG-LARGE and NG-ULTRA include ARM processors, with the latter implementing a full SoC. Moreover, the BRAVE family provides features essential for embedding computing in space, such as the SpaceWire interface for fast I/O and chip configuration, and memory scrubbing to ensure the continuous correct functionality. In this paper, we evaluate the NG-LARGE FPGA by (i) assessing the development and configuration tools, i.e., NXmap3 and NxBase2, respectively, (ii) assessing the hardware components (board+chip), and (iii) doing high-performance DSP benchmarking with realistic workloads of space applications. Our work is associated with the QUEENS-FPGA and QUEENS2 activities of ESA and aims to examine the maturity of the software tools and the chip’s performance, targeting its future usage as on-board processor. To evaluate the capabilities of such a new device, we develop a quality assessment methodology, which is based on systematic and disciplined testing at different stages (Synthesis, Placement, Routing, HW Execution). The methodology includes comparisons with the predecessor NG-MEDIUM and other competitor FPGAs. The benchmarking involves VHDL kernels from the computer vision and signal processing domains, representing the performance requirements in current and future spacecraft/rovers. Overall, the contribution of this work is twofold: (i) the presentation of a methodology for evaluating new devices/tools throughout the entire development & execution process, and (ii) the demonstration of the NG-LARGE capabilities as on-board processor. Preliminary results show that NG-LARGE provides similar resource utilization to the competitors, and in some cases even better (e.g, in RAMBs). In terms of performance, NG-LARGE offers sufficient throughput, outperforming the space-grade CPUs. Specifically, the throughput is 130 Megasamples/sec for signal filtering, 5-10 Frames/sec for feature detection on Megapixel images, and ~7 sec per high-definition image depth extraction.

In order to improve meteorological models, measurement of the atmospheric profile for water vapours is an important parameter. Space based radiometers measuring the H2O absorption band at 183 GHz can be used to measure this atmospheric profile. SAPHIR is a radiometer instrument aboard the Megha-Tropiques spacecraft measuring the H2O absorption band at 183 GHz using 6 sub-bands from 200 to 2000 MHz. For SAPHIR-NG, a hyperspectral radiometer is considered in order to measure the absorption band using at least 256 channels over more than 10 GHz. Standard solutions based on analog filters are not viable in such a configuration, so a digital solution is required. This CNES R&T study aims to develop a demonstrator of the digital backend of this hyperspectral radiometer based on FPGA and high speed ADC. Based on a previous study, an optimum is proposed for 256 channels over 10 GHz of bandwidth with a quantization between 4 to 6 bits at the ADC output. The on-board digital signal processing is a filter bank followed by a power detector. A simulation of the impact of the FIR filter length and the filter window shape (rectangular, Hamming, Blackman ...) on the measurement quality (radiometric sensitivity) is performed. The filter bank implementation in FPGA and ASIC is based on the optimised polyphase filter architecture, using a Fast Fourier Transform to efficiently channelize the signal. In order to support the 10 GHz bandwidth, a highly parallel FFT is required. A preliminary on-board data processing feasibility analysis demonstrates that 64-samples per cycle FFT-256 working at 195.3 MHz can support 12.5 complex Gsamples/s processing for a polyphase filter with 768 taps window (i.e. 3 taps per channel). For this preliminary feasibility, 12-bit quantification is used with standard FFT scaling. Further studies will focus on optimizing the FFT scaling to reduce the required quantization. In the frame of the demonstrator architecture definition, detailed trade-off between commercial and radiation tolerant FPGAs (Zynq Ultrascale+ RFSOC, Space Grade Kintex ultrascale, Versal AI Core and NG-ULTRA) and giga-sample ADC (EV12AQ600, ADC12DJ3200QML and ADC12DJ5200RF) are performed to select best candidates for the demonstrator. In parallel of the FPGA based demonstrator, study of an ASIC based solution is ongoing in order to reduce the estimated 10 W consumption for the 10GHz bandwidth acquisition and processing FPGA based solution to approximately 3 W.

Recently, there is an increasing interest in artificial intelligence(AI) and machine learning(ML) in space for on-board processing [1], as indicated by latest missions. Perseverance uses AI for navigation, however the limited processing capabilities of existing space processors permit only small navigation distances and speed. Therefore, COTS accelerators are explored such as Intel Movidius in ESA’s Φ-1 mission, where AI is used for detecting clouds in satellite earth images. However, due to the low radiation tolerance of COTS accelerators, these solutions cannot be used beyond LEO or for long term missions such as institutional ones. Moreover, COTS software stacks used in accelerators which frequently depend on non-space qualified OSes such as linux. Therefore, ML acceleration features are needed in qualified space processors and FPGAs. We provide two such solutions, one for increasing the AI performance of space processors using a low-cost vector unit co-designed for AI, and another one implementing state-of-the-art low-cost binarised neural networks(BNN) on FPGAs, using ESA’s TASTE framework for a reliable software stack. Both solutions are implemented in VHDL and are open-source. Our vector unit is portable and has been designed for both LEON3 and NOEL-V space processors. It supports 8-bit operations with optional saturation and reduction operations, and reuses the existing register file, which guarantees minimum overhead and backwards compatibility. The integration of the module with the smallest LEON3 implementation, according to Vivado shows an area increase of only 30% which is minimal compared to other vector units for microcontrollers [2]. The impact in the processor frequency is minimal, achieving 90MHz compared to 100 MHz of the baseline LEON3. Preliminary results using matrix multiplication, a universal ML building block used for the implementation of fully connected and convolutional layers, indicate speedups of up to 3.8x. Currently we are working on compiler support to enable evaluation with more relevant ML cases. We expect to have these results by the workshop date, for presentation. For the BNN accelerator, we use the TASTE model-based framework to generate a correct-by-construction software communication driver on the CPU, as well as the VHDL communication part of the accelerator, based on the ASN.1 description of the data we want to run inference on. The implementation of the BNN logic is performed manually in VHDL with a reusable, structured design for fully connected BNN layers. Thanks to the BNN properties, such operations are ideal for FPGAs, since multiply-and-accumulate operations are reduced to XNOR and bit-counting. The layer weights are stored within the FPGA block ram, so they are re-used across runs. Synthesis results report an achieved frequency of 114.943MHz. Preliminary simulation results with a fully connected 512x512 layer show performance benefits between one and two orders of magnitude compared to LEON3. Both designs are currently ported to Xilinx FPGAs to obtain higher confidence results, which will be presented in the workshop. [1] Jan-Gerd Me et al. Techniques of Artificial Intelligence for Space Applications - A Survey. In OBDP 2019. [2] M. Johns et al. A Minimal RISC-V Vector Processor for Embedded Systems. In FDL, 2020.

The use of Artificial Intelligence (AI) and Machine Learning (ML) in space missions has become a popular approach to make spacecrafts act more autonomously. Increasing autonomy is required since communication bandwidth and the availability of ground stations is limited. It is recognized that increasing autonomy may also yield detection of science opportunities and also increases reliability whereas the operational effort can be reduced. Common applications of AI and ML in space missions relate to anomaly detection, fault detection, isolation and recovery (FDIR), pose estimation and trajectory design, among others. Many approaches rely on Deep Neural Networks (DNN) due to their achievements in the past years in several application domains that formerly required human intervention. The basic mathematical operation in DNNs is the matrix multiplication. Weight matrices are consecutively applied to the input data, in order to produce a prediction. DNNs may have multiple thousands or even millions of weights, which are adapted within the training phase, such that the model is suitable for its task. Especially the training of DNNs is computationally intensive because the amount of training data required increases with the size of the network. However, computational power and memory space of embedded systems as used in space missions are limited. To circumvent this and in order to verify the trained models as far as possible, ML models are usually trained and validated on ground and deployed to the embedded system afterwards. However, there exists no standardized interface that enables to load ML models on embedded systems on space systems. Our approach aims to deploy arbitrary ML models including neural networks to space missions using the well-established packet utilization standard (PUS, cf. ECSS-E-ST-70-41C). Therefore, we extended our Open modUlar sofTware PlatfOrm for SpacecrafT (OUTPOST, available as Open Source software) with a custom PUS service for dynamically loading and executing trained and validated ML models generated by TensorFlow Lite. TensorFlow Lite is an extension of the widespread ML platform TensorFlow developed by Google. It enables to deploy TensorFlow models to embedded systems and microcontrollers by encoding and decoding them into a serialized representation of the model. Through our custom PUS service, models can be uploaded to and removed from the space system as well as updated by other versions of the model. Furthermore, the service enables the execution of models in an event-triggered fashion and makes the prediction results accessible for other components of the flight software through the PUS model. These can now make use of our custom PUS service to increase autonomy when handling their tasks. Conceivable tasks range from data-driven monitoring of the system’s health status to autonomously controlling essential parts of the space system in the foreseeable future.

Next generation space missions will require more capable computers in order to implement either advanced navigation and control algorithms needed to increase the spacecraft autonomy and agility or on the payload side with complex scientific payload data pre-processing algorithms. There is therefore a high interest for building efficient and disruptive on-board computing for future applications, based on the exploitation of multicore and manycores to increase on-board processing capability while sustaining flexibility through the use of software. The purpose of this study was to demonstrate the benefits of using one of the most well-known parallel OpenMP programming model for the development of parallel space applications, in terms of performance, programmability and portability. Two main goals were identified: • Improve overall system performance by exploiting the most advanced parallel embedded architectures targeting the space domain • Improve the parallel programming productivity by reducing the initial development efforts of systems based on parallel architectures, During the project two representative image processing use cases were selected and successfully ported to OpenMP parallel programming model with very limited effort and no modification of actual algorithmic legacy code. Two promising high-end computing devices were selected targeting rad-hard family with the GR740 and COTS family with latest Kalray Massively Parallel Processing Architecture device, Coolidge, released end of Q1 2020. OpenMP runtime and Open Source observability tools provided by Barcelona Supercomputing Center were ported from HPC mainstream to the selected hardware targets, and exercised through the selected software use cases. The project demonstrated that usage of OpenMP parallel programming could facilitate the development, and analysis of parallel real-time space applications. Development benefits from embedded runtime thus alleviating the programmer of fine parallelization orchestration burden thanks to non-intrusive and portable source code annotations. The evaluated OpenMP parallel programming model ported to relevant hardware targets accelerates the development, profiling, analysis and execution of parallel real-time space applications, while providing significant performance and portability benefits. Some questions remain open for future work amongst which: • Evaluation of state-of-the-art compiler techniques to guarantee that parallel OpenMP applications are functionally correct and safe, as an example with the absence of pathological race conditions or deadlocks. • Adaptation of the OpenMP runtime libraries to ensure that the timing guarantees devised at analysis time can be guaranteed at deployment time and continue maturing the prototyped instrumentation and observation tools. • Exploration of complementary OpenMP features such as offloading to FPGA or other remote computing devices such as neighbor clusters on a ManyCore or specialized accelerators.

In this contribution, we provide an overview of the results and lessons learnt from the on-going ESA-funded GPU4s project (GPU for Space) performed by the BSC as a prime and ADS as subcontractor. Embedded GPUs can provide significant computational power at a low-power for large amounts of data, allowing the use of software for on-board processing. They allow more flexibility, easier reconfiguration compared to FPGAs and can support several different processing tasks through reuse of compute resources. Moreover, they can leverage an abundance of specialised developers, familiar with widely-used programming models, resulting in an overall lower cost. The purpose of this exploratory project is to address the increased needs for on-board processing performance of future missions, exploring the possibility of using embedded GPUs in space and studying the initial steps required for their adoption. In particular, our goal is the evaluation of GPU IP for possible future space processors as well as the evaluation of COTS GPUs. We performed a survey of existing and future algorithms used in space across all divisions of ADS, to identify which domains expect higher needs for performance and whether their algorithms have good characteristics for GPU parallelisation. We concluded that most space algorithms are a good fit for the GPU programming model, something we confirmed also experimentally later. In another survey, we studied the available hardware solutions and their software ecosystem. We focused on embedded GPU IPs from European providers, to identify the most appropriate one for a radiation-hardened implementation in an ASIC or FPGA in the long term. Additionally, we covered the most important embedded COTS GPU solutions, to identify the most appropriate one for lower cost, short-term adoption. We also expanded our survey to open source IP and GPU-like solutions. From this extensive coverage we selected to benchmark a set of embedded GPUs. For this, we have defined GPU4S Bench [1], an open source embedded GPU benchmarking suite, consisting of algorithmic building blocks from multiple space domains, identified in our space survey. GPU4S Bench provides also the basis and optimised implementations of these algorithms for GPUs and Multi-core CPUs used in ESA’s OBPMark, an open source benchmarking suite for general on-board processing devices. In addition to these benchmarks, we ported complex space applications, such as the Euclid NIR, the image processing and CCSDS compression benchmarks from OBPMark, demonstrating that GPUs can benefit significantly existing and mainly future space processing, in terms of performance and power consumption, including efficiency. In our contribution we will present a summary of the obtained results. Finally, we identified issues such as radiation effect mitigation, thermal management and procurement of GPU devices, which need to be addressed for the adoption of GPUs in space, we proposed potential solutions and defined a roadmap. Overall, our conclusion is that embedded GPUs have a high potential for providing the performance needs of future missions, and can significantly reduce the cost, while offering new capabilities. [1] GPU4S Bench: Design and Implementation of an Open GPU Benchmarking Suite for Space On-board Processing: https://www.ac.upc.edu/app/research-reports/public/html/research_center_index-CAP-2019,en.html

State-of-the-art optical space instruments are capable of producing data rates that considerably exceed the available down-link capacities. The potential data rates are increasing as the sensor technologies progress. These trends require a signifi-cant data reduction on-board, most appropriately at the source of the data, the in-strument. Furthermore, the high availability needed to fulfill the science goals re-quire a high degree of autonomy. In this presentation we will discuss these trends using the example of the architecture of the data processing system of the PLATO payload. PLATO (PLAnetary Transits and Oscillations of stars) is the third medium (M3) mission in ESA’s Cosmic Vision programme. The goal of the PLATO mission is to detect terrestrial exoplanets in the habitable zone of so-lar-type stars and characterize their bulk properties. The PLATO instrument is comprised of 24 identical refracting telescopes each equipped with 4 CCDs, plus 2 additional telescopes which aid the space-craft fine-pointing. Altogether the PLATO payload is comprised of over 27 CPU cores, over 30 FPGAs and 6 Space-Wire Routers. In order to master this complex system of interacting soft-ware and hardware we use standardized protocols, pre-qualified operating sys-tems, Model-Based Systems-Engineering tools and techniques, code-generation, and on-board control procedures. We will give an overview of the aforemen-tioned tools and techniques, discuss their benefits and pitfalls and share the les-sons learnt so far in the development of the PLATO instrument. Finally, we will propose architectural considerations that could potentially improve the perfor-mance of similar instrument designs in the future.

In this paper, we describe the VectorBlox Accelerator software development kit (SDK) and how it is used to optimize and convert trained Artificial Intelligence (AI) models, targeting power-optimized 28nm FPGAs. Neural networks are sourced from a variety of supported input frameworks, such as TensorFlow, Caffe, ONNX, and PyTorch. The VectorBlox Accelerator SDK performs a three-step conversion flow to optimize the networks, calibrate and scale them to 8-bit representation, and finally create an image for implementation on the FPGA. Power-efficient implementation of the neural network on the FPGA is achieved by a soft IP core called CoreVectorBlox. This soft IP core comprises a RISC-V processor and firmware, a vector processor, and a convolutional neural network accelerator, which consists of a two-dimensional array of processing elements, making use of the multiply-accumulate blocks in the RT PolarFire FPGA. By implementing neural networks in a matrix processor programmed in the fabric of the new radiation-tolerant RT PolarFire FPGA, the networks can be iterated and changed without resynthesizing the FPGA, resulting in convenient, programmable low-power AI applications that can be dynamically changed at run-time. Examples of performance and utilization of a variety of neural networks sourced from TensorFlow, Caffe, ONNX and PyTorch will be provided, for implementations both with and without triple module redundancy which may be desired for radiation mitigation purposes. The radiation-tolerant RT PolarFire FPGA will be described, with emphasis on radiation test data and schedules for qualification and flight models.

Current and future space missions demand sophisticated on-board data processing functionalities, while low resources consumption remains a constraint. Thus, in-flight reconfigurable architectures are mandatory. Using dynamically reconfigurable FPGAs allows enhancement of on-board processing with unprecedented levels of flexibility, enabling the adaptation of the system regarding functional and fault-tolerance requirements, subjects to change during mission lifetime. Different operational modes can be served, especially for sharing of complex algorithms on limited FPGA resources. For instrument control and data processing of the PHI instrument on the Solar Orbiter mission (SO/PHI), we have partially adapted results of the ESA study for a Dynamically Reconfigurable Processing Module (DRPM) and implemented a flexible, in-flight reconfigurable, power efficient, and radiation tolerant processing module based on Xilinx Virtex-4 SRAM-based FPGAs. While these and the rad-hard Virtex-5 FPGAs have been used for many current space missions, they provide only insufficient logic resources and embedded memory for future usage. Instead, the Kintex Ultrascale XQRKU060 has become a state-of-the-art rad-tolerant FPGA implementation. As the Xilinx Zynq Ultrascale+ shows, integration of dedicated processors is presently used already for military and commercial space applications. After having demonstrated the usage of in-flight reconfigurability for SRAM-based FPGAs on SO/PHI, we have developed a universal platform for high performance on-board data processing, based on cPCI Serial Space standard and state-of-the-art Xilinx Zynq Ultrascale+ MPSoC device on a single board (3U). The cPCI Serial Space standard guarantees the modular extensibility of the system. The board provides two banks of 64 Gibit DDR3-SDRAM memory and TMR NOR-Flash for configuration and SW code, together with various interfaces, like PCIe, SpaceWire, and Ethernet. Optionally, this could be expanded to control NAND-flash mass memory or implemented with an EV-MPSoC including video codec to enable compression and recording of data streams from video cameras. This module is being used within the H2020 project S4Pro, which investigates how to combine state-of-the-art industrial computing technologies (namely Xilinx Zynq UltraScale+) and space qualified embedded computing platforms in order to optimize the data processing chain and support the next generation of data intensive missions. The approach targets not only the enhancement of technology transfer to nano- and small satellites, but also the enabling of institutional satellite missions that rely on operational tasks with very high bandwidth, processing, and storage requirements. Hence, the Zynq Ultrascale+ is used for on-board data elaboration, e.g. for SAR and multispectral imaging applications, due to its strong interfacing capabilities and integrated software processing units based on the ARM A53 core architecture. Additionally, we are implementing a derivate with similar cPCI Serial Space interfaces using the Xilinx XKU060 (XQRKU060) FPGA. Especially for different types of reconfiguration and scrubbing, a system controller connector is available to connect a special reconfiguration engine. This can be used also to perform fault injection into the FPGA and thus exercise FDIR aspects. Based on this, we are involved in a design study for the Lagrange PMI instrument DPU as follow-on of SO/PHI. In the full paper, we will present more details of our modules and applications.

GMV is in charge of developing the GNC subsystem of HERA mission, currently going through Phase C. At the end of Phase A of the mission, it was identified the lack of any dedicated avionics platform for the implementation of hardware accelerated image processing algorithms required by the spacecraft autonomous navigation strategy. The hardware solution proposed by GMV in order to whitstand the demanding mission GNC requirements is in the form of a dual purpose avionics processing board for image processing and interface control (including management of interfaces with OBC and in-flight reprogramming): HERA Image Processing Unit (HERA IPU). The main drivers of HERA IPU design were the nominal and redundant SpaceWire I/F with the HERA S/C OBCs, and the ability to accommodate and accelerate in hardware the required image processing algorithms having as input images taken by a navigation camera. The HERA Image Processing Unit provides isolation of Image Processing Function and Interfaces Function, as it is relying on a two FPGAs architecture, with allocated external volatile and non-volatile memories. It is composed of one small and reliable European FPGA with rad-hard equivalent, BRAVE NG-Medium, dedicated to interfaces control and monitoring and the other one is a powerful FPGA to perform as computer vision co-processor. The current Processing FPGA is the Virtex-5 VFX130T, with its space equivalent component the high-density rad-hard V5QV FPGA. The SpaceWire interfaces allow TM/TC exchange between HERA IPU and two devices/instruments using nominal and redundant SpaceWire links (nominally 2 on-board computers, but the scenario can be adapted to only one on-board computer and one navigation camera). The design and development of the computer-vision algorithms for HERA IPU are facilitated by the architectural design of the processing FPGA code, which provides an internal interfacing wrapper to integrate the required image processing module satisfying a client-consumer simple interface. HERA IPU also includes pre-processing functions for the image received from navigation camera. The two FPGAs included by HERA IPU allow flexibility and many options for the design and implementation of complex functionalities, such as high-data rate interfaces management and hardware accelerators. Different computer-vision accelerators which are not used in the same moment of time can be used during the mission by replacing bitstreams in the processing FPGA in-flight to save a potentially needed second FPGA unit. The image processing algorithms envisaged by HERA GNC strategy depend on the phase of the mission, and include as minimum: LAMB (maximum correlation with a lambertian sphere – providing the position of the asteroid in the Field of View of the camera) and RelNav Feature Tracking (significant features are extracted from the navigation image, and afterwards compared with the features extracted from previous image in order to find their correspondence). The unit was validated as TRL 4/5 Elegant breadboard, while the validation of a fully scaled Engineering Model is expected to be finished mid 2021. The roadmap towards flight hardware is deemed with high probability of success, as all the electronic components selected for the engineering model have space qualified correspondents.

Over the last years there has been an accelerated increase in the requirements for On-Board Processing (OBP) for satellite on-board systems. These requirements include higher instrument data rates and the introduction of new techniques such as on-board machine learning inference (OBMLI). Outside of a demand for increased computational performance, the need for in-flight re-configurability, as well as shortening development time and cost are increasingly driving the selection of processors and FPGAs in the design of on-board systems. To address these needs, several COTS processor and FPGA devices have been proposed for the use in space applications. While providing superior performance compared to devices specifically designed for radiation environment, the qualification status and lack of openly available radiation data may be prohibitive for the use of COTS devices in certain applications. In this work, a survey of both COTS and RHBD devices supporting high-performance OBP that are currently used or have been proposed to be used in future missions are presented. The survey presented includes device parameters such as the relative performance, qualification status and the availability of radiation test results. Devices included in the survey are: FPGAs, MPSoCs, multicore processors, manycore processors, GPUs, dedicated AI/ML accelerators, etc. Finally, an overview of the availability of machine learning tools for the listed devices is also presented. It is shown that several available devices targeting space applications, offer their users tools for the development and deployment of machine learning algorithms. The work has been carried out partially internally at ESA, and through the on-going TRP activity "FOPIEA" (Future On-board Processing Information Extraction Algorithms). The goal is to have a tentative list of devices that can be used in future ESA missions for IOD purposes. Part of the work has already been provided as input for the ESA COTS initiative working group.

The purpose of this communication is to discuss on the suitability of the HPDP device for power-efficient, on-board data processing in future satellites. Five use cases will be presented. These use cases have been analysed in the course of a GSTP de-risk activity or in the early phases of scientific missions. The performance figures reported below have been obtained on the HPDP evaluation board connected with a laptop. Moon Asteroid Strike (MAS): The MAS algorithm is meant to detect and count the asteroids striking the moon surface. Basically, the idea is to count collision flashes in the moon surface and from their intensity and duration to extract meaningful information about the collision energy and the mass of the asteroid. From the computational point of view the algorithm involves DMA engines, filtering, storage of the last 7 frames and comparison. The computational flow is shown hereinafter. The obtained performance is the on-the-fly analysis of 116 HD fps at the expense of 1.65W power consumption. This implementation is meant to be used in the Lumio mission. Vessel Detection (VD) from EO images: The VD images is meant to detect vessels from EO images. At first, the VD algorithm involves a Sobel filter. This is actually an edge detection filter, which amplifies the various features of the image. The Sobel operator consists of a pair of 3x3 convolution kernels and is designed to perform 2D spatial gradient over an image. In turn, 6 kernels of 20x20 pixels each are applied to the image and after comparison with thresholds the detected vessels are reported. This implementation can process 9.6 HD fps at the expenses of 1.65W power consumption. A step further, a machine learning version of this application has been developed in tensor flow. Targeting >90% correct answers for the application of the VD, the NN is trained outside the HPDP device by using the tensor flow framework and python and the final NN is mapped on the HPDP in order to perform the pattern recognition. For this application a convolutional network is chosen with 1 hidden layer and one final dense layer of neurons. In total, the network has about 5 thousand parameters and it is mapped in 2-3 different configurations of the HPDP array. The different configurations are applied on the fly with a delay of less than 0.5ms per reconfiguration. The obtained results are better than those presented above in terms of accuracy, the processed fps figure is similar as above and the power consumption remains idem. AES 256: Two versions of the encryption algorithm have been developed: AES256 without CBC (Cipher Block Chaining) and AES256 with CBC. First the key expansion is calculated in the FNC0 and is passed via a FIFO to the array. The key expansion process generates the expanded version of the 256b key which is used for the actual encryption. The expanded key is used repeatedly throughout the encryption process and it is stored in the FIFO of the array in order to reduce additional delays. The data to be encrypted are fetched to the incoming stream by the DMA in groups of 4 bytes. From the other side, the cypher sends data to the on-chip SRAM in packets of 4 bytes too. Two instances of the AES256 without CBC IP (resp. One instance of the AES256 IP with CDC) can fit in the array giving a total throughput of 11.7MB/s (resp. 5MB/s) at the expense of 1.65W power consumption. Image Compression: For the compression of the multi-spectral imagery the lossless flavor of the standard CCSDS 123.0-B-2 has been implemented. The compression is split into two sequential configurations and the attained throughput is 1Gb/s image data consumption. In this configuration the power consumption of the device is 1.65W and no external processing capability is required. Currently ISD is working on similar lossless algorithms featuring low entropy encoder targeting the TRUTHS mission.

ExoMars mission is ESA’s greatest commitment to reach the Red Planet in 2023 (ExoMars will take off in September 2022 and the lander will reach Mars in June 2023). ExoMars2022 aims to search for past/present life traces on Mars and to investigate the geochemical and environmental evolution of Mars. To fulfil these objectives, the Rosalind Franklin rover will be equipped with a large quantity of instruments that will allow to select and collect samples up to 2 meters in depth through a drill. Once samples are collected, the rover Sample Preparation and Distribution System will crush the samples and deliver the powdered material within the Analytical Laboratory Drawer (ALD). MicroOmega, MOMA and Raman Laser Spectrometer (RLS) are the three key scientific instruments included in ALD that will perform combined analysis to extract the most information about composition of Mars subsurface RLS is a Raman spectrometer which provides a powerful tool for identification and characterization of minerals and biomarkers. The instrument is made up from several units: a laser for samples excitation, an internal optical head (iOH) which collects the Raman signal returned by the sample and forward to through the Spectrometer Unit where it is diffracted and projected to a CCD. All the mentioned operations are controlled by ICEU, (Instrument-Control & Excitation-Unit) a sophisticated custom electronic box designed to support an autonomous software control based on an exclusive hardware architecture composed by a FPGA (where low level drivers are hosted), a LEON2 (where Application Software (ASW) is running) and associated peripherals will be used by SW to perform assigned tasks. Therefore, RLS ASW purpose is to command and control all RLS critical elements, such as the optical head focusing mechanism, the CCD imager, and thermal control for both CCD (in order to keep it cool) and laser source (to warm it until its working temperature). In order to obtain the maximum scientific performance at the Mars surface while optimizing the limited operation opportunities for RLS, according to the Rover Reference Surface Mission, and solving the technical difficulties intrinsic to space instrumentation, the RLS team has developed an advanced embedded software in a custom-hardware-architecture that provides an automated control of all RLS subsystems as well as post-processing capabilities to perform prompt in-situ analysis of Raman spectra. In addition, it also includes the logic to perform automated Raman acquisitions and applying post-processing algorithm to adapt the sample acquisition parameter of every sample spot under analysis, reducing the sample fluorescence, removing undesired spikes due to Cosmic Ray impacts and by calculating the acquisition integration time. The objective of those algorithms is to maximize the Raman signal intensity of the acquired spectra, avoiding saturation level on the detector. The RLS ASW, commands the autofocus system to guarantee that iOH is optimally focused in the accurate acquisition position over the sample. This is part of the AF algorithm function which allows effective Raman signal collection by maximizing the SNR. Finally, RLS manages communications with Rover’s MMS (Mission Management SW) implementing ‘CANopen Controller IP Core’ protocol.

The open-source real-time OS RTEMS has become quite popular in the Space Domain. Particularly with its Symmetric Multi Processing (SMP) option it may provide a very efficient use of multicore processors. In order to permit its application in more critical space missions, attempts have been made for a qualification according to ECSS standards. As an innovative approach the qualification is organised in form of an open-source approach as well. Funded by ESA a consortium with different contractors is preparing a qualification toolkit that shall be made available to the public. This qualification toolkit will provide tool-chain and test-suite for a mostly automated generation of qualification documents on level C and D, together with a basic set of interfaces for the most common system on chips used in the space domain. A later independent verification and validation is anticipated to obtain a level B qualification. Compared to proprietary software this approach shall lower the required investments and reduce the dependency to sole software providers. The presentation will provide an overview of the available software resources as well as the test equipment. It will further explain how to use those resources in order to prepare the qualification of a set application software and provide an outlook on expected future extensions and additional resources. A very special aspect is the open-source approach for qualification. After the initial steps have been supported by ESA, a continuation and extension of the database has to be organised in an open, self-organised process. Such a process will need to be established in order to sustain the use of the qualification toolkit in future environments without extensive re-developments.

GMV is developing an experimental payload focused on the assessment of in-orbit satellite servicing for tracking debris in MEO/LEO making use of ad-hoc solutions for high-performance computer-vision algorithms, FPGA implementation and new dedicated equipment development for a stand-alone smart-sensor HW combining optics, imaging sensor and computing electronics in a reduced shaped unique enclosure. The SBSS-GNSS (Space-Based Surveillance System) would be hosted as experimental payload on board Galileo satellites of Galileo constellation for offering debris surveillance with main focus in Galileo orbits (MEO) as secondary service for monitoring and detection the proliferations of debris in the most populated space area. Observation of objects in MEO and LEO can be covered by ground-based radar or optical telescopes, the case for a space-based surveillance system is considered as particularly interesting for observation of objects from a different perspective complementing the observations obtained from ground while presenting certain advantages such as higher performances due to absence of atmosphere and good timeliness. Space-based techniques significantly improves the results achievable with ground-based techniques by reducing the observable debris size and by enabling observations for different orbits. The proposed experimental payload consists on a miniaturized independent system which is able to detect debris, through dedicated image processing algorithms accelerated on-board by a HW implementation. The computer-vision solution process images performing various filters looking for debris candidates in the FOV of the camera. The algorithms includes the detection and tracking of potential debris objects, being the detected candidate features in the images. The candidates screening and selection includes different considerations such as the detection of light curves, the different relative motion of stars vs potential debris and the geometrical expected motion and detection over consecutive frames. A preliminary assessment has been already performed evaluating four different cases of debris orbits (taken from results of the PROOF SW tool analysis) keeping fixed the size of the debris. The orbital characteristics are quite different leading to different velocity of the debris on the camera sensor. Distinct values of this quantity, here called “Dwell Rate”, are analyzed from 1.4 pxl/s up to 7.5 pxl/s. The dedicated avionics include two rad-hard FPGA, European NG-MEDIUM and high-performance Virtex5QV. The smart sensor camera is embedded within the miniaturized electronics and directly connected to NG-MEDIUM, in charge of both the image acquisition and also external/internal interfacing. While embedding a camera, the system also allows the connection of eventual second external camera that can supply images from different satellite locations and pointing, covering bigger or complementary areas. Through direct connection with embedded imaging sensor it is managed the configuration, commanding and acquisition of images using FPGA logic and applying pre-processing in streaming pipelinea. Avionics incorporates a second FPGA dedicated to the implementation and HW-acceleration of the computer-vision algorithms, the most computationally demanded part. The FPGA high performance solution allow more consecutive frames being processed thanks to the utilization of FPGA and the pipeline processing in streaming thanks in part to the integration of the logic for the camera images acquisition in the presented embedded HW solution.

AI and machine learning can be employed for many good uses in Space. Examples in remote sensing include cloud detection, image understanding and analytics, image-based decision making, targeted imaging, precision agriculture, supply chain analytics, disaster monitoring, alert and analysis, change detection and anomaly detection and tracking. In telecom satellites, examples include spectrum analysis, responsive and predictive interference detection and mitigation, anomaly detection, network and traffic management, capacity and spectrum planning, and user management. Defense applications include battle management and threat analysis. Across many Space segments, AI and ML are used for cybersecurity and Space debris avoidance. Most AI/ML applications in Space employ standard methods of machine learning and neural networks model development. Models are developed on the ground, typically based on cloud computing and big data for training, or on personal computing and small workstations. Most often, a standard “framework,” or specification language, is employed for model specification and developments. Common frameworks include Google’s TensorFlow, the Keras Python front-end to TensorFlow, Torch and PyTorch, Caffe, Theano, and toolboxes on Matlab and Mathematica. Once the model is trained and can be used for inference, the model can be uploaded to Space for on-board execution, processing on board data. This seamless mode of operation allows high frequency upload of updated models and enable flexibility for whether to execute the inference model on the ground or in Space. To support model-based and framework-based execution, a framework interpreter, or Inference Engine (IE), is employed. The IE receives the model, written in the source specification language such as Keras Python, and executes it. The models, especially in deep learning, are layered and consist of predefined set of kernels, such as 1D/2D/3D convolution layers, separable and depthwise convolution, pooling layers, normalization layers, fully connected layers, flattening layers and softmax layers. The IE implements the layers in parametric form. Execution of the model happens in stages. In each stage, one layer is executed by invoking the appropriate kernel, customizing it using the parameters that are included in the model, and processing the data. A key advantage of this mode of operation is that no programming effort is required to execute each model. Framework-based interpreted model execution faces two key challenges, computational load and data movement. The most computationally challenging kernels are convolutions, which boil down to large number of vector dot (inner) products. RC64 manycore DSP is designed specifically for this type of work, having 256 multiplier-accumulators (MAC) operating in parallel at very high power efficiency. But all arguments, data (activations) and coefficients (parameters) need to be streamed continuously from memories. To fully utilize RC64 high level of parallelism, careful programming is required that orchestrate data movements, caching, pre-fetching and off-loading in a most efficient manner. Luckily, this effort is done once and is embedded in the optimized kernels. RC64, originally designed as a DSP parallel processor, , has turned out extremely efficient in inference execution. Large inference tasks are easily distributed over many RC64 chips, interconnected with very high speed SpaceFibre links for data exchange.

With support from the ESA’s Earth Observation Science for Society Programme and in cooperation with Φ-lab, Spire has created ‘Brain in Space’ - an on-ground testbed, accessible via a web-based interface , replicating Spire’s LEMUR 3U platform, the flagship of Spire’s global nanosatellite constellation of over 100 satellites. Brain in Space consists of much of the same systems that can be found on a flight version of our LEMUR 3U spacecraft including power and communications systems, processing payloads, onboard computer and other utilities. In addition to the standard LEMUR systems, the testbed includes the Google Coral, Jetson Nano and UP Myriad X. embedded edge AI/ML modules. Spire is currently operating several computing platforms for in-situ data processing (including AI/ML applications) that are part of standard Spire SDR products (Zynq Ultrascale+, Jetson TX2i). Spire is planning to expand its computing capabilities in the near future, by launching more and new types of boards. The AI/ML modules allow for scheduling, uploading and testing of AI/ML-powered applications to rapidly process space sensor data of various types and from different sources, i.e.: Automated Identification System (AIS), Automatic Dependent Surveillance - Broadcast (ADS-B), GNSS-RO or Space Environment Monitoring . It also provides a test environment to trial different AI frameworks and algorithms for space applications. Within this simulated environment, users are able to test how well the AI/ML computing platforms can support the development of advanced AI-enabled analytics and edge computing in space. The use of easily accessible on-the-ground testing environment like Brain in Space enables acceleration of new services development, innovations within smart data processing and edge computing directly on-board of small satellites and, stress-testing new solutions ahead of the launch to space. It also creates an opportunity to pilot new, AI-ML - empowered ways of how nanosatellite constellations are operated and managed. The “Brain in Space” was developed under a programme of, and funded by, the European Space Agency. The views expressed above are in no way to be taken to reflect the official opinion of the European Space Agency.

Computing power requirements of space applications, in particular for payload functions, are constantly rising with the objective of increasing the number of processing operations that can be carried out on board to achieve higher spatial, spectral, temporal and radiometric resolution. Moreover, next generation projects are seeking a breakthrough by embedding new innovative processing to gain efficiency, flexibility and autonomy, putting more emphasis on the necessity to have powerful processing components. Two kinds of solutions are available to meet the need of on-board processing power: radiation tolerant COTS hardware subject to component unavailability issues, or space dedicated hardware with lower capabilities. Instead, the RC64 component from Ramon.Space proposes a good compromise, being radiation-hardened (Rad-Hard) and proposing significant processing capacity with 64 DSP cores. The RC64 component is a many-core radiation-hardened processor, built around spatial DSPs [1], and designed for applications requiring high computing power and very high speed processing [2][3]. This component has a massively parallel architecture that offers processing performances close to FPGAs or specialized ASIC, but also provides the flexibility of software programming and low power consumption. However, its architecture requires rethinking the algorithms in order to make the most of the RC64 capabilities. Last few years, CNES has been working on the evaluation of this component for different space applications. As part of this effort, a study has been launched to evaluate the suitability of using the RC64 component with high complex algorithms. The algorithm used on this study is based on the latest CCSDS 123 standard [4], which introduces, among other new features, near-lossless compression. However, this standard has been designed in a sequential way, which means that modifying the execution model is necessary for parallel operation, and it was a required step before any attempt at efficient implementation on a many-core. Thus, the first step was to familiarise with this component through micro-benchmarks, to learn good practices and gain technical knowledge for an efficient use of the RC64 programming model, as well as on its architectural specificities: the impact of DSPs in parallel, the memory hierarchy and the hardware multi-tasking scheduler needed to be analysed to achieve good processing performances. Then we applied this knowledge to the considered CCSDS algorithm, testing different configurations, in both the prediction and the encoding steps which have an impact on parallelisation performance and memory usage, so as to realise a preliminary implementation of the CCSDS-123.0-B-2 that tried to achieve efficient performance and scalability on the RC64. In this paper, we propose an overview of the RC64 architecture and a description of the methodology used to parallelise efficiently the application on this target. Then, we present the results of the implementation as well as the conclusions of the evaluation. [1] CEVA DSP processors for MacSpace, MacSpace Symposium and Summer Seminar, 2016 [2] RC64: High Performance Rad-Hard Manycore, Aerospace Conference, 2016 IEEE [3] MacSpace / RC64 architecture, MacSpace Symposium and Summer Seminar, 2016 [4] Low-Complexity Lossless and Near-Lossless Multispectral & Hyperspectral Data Compression, Recommended Standards. CCSDS-123.0-B-2. Blue Book, 2019

The need for a significant increase of the on-board data processing performance/power ratio with higher flexibility in space systems is asserted for several years already. The assigned targets for future space data processors led to the development of the NG-Ultra processing component through several European Research and Development programs supported by major institutional and industrial stakeholders. This device is a key building block for achieving the European objectives of high competitiveness and technology non-dependence in future space systems. The NanoXplore's NG-Ultra is a Rad-Hard Multi-Processor System on a Chip (MPSoC) embedding 4 cores Arm® and a very large FPGA matrix with pre-developed generic functions for spacecraft data-handling. It provides room to embark more processing functions as needed by user applications. Major milestones of the NG-Ultra development have been achieved in 2020 in particular with the successful production of the first samples components. Evaluation boards have been delivered to Airbus Defence and Space to validate the design and provide feedback also on the associated development tools. In parallel to these bring-up activities, the next generation computer development has started, built on NG-Ultra component as its processing core. It follows the Advanced Data Handling Architecture modularity requirements with the objective to provide a high performance processing function covering all kind of space applications for the next On Board Data Processors generation. The presentation will provide an overview and update on these activities, together with a perspective for the utilization of the NG-Ultra in future on-board processing products.

As the number of earth observation missions increases, and so does the number of images acquired by satellites, the need of optimizing on-board mass-memory allocation and data transmitted to ground becomes more and more important, as both are limited resources. Currently, large players in the space industry are still designign Instrument Control Units which are largely customized so, once developed, they can only be exploited for the single mission they have been designed for. On the other hand, the unavailability of commercial flexible, programmable and space-qualified solutions ready to be integrated keeps small-medium enterprises out of this market, since mostly large-scale integrators have the resources needed to undertake custom development at unit level. The Instrument Control Unit with Artificial Intelligence Engine System on Chip presented in this work aims at changing the rules of image/data handling and processing on-board satellites, overcoming the downsides of the current scenario both at the technological and industry level. The core of the proposed system is a soft-programmable hardware GPU, referred to as FGPU from now on, which integrates an artificial intelligence engine able to deliver an innovative fully programmable data handling and data processing system on a chip. The envisaged embedded system can analyse and process the data acquired directly on-board, using artificial intelligence and computer vision algorithms, to discharge unwanted data and transmit only the meaningful ones. It is also fully re-configurable, to allow for the retargeting of earth observation missions as monitoring needs change, even temporarily. At a broader industry level, the commercial availability of such a solution would translate into an enlarged room for Small Medium Enterprises for leading independently space missions and projects, hence wider business opportunities in the space domain. The adopted approach leverages three highly innovative, open-source components at the advanced technology readiness level: the RISC-V processor as a command-and-control platform; the FGPU provided by B-TU as hardware accelerator for computer vision and artificial intelligence algorithms; and the SpaceFibre/SpaceWire as communication module. The three components are integrated into a single modular, flexible, and powerful embedded architecture on a single chip, and more in particular on a space-grade FPGA.

Detecting anomalies in telemetry data captured on-board a spacecraft is a critical aspect of its safe operation, and it allows us to effectively and timely respond to failures and hazards. There exist three main types of anomalies that should be considered for such complex missions. In point anomalies, telemetry values fall outside the nominal operational range. The collective anomalies refer to the overall sequences of consecutive telemetry values that are anomalous (a single data point does not necessarily manifest an anomaly), whereas in contextual anomalies, the single values are anomalous within their local neighborhood. There have been various approaches for automating the process of detecting anomalies from telemetry data. The basic yet widely exploited algorithms include the out-of-limit techniques that are built upon the assumption that we have the prior expert knowledge allowing us to exploit a rule-based approach for detecting unexpected events. There are machine learning algorithms for this task, but they are commonly heavily parameterized and require large amounts of ground-truth (manually delineated) data, ideally with captured anomalies. Since acquiring such data is infeasible in practice, unsupervised techniques have attracted the research attention, as they do not require having large training samples to train well-generalizing models. In this paper, we not only review the current state of the art in anomaly detection from telemetry data, but also present our algorithm for this task – being developed as a part of our Antelope on-board computer with predictive maintenance capabilities – which exploits recurrent neural networks (we are currently utilizing long short-term memory networks) to model the expected telemetry signal. Such models can be trained from a set of the simulated nominal telemetry signals (e.g., using the software or hardware-in-the-loop simulators), or from a set of real-life telemetry presenting the nominal operation. Importantly, we can learn from the correct examples that do not contain anomalous events – it allows us to abstract from the type of anomalies that we want to target. Once the expected signal is elaborated, it is confronted with the actual one, and the obtained error triggers the alert showing that the anomaly has appeared. We additionally show how to thoroughly verify the anomaly detection techniques in a quantitative way, and what kind of metrics comprehensively reflect the underlying abilities of such deep learning techniques. Finally, we will present our visualization tools that help us better understand the advantages and shortcomings of various anomaly detection methods and will discuss our experiments performed over benchmark one-dimensional signal, and real-life telemetry captured on-board the European Space Agency’s OPS-SAT satellite. Since the Antelope will be exploited on-board a satellite, our resource-frugal models will ultimately help us respond to the events quicker and could be used to reduce the amount of data to transfer back to Earth through annotating the most important parts of the signal that enable further analysis and interpretation.

System-on-Chip such as the Zynq UltraScale+ combine multi-core processors (PS), programmable logic (PL) and peripherals in a single device. For space applications, these devices offer the possibility for unprecedented functional integration and performance in a smaller form factor. Challenges arise in mixed-critical use-cases such as onboard computers with payload integration where the dependability of critical functions can not be compromised by adjacent functions. To improve the adoption of the MPSoC technology in space, avionics developers should consider providing a baseline design framework with native functionality and the possibility for users to further exploit other MPSoC resources. For both, a safe level of dependability shall be guaranteed, which can be balanced with performance according to the use-case. One possible solution is to exploit native MPSoC isolation and fault detection features. This is often not sufficient, especially when use-cases require data to be shared across domains with different criticality levels which may lead to failure propagation. EVOLEO and AIRBUS, in the frame of the ESA GSTP project CHICS, are developing an ADHA compatible radiation tolerant 3U computer, based on the Zynq Ultrascale+ for mixed criticality space applications. The solution considers a clear separation between platform and payload functions within the MPSoC. It is oriented towards parallel but independent developments for platform and payload functions, which are often the responsibility of different entities. The platform side includes lockstep ARM R5 cores and peripherals such as DDR4, SpaceWire router, CAN and UART. The solution offers IP cores, drivers, handlers and software to command and control these peripherals. The payload side considers four ARM A53 cores with XEN Hypervisor and cache coloring, high speed serial transceivers, and dedicated DDR4 and PL resources. These elements are connected via a generic AXI infrastructure. Multiple user applications (SW or logic based) can be easily integrated. Both criticality sides are connected via a bespoke “secure data exchange unit” in the PL. Besides the data exchange, this unit supports an “exchange monitor” for fault detection and isolation with the goal of avoiding fault propagation between criticality levels. The complexity of this exchange monitor is scalable to user needs and available FPGA resources. It can range from ECC, contextual aware limit-type checks up to machine learning algorithms. All alarms are collected and managed by a configurable FDIR function running on the secure side. The consortium explores this concept in a scenario with a SAVOIR OBC on the secure side and GNSS, star tracker on the user side. Position, velocity and time (PVT) are shared to an AOCS application running on the secure side. The exchange monitor detects any out-of-range PVT values before these are fed into the AOCS algorithms. Recurrence oriented HW/SW frameworks are a key enabler to explore the functional integration capabilities of state-of-the-art SoC. These have the possibility of simplifying payload integration into avionics systems, reducing overall costs. Fault detection and isolation may be handled by this infrastructure, providing baseline levels of dependability with upper dependability levels still limited by the device itself.

While seeking the highest amount of protection and reliability in Space (against harsh environments, radiations) has nailed down for a long time the computing performances of Space systems towards low levels versus commercially available devices, some recent use-cases would probably push this into the opposite direction, i.e. much higher amounts of computing capabilities with lower but decent tolerance to Space constraints. Therefore, New Space systems’ designers may think the specifications of their systems differently, with new compromises. Yesterday’s requirements for high protections / low CPU may be transformed into mitigating the protection levels for a disruptive amount of CPU. Teledyne e2v has chosen for long to offer radiation tolerant Gigahertz class compute intensive devices for Space payloads and platforms. Recently, the most powerful CPU for the Space market was introduced, a Quad ARM® Cortex®-A72 (30,000 DMIPS), as well as high-speed high density DDR4 memory (4GB - gigabytes). A substantial amount of radiation testing campaigns on these Teledyne e2v Space parts have allowed to characterize all of them against radiations, to provide reports, guidelines, mitigation schemes to their users. Those characterizations against radiation effects are perfectly complementing the inherent Space quality level and robustness of Teledyne e2v parts (QML-Y or ECSS/NASA grades). This paper presents the radiation testing techniques, results and qualification levels of Teledyne e2v’s high capability compute intensive components. Results on LS1046-Space (Radiation Tolerant Quad ARM Cortex-A72), DDR4T04G72 (Radiation Tolerant high speed compact 4GB DDR4), and Qormino® QLS1046-4GB-Space (compact computing module embedding LS1046-Space & DDR4T04G72 components) will be presented. These results highlight that latest campaigns on TID and SEE (SEU, SEFIs) performed by Teledyne e2v allow any designer of Space systems to consider those devices. In order to better understand the level of computing capabilities from Teledyne e2v components, and how their usage allow to contemplate very modern use cases for Space, this paper also presents example of applications that can be considered with such components. A specific focus, is made on the preliminary results obtained on a project aiming at using Qormino for Artificial Intelligence (AI) in Earth observation applications. In particular, benchmark activities were performed to evaluate the Qormino performance on classical neural networks, and compare it with standard components. Some extracts of this benchmark study are presented, in order to outlook what the new space applications may foresee and benefit from safe Space qualified compute intensive parts.

Detecting anomalies in telemetry data captured on-board a spacecraft is a critical aspect of its safe operation, and it allows us to effectively and timely respond to failures and hazards. There exist three main types of anomalies that should be considered for such complex missions. In point anomalies, telemetry values fall outside the nominal operational range. The collective anomalies refer to the overall sequences of consecutive telemetry values that are anomalous (a single data point does not necessarily manifest an anomaly), whereas in contextual anomalies, the single values are anomalous within their local neighborhood. There have been various approaches for automating the process of detecting anomalies from telemetry data. The basic yet widely exploited algorithms include the out-of-limit techniques that are built upon the assumption that we have the prior expert knowledge allowing us to exploit a rule-based approach for detecting unexpected events. There are machine learning algorithms for this task, but they are commonly heavily parameterized and require large amounts of ground-truth (manually delineated) data, ideally with captured anomalies. Since acquiring such data is infeasible in practice, unsupervised techniques have attracted the research attention, as they do not require having large training samples to train well-generalizing models. In this paper, we not only review the current state of the art in anomaly detection from telemetry data, but also present our algorithm for this task – being developed as a part of our Antelope on-board computer with predictive maintenance capabilities – which exploits recurrent neural networks (we are currently utilizing long short-term memory networks) to model the expected telemetry signal. Such models can be trained from a set of the simulated nominal telemetry signals (e.g., using the software or hardware-in-the-loop simulators), or from a set of real-life telemetry presenting the nominal operation. Importantly, we can learn from the correct examples that do not contain anomalous events – it allows us to abstract from the type of anomalies that we want to target. Once the expected signal is elaborated, it is confronted with the actual one, and the obtained error triggers the alert showing that the anomaly has appeared. We additionally show how to thoroughly verify the anomaly detection techniques in a quantitative way, and what kind of metrics comprehensively reflect the underlying abilities of such deep learning techniques. Finally, we will present our visualization tools that help us better understand the advantages and shortcomings of various anomaly detection methods and will discuss our experiments performed over benchmark one-dimensional signal, and real-life telemetry captured on-board the European Space Agency’s OPS-SAT satellite. Since the Antelope will be exploited on-board a satellite, our resource-frugal models will ultimately help us respond to the events quicker and could be used to reduce the amount of data to transfer back to Earth through annotating the most important parts of the signal that enable further analysis and interpretation.

An important challenge faced by mission-critical computers is the ability to scale the processing performance, while maintaining a high level of dependability in a harsh environment. The adoption of COTS multi-core processors, as in non-critical industries, poses difficulties both in terms of timing interference due to concurrent access to shared hardware resources, and reliability under thermal and radiation stress. Specific dependability-related features are thus required for space computers. In this domain, the LEON processors [1] are a European success-story, adopting an open architecture, being available as open-source implementations to allow validation by a wide user base, and having fault-tolerant implementations available to support missions with high-reliability requirements. The recent RISC-V [2] open-source instruction set architecture is a great opportunity to push this concept further, with a renewed potential for growth and wide adoption. The De-RISC project (Dependable Real-time Infrastructure for Safety-critical Computer) aims at providing the first complete processing platform for space, leveraging RISC-V cores and state-of-the-art hypervisor technology. The platform is composed of an FPGA-based SoC with high-performance NOEL-V cores [3], minimized interference channels and many space-grade peripherals. The SoC platform hosts both the XtratuM Next Generation (XNG) hypervisor [4] and LithOS guest operating system [5] for applications isolation and scheduling. In addition, the platform implements advanced monitoring techniques that help to ensure the real-time behaviour in a multicore context. In order to validate the platform, in addition to basic benchmarks and tests, a realistic space use-case will be deployed, based on the LVCUGEN (Logiciel de Vol Charge Utile GENerique) framework [6], the CNES generic payload software based on Time & Space Partitioning. WIth the CCSDS123 hyperspectral image compression [7] as a high-throughput application and TM/TC communications as low-latency critical application, it covers a complete mixed-critical system. The current status of the project is in line with the plans: the prototype platform is already functional and almost complete, with new features added in scheduled internal releases. The validation phase has started, and will proceed incrementally until the end of the project. The commercial release of the platform is expected for Q2 2022. References: [1] https://www.gaisler.com/index.php/products/processors/leon5 [2] https://riscv.org/ [3] https://www.gaisler.com/index.php/products/processors/noel-v [4] https://fentiss.com/products/hypervisor/ [5] https://fentiss.com/products/lithos/ [6] Julien Galizzi, Jean-Jacques Metge, Paul Arberet, Eric Morand, Fabien Vigeant, et al.. LVCUGEN (TSP-based solution) and first porting feedback. Embedded Real Time Software and Systems (ERTS2012), Feb 2012, Toulouse, France. [7] Lucana Santos Falcon, Roberto Camarero. Introduction to CCSDS compression standards and implementations offered by ESA, European Workshop on On-Board Data Processing (OBDP2019), Feb 2019, Noordwijk, Netherlands.

The Horizon 2020-funded SURPRISE project targets the development and demonstration of a geostationary compressive optical imaging instrument in the visible, NIR and MIR spectral bands. The instrument leverages the compressed sensing paradigm in several ways: first, the aim is to perform native data compression and encryption, thereby avoiding the need of dedicated hardware; second, the instrument targets super-resolution reconstruction, achieving a number of spatially resolved pixels that is larger than the number of detectors; third, the instrument is intended to have the capability to perform onboard analysis of the acquired scenes in order to detect events of interest. This paper presents an overview of the SURPRISE instrument concept and the related demonstrator, and an in-depth analysis of the related reconstruction algorithms. Specifically, we focus on the super-resolution imaging approach, and leave the discussion of encryption and onboard analysis for subsequent papers. Indeed, while compressed sensing has traditionally employed sparsity-based methods to recover the images from a set of random projections, the performance of these methods has often been unsatisfactory. A new generation of deep learning methods has shown that it is possible to learn strong priors from training data, obtaining reconstruction accuracy far better than that exhibited by the conventional methods. This paper reports on the performance and the features of a deep learning method selected for image reconstruction. The method is based on the ISTA-NET+ neural network, which has suitably generalized in order to match the optical design of the SURPRISE instrument and the related functional requirements. ISTA-NET+ mimics a few unrolled iterations of an iterative shrinkage method. Opposite to traditional methods, which employ a given fixed prior for reconstruction, e.g. sparsity or total variation, the deep learning approach learns the domain that is more suitable for image reconstruction from a training set. We show results of image sensing and reconstruction in a variety of conditions, and discuss the impact of micro-mirror type, model noise, amount of super-resolution. Current results on Earth observation images indicate that deep learning methods largely outperform existing methods. We also discuss the ability of the deep learning model to reconstruct images whose type is different from those employed during the training stage. Moreover, while traditional methods are typically iterative and therefore computationally intensive, deep learning methods are much simpler and can be trained to be robust to noise. In the final paper we will also include results on simulated images having the same optical characteristics as those acquired via the demonstrator, which is currently in its advanced design stage.

The timing and power of an embedded neural network application is usually dominated by the access time and the energy cost per memory access. From a technical point of view, the hundreds of thousands of look-up tables (LUT) of a field programmable gate array (FPGA) circuit are nothing more than small but fast and energy-efficiently accessible memory blocks. If the accesses to the block memory can be reduced or, as in our case, avoided altogether, the resulting neural network would compute much faster and with far lower energy costs. We have therefore developed a design scheme that uses precomputed convolutions and stores them in the LUT memories. This allows small (mostly one-dimensional) convolutional neural networks (CNN) to be executed without block memory accesses: Activations are stored in the local per LUT registers and the weights and biases of all neurons are encoded in the lookup tables. Each neuron is assigned its exclusive share of logic circuits. This completely avoids the need for memory accesses to reconfigure a neuron with new weights and allows us to perform weight optimisations at design time. However, it limits the applicability of the overall method to comparatively small neural networks, since we need several LUTs per neuron and even the largest FPGAs only provide hundreds of thousands of LUTs. To make this "in LUT processing" possible, we had to limit the set of available neural network functions. We have identified and implemented a set of functions that are sufficient to make the neural network work, but which can all be implemented efficiently in an FPGA without memory access. Our philosophy is that it is better to adapt the FPGA during training to make the best use of the limited resources available than to try to optimise the functions in hardware, resulting from a non-limited neural network. To make this design scheme usable, we had to develop a set of design tools, helping the AI designer to convert a given reference AI in TensorFlow into an equivalent network of the available hardware functions and to finetune the AI to help to compensate the accuracy loss from changing the implementation. The two most powerful optimization techniques we applied are a variable bitwidht quantization and a depth-wise separation of the convolution. In order to demonstrate the performance of this method, we implemented a CNN based ECG detection. Our implementation only used 40% of the available LUTs on the Spartan S15 chip and none of the blockram or DSP circuits. The system processed 500 pre-recorded ECGs of 5575 samples in 281ms, using only 73mJ in total, resulting in 10 million samples per second and an energy cost of 26.2nJ per sample.