Search

ESA Datalabs is a collaborative scientific platform of the European Space Agency to exploit data across the ESA science directorate missions’ archives (astronomy, planetary and heliophysics). It allows you to bring your code to the data under a private account, shareable with colleagues. Large amount of data such as the GAIA multi-billion stars catalogues can be easily mounted and searchable, allowing large scale scientific investigations impossible to achieve on a regular laptop. It also provides multiple tools to access, process and visualize JWST data and was used during the recent commissioning of JWST. In other words, it handles both public and restricted access data.In the heliophysics domain, data from a few missions are already mounted, including all public data from the Solar Orbiter mission. A few Jupyter notebooks are already available to help the community making use of the full capabilities of the ESA archives. More will be made available in the future including tools such as JHelioviewer and data mining. Interoperability is at the heart of the ESA datalabs infrastructure and connection to clouds such as the NASA heliocloud and Amazon Web Services accounts (AWS) are in progress. Developed over the past few years, ESA Datalabs is scheduled to be public to the scientific community in 2023.

Introduction: The present abstract is intended to show the current ESA’s Planetary Science Archive (PSA) [1] in terms of architecture/infrastructure and the future interfaces and technologies which will be used in the next generation archive. These improvements range from a new graphical user interface developed in Angular to a TAP+ (Table Access Protocol) service as a single access to the data, through a new way of releasing new versions of the PSA more frequently to the scientific community. The PSA development team expects to release this new generation of the PSA this summer 2022. PSA current architecture: The PSA architecture and the technologies involved in its development have only undergone incremental updates in the last 6 years, and are now seen as somewhat obsolete. The front-end has been implemented using the Vaadin framework, which was initially a good strategy, but over time became onerous to maintain (e.g., dealing with JavaScript libraries, wasting time in wrapping some required extensions in Java). Also, from the back-end point of view, there are many interfaces/libraries to access the database (JDBC, PDAP, Data Distribution, etc) forcing us to double the effort when changing the API (see Figure 1: PSA current architecture) In addition, the PSA release approach has not followed a truly Agile approach, taking too long in releasing operationally. This is mainly due to the fact there is no a fully CI/CD strategy to be executed in the environments, leading to very manual release process with manual interventions. Also there are additional problems such as the synchronisation of the repositories when releasing, which strongly depends on the IT department. Figure 1: PSA current architecture New interfaces, technologies and infrastructure at PSA: After one year and half of development, the PSA development team has been able to achieve several goals on the roadmap to a new archive. Mainly, we will rely on a new graphical user interface implemented under the Angular framework (see Figure 2: Future PSA Graphical User Interface). There have been various reasons to migrate to this new technology: Faster development for maps visualisation and 3D interfaces, alignment within the ESDC department in a common front-end framework and also, the discontinuation of Vaadin 8 from March 2022 on. This future GUI will have a modern look and feel, with some relevant changes in line with the new ESA branding. Specially on the home view, where there are now card layouts to access the data from missions/instruments, targets and maps and a prototype traverse view for the ExoMars Rover mission, among other features. This new Angular framework has definitely sped the development up when modifying some JavaScript libraries, creating some end-to-end tests on top through Cypress, etc. This will also increase the performance on the client side consequently improving the user experience. In addition, this implementation is also mobile and tablet friendly/responsive. Figure 2: Future PSA Graphical User Interface Also, the new PSA will count on a single access point to the data through TAP+ (even private data) to homogenise the access by offering a single API, instead of using different interfaces/protocols (JDBC, PDAP, etc.) to access the information. In addition to these new interfaces, the PSA is making a huge effort to set up an infrastructure to support a faster deployment cycle in order to be more agile according to the scrum methodology. This implies integrating and deploying the software (after checking metrics in Sonar, passing the end-to-end tests, etc.) as nightly builds into a safe environment (pre-production) so that the Archive Scientists can test the latest features which, once approved, will go to the operational environment. This follows mostly a DEV-OPS infinite loop but having a middle environment (PRE) in which the scientists can safely test the features. Conclusion: The current PSA development team along with the Science Lead and all of the Archive Scientists are working together to produce a new generation of the planetary archive, with these features: a more modern and responsive GUI based on a stable and well-known technology a single access route to the data with authentication and authorization for private products (TAP+) a new infrastructure of environments which allow a more efficient CI/CD so that the features can be validated earlier, allowing the PSA to offer releases in the operational environment more frequently. References: [1] Besse, S. et al. (2017) Planetary and Space Science, 10.1016/j.pss.2017.07.013, ESA's Planetary Science Archive: Preserve and present reliable scientific data sets. Acknowledgments: The authors would like to thank everybody, especially the PSA development team, who have contributed to the development of the PSA in the recent years and the incoming new generation of the archive.

With new missions being selected, missions moving to post-operations, and missions starting their journey to various targets in the Solar System, the European Space Agency’s Planetary Science Archive [1] (http://psa.esa.int) (PSA) is in constant evolution to support the needs of the projects and of the scientific community. What happened since last year? The past year has been good for the European Space Agency (ESA) Solar System missions and the PSA, with the successful flyby of Earth by the BepiColombo mission to Mercury. The ExoMars 2016 mission is performing nominally and is quickly delivering numerous scientific observations. As is common for ESA missions, access to the data is protected and reserved to members of the science team for the first months of the mission. Once the products are ready to go public, the PSA performs a scientific peer-review to ensure that the products to be made public are of excellent quality for all future users. During the first half of 2020, the PSA has successfully peer-reviewed the CaSSIS and NOMAD observations. Those products are now being made public on a systematic basis once the proprietary period elapses (generally between 6 and 12 months). Early in 2020, filters to search data with geometrical values (i.e., longitude, phase angle, slant distance, etc.) were enabled. For now this service works for Mars Express and Rosetta, but will be soon extended to other missions. One of the main new services provided to the scientific community in 2020 is the Guest Storage Facility (GSF), which allows users to archive derived products. Products such as geological maps, Digital Terrains Models, new calibrated files, and others can be stored in the GSF in the format most used by the users. Contact us to preserve your science! Finally, by the end of 2020 users of the PSA will have access to new services based on Geographical Information Systems. You can contribute to the PSA! At the PSA we constantly interact with our users to ensure that our services are in line with the expectations and needs of the community. We encourage feedback from community scientists through: PSA Users Group: A group of scientific experts advising the PSA on strategic development; Direct interactions: Scientists from the PSA are available and eager to receive your comments and suggestions; ESA missions: If you are part of a mission archiving its data at the PSA, tell us how your data should best be searched and used. Acknowledgement The authors are very grateful to all the people who have contributed over the last 17 years to ESA's Planetary Science Archive. We are also thankful to ESA’s teams who are operating the missions and to the instrument science teams who are generating and delivering scientific calibrated products to the archive. References [1] Besse, S. et al. (2017) Planetary and Space Science, 10.1016/j.pss.2017.07.013, ESA's Planetary Science Archive: Preserve and present reliable scientific data sets.

The ESAC Science Data Centre (ESDC) is in the process of registering Digital Object Identifiers (DOIs) for datasets or group of datasets accessible across the ESA Space Science Archives managed by ESDC. These DOIs are persistent URL that point to DOI landing pages setup and managed by ESDC, actually located at https://www.cosmos.esa.int/web/esdc/doi In the heliophysics domain, the first step has been to register DOIs related to the datasets measured by experiments onboard ESA heliophysics spacecraft. Around 60 experiments are flying or have been flown so far onboard ESA heliophysics missions. At the moment, 47 DOI have been registered with CrossRef pointing to DOI landing pages for each experiment onboard SOHO, Proba-2, Cluster, Double Star, Ulysses and ISS-Solaces and are publicly accessible at https://www.cosmos.esa.int/web/esdc/doi/heliophysics. Discussions are on-going with editors of major journal to promote and acknowledge the use of data from any of these experiments by citing these DOIs. Eventually, this would improve the traceability of the usage of datasets from these experiments. Additionally, a JavaScript Object Notation (JSON) script has been added to these DOI landing pages to make them discoverable through Google Dataset Search (GDS). GDS is a new search engine that helps users to find datasets online. Twenty five millions datasets were already indexed and searchable when this service was launched in early 2020, after a beta version released in late 2018. Thanks to a close interaction with the PI teams, DOIs for ESA heliophysics experiments are now easily discoverable through GDS and not only by looking for a particular experiment name. The use of structured metadata, based on schema.org, has enabled the search through physical processes investigated by these experiments, the type of experiments, the time coverage, the PI names or their affiliation… Various examples will be provided while the next steps of development will be outlined.

Geographical information systems (GIS) are becoming increasingly used for planetary science. GIS are computerised systems for the storage, retrieval, manipulation, analysis, and display of geographically referenced data. Some data stored in the Planetary Science Archive (PSA)[1] have spatial metadata associated to them. To facilitate users in handling and visualising spatial data in GIS applications, the PSA should support interoperability with interfaces implementing the standards approved by the Open Geospatial Consortium (OGC). These standards are followed in order to develop open interfaces and encoding that allow data to be exchanged with GIS Client Applications (e.g. OpenLayers, Cesium...). Access to this data for use in applications can be provided through OGC Web Service (OWS) implementations. An existing open source server is GeoServer, an instance of which has been deployed for the PSA, that uses the OGC standards to allow the sharing, processing and editing of data and spatial data through the Web Map Service (WMS) and Web Feature Service (WFS) standards. On the back-end side, a PostgreSQL/PostGIS instance allows the spatial queries. The final goal is to enhance the PSA (accessible through http://psa.esa.int) further as a portal which enables science exploitation of ESA's planetary missions datasets. This can be facilitated through the GIS framework, offering interfaces (both web GUI and scriptable APIs) that can be used more easily and scientifically by the community, and that will also enable the community to build added value services on top of the PSA. Introduction Some of the current operational ESA planetary missions, such as Mars Express, ExoMars 2016, and BepiColombo, as well as other future missions such as ExoMars 2020, Juice, etc. will benefit of a GIS tool to visualize their targets (Mars, Mercury, Jupiter…) allowing spatial queries to retrieve geometrical information like features, footprints, rover path tracking, rover drill sites, etc. GIS Architecture The PSA relies on 3-tiered system for the GIS architecture (see Figure 1). The database layer is composed of a PostgreSQL database with the PostGIS extension to store the spatial information. The server layer uses GeoServer as a map server to provide WMS/WFS responses (e.g. GeoJson, kml…) to the web application’s requests (implemented on the Vaadin framework). Finally, the client layer (browser) runs the OpenLayer Javascript library to render the map. Other external GIS tools like QGIS might be used to get the PSA spatial data from either the GeoServer or the database. Figure 1: GIS architecture diagram for the PSA Views Consistency The PSA provides different views to show the same planetary data. These views are integrated and synchronized to each other to visualize the information as the data type requires. All of them use the filter menu to search by a given criteria and offer similar features such as sorting, pagination, downloading and product detailed info. Once a query is executed on a view, the information is automatically loaded when changing views. The map view is integrated in the current PSA (see Figure 2) as the other views (Table, Image) giving other perspective of displaying results when it comes to search for spatial data. Figure 2: Views Consistency in the PSA GIS Applicability GIS technology on the PSA will offer a common way to filter (by mission, instrument, target, dates, geometry…) and search for spatial data, even for legacy missions, thanks to the homogenization of the geometrical information with per-product spatial metadata computed in a consistent way via SPICE. PSA will provide spatial data retrieval of both versions of the NASA Planetary Data Systems archival formats, PDS3 and PDS4, based on a criteria search, and, the possibility of selecting PDS3/PDS4 products from a particular region of interest (ROI) (see Figure 3). Figure 3: Query by ROI and footprint selection PSA also provides other useful GIS tools such as switching projections for better visualization and analysis of footprints over the poles (see Figure 4) as well as switching between different base maps of Mars for better visualization, enabling/disabling layer feature, overlapping footprint selection by popup and a grid/graticule layer. Figure 4: Footprints over the Mars North Polar projection PSA also allows the user to add customized and external data through a GeoJSON file uploader tool (see Figure 5). Figure 5: GeoJSON data uploader feature Acknowledgement Although the whole PSA team has somehow been involved in this GIS implementation, I would specially like to thank Francisco Raga for his huge contribution in many of the presented GIS features. References [1] Besse, S. et al. (2017) Planetary and Space Science, 10.1016/j.pss.2017.07.013, ESA's Planetary Science Archive: Preserve and present reliable scientific data sets.