Search

D10.5 “Final Report on Implementation Activities” describes the work towards making the RIs in the solid Earth subdomain FAIR in terms of FAIRness of the assets and EOSC-interoperability. In this deliverable we therefore describe for each WP10 task, the work done to address the gaps identified by Deliverable “D10.2 Roadmap for implementation of FAIR concepts”, prioritised and harmonised in D10.3 “D10.3 Report on coordination, prioritisation, and harmonisation in the Solid Earth Subdomain”.

Scientific progress requires research outputs to be reproducible, or at least persistently traceable and analysable for defects through time. This can be facilitated by coupling analysis tools that are already familiar to scientists, with reproducibility controls designed around common containerisation technologies and formats to represent metadata and provenance. Moreover, modern interactive tools for data analysis and visualisation, such as computational notebooks and visual analytics systems, are built to expose their functionalities through the Web. This facilitates the development of integrated solutions that are designed to support computational research with reproducibility in mind, and that, once deployed onto a Cloud infrastructure, benefit from operations that are securely managed and perform reliably. Such systems should be able to easily accommodate specific requirements concerning, for instance, the deployment of particular scientific software and the collection of tailored, yet comprehensive, provenance recordings about data and processes. By decoupling and generalising the description of the environment where a particular research took place from the underlying implementation, which may become obsolete through time, we obtain better chances to recollect relevant information for the retrospective analysis of a scientific product in the long term, enhancing preservation and reproducibility of results.In this contribution we illustrate how this is achievable via the adoption of microservice architectures combined with a provenance model that supports metadata standards and templating. We aim at empowering scientific data portals with Virtual Research Environments (VREs) and provenance services, that are programmatically controlled via high-level functions over the internet. Our system SWIRRL deals, on behalf of the clients, with the complexity of allocating the interactive services for the VREs on a Cloud platform. It runs staging and preprocessing workflows to gather and organise remote datasets, making them accessible collaboratively. We show how Provenance Services manage provenance records about the underlying environment, datasets and analysis workflows, and how these are exploited by researchers to control different reproducibility use cases. Our solutions are currently being implemented in more contexts in Earth Science. We will provide an overview on the progress of these efforts for the EPOS and IS-ENES research infrastructures, addressing solid earth and climate studies, respectively.Finally, although the reproducibility challenges can be tackled to a large extent by modern technology, this will be further consolidated and made interoperable via the implementation and uptake of the FDOs. To achieve this goal, it is fundamental to establish the conversation between engineers, data-stewards and researchers early in the process of delivering a scientific product. This fosters the definition and implementation of suitable best practices to be adopted by a particular research group. Scientific tools and repositories built around modern FAIR enabling resources can be incrementally refined thanks to this mediated exchange. We will briefly introduce success stories towards this goal in the context of the IPCC Assessment Reports.

Doing high quality research involves complex workflows and intermediate datasets. An important part is also sharing of those datasets, software tools and workflows among researchers, and tracking provenance and lineage. It also needs to be stored in a citable permanent repository in order to be referenced in papers and reused subsequently by other researchers. Supporting this research data life cycle properly is a very challenging objective for research infrastructures. This is especially true with rapidly evolving technologies, sustainable funding problems and human expertise. In the climate research infrastructure, many efforts have been made to support end-users and long tail research. There is the basic data distribution, the ESGF data nodes, but this is to support mainly specialized researchers in climate science. This basic infrastructure implements quite strict standards to enable proper data sharing in the research community. This is far from FAIR compliance, but this has proven to be extremely beneficial for collaborative research. Of course, high level components and services can be built on top. This is not an easy task, and a layered approach is always better to hide the underlying complexity and also to prevent technology locking and too complex codes. One example is the IS-ENES C4I 2.0 platform (https://dev.climate4impact.eu/ ), a front-end that eases very much data access, and is acting like a bridge between the data nodes and computing services. The C4I platform provides a very much enhanced Jupyter-la like interface (SWIRRL), with many services to support sharing of data and common workflow for data staging and preprocessing, as well as the development of new analysis methods in a research context. Advanced tools that can calculate end-user products are also made available along with some example notebooks implementing popular workflows. One of these tools is icclim (https://github.com/cerfacs-globc/icclim), a python software package. C4I also includes high-level services such as on-the-fly inter-comparisons between climate simulations with ESMValTool (https://github.com/ESMValGroup/ESMValTool). All this work is also including large efforts to standardize and to become closer to FAIR for data, workflows and software.Another way of helping researchers is to pre-compute end-users products like climate indices. This is extremely useful for users because it can be really complex and time consuming to calculate those products. One example is to provide those users datasets of climate indices pre-computed on CMIP6 simulations would be very valuable for those users. Of course all specific needs cannot be taken into account but the most general ones can be fulfilled. The European Open Science Cloud (EOSC) is providing computing and storage resources through the EGI-ACE project, enabling the possibility to compute several climate indices. In this EGI-ACE Use Case, icclim will be used to compute 49 standard climate indices on a large number of CMIP6 simulations, starting with the most used ones. It could also be extended to ERA5 reanalysis, CORDEX and CMIP5 datasets.This project (IS-ENES3) has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement N°824084.

Many end users of climate change information often need specialized products to perform their research or data analysis. For example, climate indices, like the standard ones defined by ECA&D and ETCCDI, cover most of the general needs. However, datasets provided on the climate data infrastructure ESGF are climate model output and only provide standard variables, such as temperature and precipitation, and not climate indices, such as the number of summer days or the number of days with heavy precipitation, for example. A python package to calculate climate indices, called icclim https://github.com/cerfacs-globc/icclim, is currently developed within the H2020 IS-ENES3 project, with a strong collaboration with the xclim https://github.com/Ouranosinc/xclim developer team. This package is using xarray and dask for very fast execution and smaller memory footprint. It also aims at providing increasing support for all FAIR aspects. It also provides the possibility to users to define their own climate indices using a pseudo-language close to a Domain Specific Language (DSL), along with proper metadata generation. This tool is integrated in the IS-ENES C4I 2.0 platform (https://dev.climate4impact.eu/), using a Jupyter notebook collection https://gitlab.com/is-enes-cdi-c4i/notebooks in a SWIRRL environment (Software for Interactive Reproducible Research Labs https://gitlab.com/KNMI-OSS/swirrl). Having access to this type of complex analysis tool is very useful, and integrating them with front-ends like C4I enables the use of those tools by a larger number of researchers and end users. Providing those users tools and also datasets of climate indices pre-computed on CMIP6 simulations would be very valuable for those users. Of course all specific needs cannot be taken into account (such as specific seasons, specific reference periods, etc.), but the most general ones can be fulfilled. The European Open Science Cloud (EOSC) is providing computing and storage resources through the EGI-ACE project, enabling the possibility to compute several climate indices. In this EGI-ACE Use Case, icclim will be used to compute 49 standard climate indices on a large number of CMIP6 simulations, starting with the most used ones. It could also be extended to ERA5 reanalysis, CORDEX and CMIP5 datasets. The resulting climate indices datasets will later be made available in the C4I portal. The tool icclim will be presented, along with preliminary results of the pre-computed climate indices dataset., https://github.com/cerfacs-globc/icclim

Thedata and code licensing policy for the IPCC endorsed by the IPCC Task Group on Data Support for Climate Change Assessments. Data needs to be accessible to both IPCC authors and external users. Undue restrictions should not be placed on the users, while, at the same time, the legitimate interests of the owners of data should be protected. In the interests of preserving clarity and transparency while protecting the rights of data providers, the Creative Commons Attribution Licence (CC BY 4.0) is the recommended data licence for the IPCC. Open-source licenses are recommended for code that underlies IPCC data products.&nbsp

With the recent acceleration of climate change impacts throughout the world, it is getting even more important and urgent to provide climate data in a way that it can be used seamlessly and easily. This can really make a difference in making climate data actionable: data that can lead to stakeholders and policy actions. Climate data users often face real challenges when dealing with climate data because of data volumes, distributed datasets in a collection of hundreds of files, and very long processing time in order to create useful products. Providing ready-made products is an important part of making climate data more accessible to end users. Such tools like icclim1, a flexible python software package to calculate climate indices and indicators, is very valuable in order to pre-compute end users’ products. Those tools must adhere as much as possible to metadata conventions such as CF, implementing also provenance information, and be as close to FAIR. In the context of the EGI-ACE H2020 project, a dataset of standard climate indices is being created, using data from CMIP6 climate simulations stored on the ESGF. The calculations are performed using the 5.x version of icclim1. The icclim software is also integrated with the C3S toolbox2 to provide tutorials on how to calculate climate indices. This tool and dataset are in the process to be integrated in the IS-ENES C4I 2.0 platform3. A Jupyter notebook collection is provided in a SWIRRL3 environment (Software for Interactive Reproducible Research Labs). Having access to this type of complex analysis tool and ready-made products in an extensive dataset are crucial for end users and to make climate data more readily usable. The interfaces and front-ends to access those tools and datasets are also an important part of making data more actionable for end users. In C4I there is also documentation and scientific information on climate data and climate simulations, as well as guidance. This project (IS-ENES3) has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement N°824084. REFERENCES [1] https://github.com/cerfacs-globc/icclim [2] https://cds.climate.copernicus.eu/cdsapp#!/toolbox [3] https://dev.climate4impact.eu/ [4] https://gitlab.com/KNMI-OSS/swirrl

Thedata and code licensing guidelines for the IPCC endorsed by the IPCC Task Group on Data Support for Climate Change Assessments. Data needs to be accessible to both IPCC authors and external users. Undue restrictions should not be placed on the users, while, at the same time, the legitimate interests of the owners of data should be protected. In the interests of preserving clarity and transparency while protecting the rights of data providers, the Creative Commons Attribution Licence (CC BY 4.0) is the recommended data licence for the IPCC. Open-source licenses are recommended for code that underlies IPCC data products.&nbsp

Many end users of climate change information often need to rely on external data analysis environments, having proper tools to produce meaningful information for their specific applications. Data volumes as well as the number of datasets are increasing very rapidly. The ability to select, process and download all needed data is getting complex, technical, and very time-consuming, especially when datasets are distributed among several data centres and into a large quantity of files. This is the case for the climate data distribution infrastructure ESGF (Earth System Grid Federation). This means that science gateways are very important for end users to ease the access to climate data analysis. There are several platforms available to hide this complexity from users and provide a seamless access to climate analysis on large pre-generated archives of datasets’ copies. We can cite the Copernicus Data Store, Pangeo and ESGF Amazon Web Services, as they provide a (limited) subset of the most used climate datasets. Another important platform is developed within the H2020 IS-ENES3 project: climate4impact 2.0, providing access to all ESGF datasets. It is using an enhanced Jupyter-Lab environment orchestrated via the SWIRRL5 API [1] along with a collection of Jupyter notebooks as a useful set of examples on how to use and process data. Similar solution to EODAG Jupyter extension can be envisaged once the backend of the ESGF will be migrated. The selected datasets, using an intuitive and simplified interface (that can also be set in "Expert Mode''), are asynchronously staged from the ESGF, via SWIRRL workflows, onto the user space, obtaining custom and limited sized data-spaces. Users can choose to reduce the size of the transferred data by configuring a subsetting workflow. This invokes the remote WPSs available at data nodes of the ESGF, which will slice the data before sending. The portal also provides interactive pages for the evaluation of climate models, using ESMValTool [2], to guide users on selecting appropriate climate datasets. The notebooks that can be executed in climate4impact, are developed using a very convenient software library to calculate climate indices and indicators called icclim [3]. This library is a flexible python software package to calculate climate indices and indicators. This tool adheres as much as possible to metadata conventions such as the Climate & Forecasting Conventions (CF-1.x) as well as the clix-meta work that is being done in IS-ENES3. It provides the implementation of most of the international standard climate indices. It has been validated against Climpact as well. This project (IS-ENES3) has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement N°824084., {"references":["[1] A. Spinuso, M. Veldhuizen, D. Bailo, V. Vinciarelli, T. Langeland; SWIRRL. Managing Provenance-aware and Reproducible Workspaces. Data Intelligence 2022; doi: https://doi.org/10.1162/dint_a_00129","[2] Righi, M., Andela, B., Eyring, V., Lauer, A., Predoi, V., Schlund, M., ... & Zimmermann, K. (2020). Earth System Model Evaluation Tool (ESMValTool) v2. 0–technical overview. Geoscientific Model Development, 13(3), 1179-1199","[3] Abel Aoun, Christian, Natalia, Krahshtrea, marev711, Lars Bärring, bekozi, Petros Gasteratos, & bascrezee. (2022). cerfacs-globc/icclim: v5.1.0 (v5.1.0). Zenodo. https://doi.org/10.5281/zenodo.6380604"]}

Presentation delivered at EGU 2022 (Vienna, Austria): End users of climate change information are relying on climate services and tools in order to produce meaningful information for specific applications. Data volumes as well as the number of datasets are increasing very rapidly, and the ability to select, process and download all needed data is getting complex, technical and very time-consuming. It is especially true since those datasets are often distributed among several data centres and into a large quantity of files. Several platform are being developed to hide this complexity to users and provide a seamless access to climate data, as well as to provide on-demand data analysis capabilities. We can cite, for example, theCopernicus Data Store(CDS https://cds.climate.copernicus.eu), along with its toolbox to perform online data analysis. Another platform is developed within theH2020 IS-ENES3project, calledclimate4impact(C4I 2.0 https://dev.climate4impact.eu ). It is using an enhancedJupyter-Labenvironment called SWIRRL (Software for Interactive Reproducible Research Labs https://gitlab.com/KNMI-OSS/swirrl ) along with a collection ofJupyter notebooks(https://gitlab.com/is-enes-cdi-c4i/notebooks) as useful set of example on how to use the data. Finally, the portal provides interactive pages for the evaluation of climate models (usingESMValTool) to guide users on selecting climate datasets. The notebooks that can be executed in C4I, are developed using a very convenient software library, which is made available via SWIRRL, tocalculateclimate indices and indicatorscalledicclim(v5.0 https://github.com/cerfacs-globc/icclim ). This library, which is also in the process of being integrated into the C3S, is a flexiblepython software packageto calculate climate indices and indicators. This tool adhere as much as possible tometadata conventionssuch as the Climate & Forecasting Conventions (CF-1.x) as well as the clix-meta (https://github.com/clix-meta) work that is being done in IS-ENES3. Properprovenance informationstill needs to be added. The ultimate goal is to be as close as possible to allFAIRaspects. icclim is designed with performance and optimisation in mind, because the goal is to provide on-demand calculations for users. It provides the implementation of most of the international standard climate indices such asECAD,ETCCDI,ET-SCI, including the correct methodology for calculatingpercentile indicesusing the bootstrapping method. It has been validated againstR.Climdexas well (https://cran.r-project.org/web/packages/climdex.pcic/index.html). This new 5.x version of icclim is based on functions from thexclim(https://github.com/Ouranosinc/xclim) python library, which was inspired by earlier versions of icclim, but usingxarrayanddaskfor data access and processing. In this presentation, the climate4impact 2.0 platform will be described along with the icclim climate indices tool. Important metadata aspects will also be discussed (clix-meta). A few examples using the jupyter notebook collection will be shown.

The IPCC Task Group on Data Support for Climate Change Assessments (TG-Data) recommends the implementation of FAIR (Findable, Accessible, Interoperable, Reusable) data principles and best practices for the documentation and curation of data that is assessed by the IPCC. The motivation to implement FAIR in the IPCC is to increase transparency and accessibility of the assessment, the implementation of the IPCC Error Protocol, and the long-term curation of the assessed digital information. This document introduces the implementation of FAIR data principles into the IPCC process and reflects the four elements of FAIR to find the data, produce and reproduce figures and, finally, to document the provenance for reusability. It presents standard (basic) measures that are recommended for all digital data that is assessed, intermediate measures that achieve reproducibility of assessed digital information, for example through the use of collaborative platforms for figure development, and also full (advanced) measures to achieve reusability of digital products with complete provenance documentation.

A white paper setting out strategic requirements of the European Network for Earth System Modelling (ENES) Climate Data Infrastructure (CDI).

The European Network for Earth System Modelling (ENES) has operated since 2001 to support advances in climate modelling and understanding. ENES has been supported by a succession of infrastructure projects (IS-ENES, IS-ENES2, and IS-ENES3; 2009-2022) as well as by an HPC Centre of Excellence (ESiWACE, ESiWACE2; 2015-2022). These activities, carried across multiple institutions exploiting mutual support, have been delivering on a community infrastructure strategy first published in 2012 and updated in 2017 [isenes2012, isenes2017], and jointly deliver the ENES Research Infrastructure (ENES-RI). The data service element, also known as the ENES Climate Data Infrastructure (ENES-CDI), provides a range of services which are critical for the efficient exchange and exploitation of coordinatedclimate simulations which underpin major advances in climate science and decisions on social, political and economic adjustments associated with climate change. This document outlines the overall objectives of the ENES climate data infrastructure, with some detail in terms of current activities, and near-term aspirations. It will be updated as the community infrastructure strategy evolves. &nbsp

End users of climate change information are relying on climate services and tools in order to produce meaningful information for specific applications. Data volumes as well as the number of datasets are increasing very rapidly, and the ability to select, process and download all needed data is getting complex, technical and very time-consuming. It is especially true since those datasets are often distributed among several data centres and into a large quantity of files.Several platform are being developed to hide this complexity to users and provide a seamless access to climate data, as well as to provide on-demand data analysis capabilities. We can cite, for example, the Copernicus Data Store (CDS https://cds.climate.copernicus.eu), along with its toolbox to perform online data analysis. Another platform is developed within the H2020 IS-ENES3 project, called climate4impact (C4I 2.0 https://dev.climate4impact.eu ). It is using an enhanced Jupyter-Lab environment called SWIRRL (Software for Interactive Reproducible Research Labs https://gitlab.com/KNMI-OSS/swirrl ) along with a collection of Jupyter notebooks (https://gitlab.com/is-enes-cdi-c4i/notebooks) as useful set of example on how to use the data.Finally, the portal provides interactive pages for the evaluation of climate models (using ESMValTool) to guide users on selecting climate datasets. The notebooks that can be executed in C4I, are developed using a very convenient software library, which is made available via SWIRRL, to calculate climate indices and indicators called icclim (v5.0 https://github.com/cerfacs-globc/icclim ). This library, which is also in the process of being integrated into the C3S, is a flexible python software package to calculate climate indices and indicators. This tool adhere as much as possible to metadata conventions such as the Climate & Forecasting Conventions (CF-1.x) as well as the clix-meta (https://github.com/clix-meta) work that is being done in IS-ENES3. Proper provenance information still needs to be added. The ultimate goal is to be as close as possible to all FAIR aspects. icclim is designed with performance and optimisation in mind, because the goal is to provide on-demand calculations for users. It provides the implementation of most of the international standard climate indices such as ECAD, ETCCDI, ET-SCI, including the correct methodology for calculating percentile indices using the bootstrapping method. It has been validated against R.Climdex as well (https://cran.r-project.org/web/packages/climdex.pcic/index.html). This new 5.x version of icclim is based on functions from the xclim (https://github.com/Ouranosinc/xclim) python library, which was inspired by earlier versions of icclim, but using xarray and dask for data access and processing.In this presentation, the climate4impact 2.0 platform will be described along with the icclim climate indices tool. Important metadata aspects will also be discussed (clix-meta). A few examples using the jupyter notebook collection will be shown. This project (IS-ENES3) has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement N°824084.

Researchers and end users using climate data face a challenge when they analyze the data they need. Data volumes are increasing very rapidly, and the ability to download all needed data is often no longer possible. Most of the climate analysis tools for research and application needs must use very large datasets, often distributed among several data centres and into a large quantity of files. This is especially true when they are stored in a federated architecture like the ESGF. One of these tools is icclim (https://github.com/cerfacs-globc/icclim ), a flexible python software package to calculate climate indices and indicators. This tool adhere as much as possible to metadata conventions such as CF, implementing also provenance information. It also aims at providing increasing support for all FAIR aspects. It is designed with performance and optimisation in mind, because the goal is to provide on-demand calculations for users. It provides the implementation of most of the international standard climate indices such as ECAD, ETCCDI, ET-SCI, including the correct methodology for calculating percentile indices using the bootstrapping method. It has been validated against R.Climdex as well (https://cran.r-project.org/web/packages/climdex.pcic/index.html ). The new 5.x version of icclim is now based on functions from the xclim python library, which was inspired by earlier versions of icclim, but using xarray and dask for data access and processing. icclim is also a candidate as the software to calculate climate indices for the C3S toolbox (https://cds.climate.copernicus.eu/cdsapp#!/toolbox ). An example of a complex analysis tool used in climate research and adaptation studies is a tool to follow storm tracks. In the context of climate change, it is important to know if storm tracks will change in the future, in both their frequency and intensity. Storms can cause significant societal impacts, hence it is important to assess future patterns. These tools are integrated in the IS-ENES C4I 2.0 platform (https://climate4impact.eu/ ), using a Jupyter notebook collection in a SWIRRL environment (Software for Interactive Reproducible Research Labs https://gitlab.com/KNMI-OSS/swirrl ). Having access to this type of complex analysis tool is very useful, and integrating them with front-ends like C4I enable the use of those tools by a larger number of researchers and end users. This project (IS-ENES3) has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement N°824084.

Researchers and end users using climate data face a challenge when they analyze the data they need. Data volumes are increasing very rapidly, and the ability to download all needed data is often no longer possible. Most of the climate analysis tools for research and application needs must use very large datasets, often distributed among several data centres and into a large quantity of files. This is especially true when they are stored in a federated architecture like the ESGF. One of these tools is icclim (https://github.com/cerfacs-globc/icclim ), a flexible python software package to calculate climate indices and indicators. This tool adhere as much as possible to metadata conventions such as CF, implementing also provenance information. It also aims at providing increasing support for all FAIR aspects. It is designed with performance and optimisation in mind, because the goal is to provide on-demand calculations for users. It provides the implementation of most of the international standard climate indices such as ECAD, ETCCDI, ET-SCI, including the correct methodology for calculating percentile indices using the bootstrapping method. It has been validated against R.Climdex as well (https://cran.r-project.org/web/packages/climdex.pcic/index.html ). The new 5.x version of icclim is now based on functions from the xclim python library, which was inspired by earlier versions of icclim, but using xarray and dask for data access and processing. icclim is also a candidate as the software to calculate climate indices for the C3S toolbox (https://cds.climate.copernicus.eu/cdsapp#!/toolbox ). icclim is integrated in the IS-ENES C4I 2.0 platform (https://climate4impact.eu/ ), using a Jupyter notebook collection in a SWIRRL environment (Software for Interactive Reproducible Research Labs https://gitlab.com/KNMI-OSS/swirrl ). Having access to this type of analysis tool is very useful, and seamless integration with front-ends like C4I enable the use of those tools by a larger number of researchers and end users. This project (IS-ENES3) has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement N°824084.

Modern interactive tools for data analysis and visualisation are designed to expose their functionalities as a service through the Web. We present in this paper a Web API (SWIRRL) that allows Virtual Research Environments (VREs) to easily integrate such tools in their websites and re-purpose them to their users. The API deals, on behalf of the clients, with the underlying complexity of allocating and managing resources within a target cloud platform. By combining storage and containerised services, offering analysis notebooks and other visualisation software, the API creates dedicated working sessions on-demand, which can be accessed collaboratively. Thanks to the API's support for workflow execution, SWIRRL workspaces can be automatically populated with data of interest collected from external data providers. The system keeps track of updates and changes affecting the data and the tools by adopting versioning and standard provenance technologies. Users are provided with interactive controls enabling traceability and recovery actions, including the possibility of creating executable snapshots of their environments. SWIRRL is built in cooperation with two research infrastructures in the field of solid earth science and climate data modeling. We report on the particular adoptions and use cases.

The FAIR principles have been accepted globally as guidelines for improving data-driven science and data management practices, yet the incentives for researchers to change their practices are presently weak. In addition, data-driven science has been slow to embrace workflow technology despite clear evidence of recurring practices. To overcome these challenges, the Canonical Workflow Frameworks for Research (CWFR) initiative suggests a large-scale introduction of self-documenting workflow scripts to automate recurring processes or fragments thereof. This standardised approach, with FAIR Digital Objects as anchors, will be a significant milestone in the transition to FAIR data without adding additional load onto the researchers who stand to benefit most from it. This paper describes the CWFR approach and the activities of the CWFR initiative over the course of the last year or so, highlights several projects that hold promise for the CWFR approaches, including Galaxy, Jupyter Notebook, and RO Crate, and concludes with an assessment of the state of the field and the challenges ahead.

Climate impact and adaptation measures are becoming urgent to be put in place and anticipated. During the past years, climate change effects have been producing adverse conditions in many parts of the world, with significant societal and financial impacts. Advanced analysis tools are needed to process ensembles of simulations of the future climate, in order to generate useful and tailored products for end users.An example of a complex analysis tool used in climate research and adaptation studies is a tool to follow storm tracks. In the context of climate change, it is important to know how storm tracks will change in the future, in both their frequency and intensity. Storms can cause significant societal impacts, hence it is important to assess future patterns. Having access to this type of complex analysis tool is very useful, and integrating them with front-ends like the IS-ENES climate4impact (C4I) can enable the use of those tools by a larger number of researchers and end users.Integrating this type of complex tool is not an easy task. It requires significant development effort, especially if one of the objectives is also to adhere to FAIR principles. The DARE Platform enables research developers to faster develop the implementations of scientific workflows more rapidly. This work presents how such a complex analysis tool has been implemented to be easily integrated with the C4I platform. The DARE Platform also provides easy access to e-infrastructure services like EUDAT B2DROP, to store intermediate or final results and powerful provenance-powered tools to help researchers manage their work and data.This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreements N°824084 and N°777413.

Modern interactive tools for data analysis and visualisation are designed to expose their functionalities as a service through the web. We present an open source web API (SWIRRL) that allows Science Gateways to easily integrate such tools in their websites and re-purpose them to their users. The API, developed in the context of the ENVRIFair and IS-ENES3 EU projects, deals on behalf of the clients with the underlying complexity of allocating and managing resources within a target container orchestration platform on the cloud. By combining storage and third parties' tools, such as JupyterLab and the Enlighten visualisation software, the API creates dedicated working sessions on-demand. Thanks to the API’s staging workflows, SWIRRL sessions can be populated with data of interest collected from external data providers. The system is designed to offer customisation and reproducibility thanks to the recording of provenance, which is performed for each method of the API’s affecting the session. This is implemented by combining a PROV-Templates catalogue and a graph database, which are deployed as independent microservices. Notebooks can be customised with new or updated libraries, and the provenance of such changes is then exposed to users via the SWIRRL interactive JupyterLab extension. Here, users can control different types of reproducibility actions. For instance, they can restore the libraries and data used within the notebook in the past, as well as creating snapshots of the running environment. This allows users to share and rebuild full Jupyter workspaces, including raw data and user generated methods. Snapshots are stored to Git as Binder repositories, thereby compatible with mybinder.org. Finally, we will discuss how SWIRRL is and will be adopted by existing portals for Climate analysis (Climate4Impact) and for Solid Earth Science (EPOS), where advanced data discovery capabilities are combined with customisable, recoverable and reproducible workspaces.

This report provides an overview of the second and final release of the DARE Lineage Services.