Search

This deliverable D2.8 describes third and final release of the Blue-Cloud architecture as it is at Month 33 (June 2022) and it is and an update of the earlier 2nd release of the Blue-Cloud architecture document D2.7. Given the agreed extension allowed to the Blue-Cloud project until March 2023, there might be some further refinements to the architecture in the upcoming 9 months to allow to optimise some of its services to better respond to user needs. In order to make it easier for readers and reviewers, a table is included as part of Chapter 1, which indicates the elements and sections of this Deliverable 2.8, which have been updated or added in comparison with the earlier Deliverable 2.7. In this report, the current architecture and functionalities of each of the following components, part of the Blue-Cloud technical framework, are described in detail as well as the roles of partners that are developing and hosting modules: 1) the Blue-Cloud Data Discovery and Access service to serve federated discovery and access to blue data infrastructures; 2) the Blue-Cloud Virtual Research Environment (VRE) to provide a Blue-Cloud VRE as a federation of computing platforms and analytical services. The Blue-Cloud Data Discovery and Access service architecture is based upon a combination of the DAB metadata broker service of CNR-IIA, and the SeaDataNet CDI service modules as developed by MARIS, IFREMER, and EUDAT in the framework of the EU SeaDataCloud project. For the Blue-Cloud Data Discovery and Access service and its modules, additional developments were implemented in the period May 2021 - June 2022 such as adapting and upgrading existing services, adding new services, testing modules, integrating modules, and testing the integrated service, in order to achieve the planned functionality. The Blue-Cloud VRE is largely based upon the D4Science e-infrastructure as earlier developed and managed by CNR-ISTI [1]. This e-infrastructure hosted, already from the start, multiple Virtual Labs and offered a variety of services. These services have been adopted and adapted for Blue-Cloud, new services have been added and several original services have been upgraded. Moreover, new Virtual Labs have been constructed and deployed as part of the Blue-Cloud Demonstrators. The D4Science e- infrastructure also already included proven solutions for connecting to external computing platforms and means for orchestrating distributed services, which are instrumental for smart connections to the other e-infrastructures in the Blue-Cloud system, while further evolutions have taken place as part of the Blue-Cloud project, in response to the needs of the Virtual Labs and their users. With respect to the Virtual Labs, they are developed as real-life demonstrators embedded in the D4Science VRE and supported by data input from the Blue-Cloud Data Discovery and Access service, other data resources and additional computing services. They have been worked out in cooperation between WP3 and WP4 which have analysed their scientific workflows and identified the best technical set-up considering the D4Science VRE infrastructure and services. As part of their development, the demonstrators required upgrading of existing functionality and development of additional services. This is described, where relevant, in this document. In addition, consideration is given to integration aspects, such as two-way linking between the Blue- Cloud components, expanding the VRE with additional platforms for computing and algorithms, and direct access to data infrastructures where needed for specific Virtual Labs. Moreover, aspects of authentication and monitoring are considered on a full Blue-Cloud scale. The Blue-Cloud architecture as described in this report, is designed to be scalable and sustainable for near-future expansions, such as connecting additional blue data infrastructures, implementing more and advanced blue analytical services, configuring more dedicated Virtual Labs, and targeting more (groups of) users.

This deliverable D2.7 describes the Blue Cloud architecture as it is known at Month 20. It is the second release of the architecture and an update of the earlier 1st release of the Blue-Cloud architecture document D2.6. It is expected that there will be further developments and refinements to the Blue Cloud system. For that purpose, one more release of the architecture document is planned, namely report D2.8 in Month 27. In order to make it easier for readers and reviewers, a table is included as part of Chapter 1, which indicates the elements and sections of this Deliverable 2.7, which have been updated or added in comparison with the earlier Deliverable 2.6. The technical framework of the pilot Blue-Cloud features: 1) the Blue Cloud Data Discovery and Access service component to serve federated discovery and access to blue data infrastructures 2) the Blue Cloud Virtual Research Environment (VRE) component to provide a Blue Cloud VRE as a federation of computing platforms and analytical services. In this report, the current architecture and functionalities of each of these components are described in detail as well as the roles of partners that are developing and hosting modules. The Blue Cloud Data Discovery and Access service architecture is based upon a combination of the DAB metadata broker service of CNR-IIA, and the SeaDataNet CDI service modules as developed by MARIS, IFREMER, and EUDAT in the framework of the EU SeaDataCloud project. For the Blue-Cloud Data Discovery and Access service and its modules, additional developments were needed such as adapting and upgrading of existing services, adding new services, testing modules, integrating modules, and testing the integrated service, in order to achieve the planned functionality. The Blue Cloud VRE is largely based upon the existing D4Science e-infrastructure as developed and managed by CNR-ISTI. This e-infrastructure from the start hosted already multiple Virtual Labs and offered a variety of services. These services have been adopted and adapted for the Blue Cloud and new services have been added, while also new Virtual Labs have been constructed and deployed as part of the Blue-Cloud Demonstrators. The D4Science e-infrastructure also already had proven solutions for connecting to external computing platforms and means for orchestrating distributed services, which are instrumental for smart connections to the other e-infrastructures in the Blue-Cloud system. The Blue Cloud demonstrators are developed as Virtual Labs embedded in the D4Science VRE e- infrastructure and supported by data input from the Blue Cloud Data Discovery and Access service and other data resources, and additional computing services. The demonstrators are worked out in a cooperation between WP3 and WP4, analysing their scientific workflows and technical set-up, and considering the D4Science VRE infrastructure and services that provide the basis platform. As part of their development, the demonstrators have required upgrading of existing functionality and development of additional services. This is described, where relevant, in this document. In addition, consideration is given to integration aspects, such as two-way linking between the 2 Blue Cloud components, and expanding the VRE with additional platforms for computing and algorithms, and where needed for specific demonstrators, direct access to data infrastructures. Moreover, aspects of authentication and monitoring are considered on full Blue Cloud scale. The Blue Cloud architecture as described in this report, is designed to be scalable and sustainable for near-future expansions, such as connecting additional blue data infrastructures, implementing more and advanced blue analytical services, configuring more dedicated Virtual Labs, and targeting more (groups of) users.