Blue Cloud - D2.7: Blue Cloud Architecture (Release 2)

This deliverable D2.7 describes the Blue Cloud architecture as it is known at Month 20. It is the second release of the architecture and an update of the earlier 1st release of the Blue-Cloud architecture document D2.6. It is expected that there will be further developments and refinements to the Blue Cloud system. For that purpose, one more release of the architecture document is planned, namely report D2.8 in Month 27. In order to make it easier for readers and reviewers, a table is included as part of Chapter 1, which indicates the elements and sections of this Deliverable 2.7, which have been updated or added in comparison with the earlier Deliverable 2.6. The technical framework of the pilot Blue-Cloud features: 1) the Blue Cloud Data Discovery and Access service component to serve federated discovery and access to blue data infrastructures 2) the Blue Cloud Virtual Research Environment (VRE) component to provide a Blue Cloud VRE as a federation of computing platforms and analytical services. In this report, the current architecture and functionalities of each of these components are described in detail as well as the roles of partners that are developing and hosting modules. The Blue Cloud Data Discovery and Access service architecture is based upon a combination of the DAB metadata broker service of CNR-IIA, and the SeaDataNet CDI service modules as developed by MARIS, IFREMER, and EUDAT in the framework of the EU SeaDataCloud project. For the Blue-Cloud Data Discovery and Access service and its modules, additional developments were needed such as adapting and upgrading of existing services, adding new services, testing modules, integrating modules, and testing the integrated service, in order to achieve the planned functionality. The Blue Cloud VRE is largely based upon the existing D4Science e-infrastructure as developed and managed by CNR-ISTI. This e-infrastructure from the start hosted already multiple Virtual Labs and offered a variety of services. These services have been adopted and adapted for the Blue Cloud and new services have been added, while also new Virtual Labs have been constructed and deployed as part of the Blue-Cloud Demonstrators. The D4Science e-infrastructure also already had proven solutions for connecting to external computing platforms and means for orchestrating distributed services, which are instrumental for smart connections to the other e-infrastructures in the Blue-Cloud system. The Blue Cloud demonstrators are developed as Virtual Labs embedded in the D4Science VRE e- infrastructure and supported by data input from the Blue Cloud Data Discovery and Access service and other data resources, and additional computing services. The demonstrators are worked out in a cooperation between WP3 and WP4, analysing their scientific workflows and technical set-up, and considering the D4Science VRE infrastructure and services that provide the basis platform. As part of their development, the demonstrators have required upgrading of existing functionality and development of additional services. This is described, where relevant, in this document. In addition, consideration is given to integration aspects, such as two-way linking between the 2 Blue Cloud components, and expanding the VRE with additional platforms for computing and algorithms, and where needed for specific demonstrators, direct access to data infrastructures. Moreover, aspects of authentication and monitoring are considered on full Blue Cloud scale. The Blue Cloud architecture as described in this report, is designed to be scalable and sustainable for near-future expansions, such as connecting additional blue data infrastructures, implementing more and advanced blue analytical services, configuring more dedicated Virtual Labs, and targeting more (groups of) users.