Architecture overview

EUBrazil Cloud Connect applications involve complex workflows and access to huge datasets. Thanks to the collaboration between European and Brazilian research centres the project leverages a set of components for the use of supercomputing, private cloud and cloud opportunistic resources. Cloud Connect will expose these resources through programming frameworks and scientific gateways, easing the adaptation and deployment of the applications that use data and computing resources in both sides of the Atlantic.

All this constitutes the EUBrazil Cloud Connect einfrastructure.

The architecture at a glance

The architecture integration

The integration of the components takes into account existing standards to maximize interoperability with provisioning systems and existing infrastructures. Tools for transparent access to supercomputers such as CSGrid are combined with execution frameworks such as COMPSs in order to deploy applications on multiple private and public cloud resources and with complex workflow managers that run on top of different infrastructures through e-Science Central. EUBrazilCC includes the parallel data analysis service (PDAS) for big data analytics, the Fogbow middleware to federate opportunistic resources and the mc2 framework for scientific gateways.

The programming frameworks & execution environments

Both e-Science Central (e-SC) and COMPSs provide programmatic access through APIs to the workflow management and execution functionalities that are provided by these tools. Those APIs are part of the programming frameworks layer and are available to the end-user applications. At the same time, both e-Science Central and COMPSs provide the necessary bridges to the execution environments like CSGrid and PDAS that leverage on the available resource management mechanisms to allocate computing and storage resources from the different EUBrazilCC infrastructure providers. The resource management mechanisms can be systems like IM+VMRC or cloud standards such as OCCI.

The data sources

The lowest layer consists of the data sources that are integrated in the infrastructure via different protocols and mechanisms that are specific to the data provider. For use case 1, EUBrazil Cloud Connect will retrieve sequences from existing Leishmania and Sand Fly collections such as ISCIII-WHO-CCL collection, COLFLEB and CLIOC, PubMed and GenBank. For Use Case 2 on Cardiovascular Simulation EUBrazil Cloud Connect will integrated in the cloud environment clinical data coming from hospitals and biomedical research centres in Brazil and Europe.  For Use Case 3 different data sources, such as Meteorological data from land surface monitoring stations, satellite images and LiDAR data, will be used for the analysis of biodiversity impacts of climate change (more here).