Transparent data movement from EGI to EUDAT

The main goal of the EOSC-hub project is to synergize the services and infrastructures from major European providers including EGI and EUDAT, enabling researchers to freedom of choice of the services, data and infrastructure necessary for their work.  Users from many communities have reported interest in being able to use resources provisioned by these 2 infrastructures simultaneously in a transparent fashion, i.e.:

• using single sign on mechanisms between infrastructures
• being able to transparently transfer data between infrastructures

EOSC-hub WP6 has been endeavoring to enable transparent data access between the infrastructures. The main data management components in EGI and EUDAT are respectively, EGI-DataHub based on Onedata platform and B2SAFE/B2STAGE based on iRODS. Currently no transparent mechanisms allowing users to move data between these infrastructures exists, the only options are manual transfers using such tools as GridFTP.

These two demo videos present the managment of data transfer across EGI, EUDAT, INDIGO data and compute services. 

The first demo video shows the how data can be transparently moved from EGI to EUDAT. It divided into 4 sections:

• Registration of B2STAGE storage in EGI-DataHub
• Creation of dsata space in EGI-DataHub supported by the B2STAGE storage
• Upload of some data to the data space
• Data access from GUI and command line

The second demo video presents import of existing data from EUDAT to EGI. The demo is divided into the following 4 sections:

• Registration of B2STAGE storage in EGI-DataHub
• Creation of data space in EGI-DataHub supported by the B2STAGE storage from which existing data is imported to EGI-DataHub
• Access of data from B2STAGE using EGI-DataHub user interface
• Automatic discovery of new data added on the B2STAGE storage