Staging data via Flows
A common use-case for data transfer is staging large volumes on data via the DataHub. To make staging data via DataHub easier for users we created a specific Flow that performs the two stage transfer in one logical workflow.
Globus Flows is a service which allows users to define and execute data workflows using the Globus Subscription attached to servers at MPCDF, such as DataHub. These workflows consist of Actions which can be combined into a single logical operation.
The Flow, depicted in the figure below, is formed form the following 3 actions.
Copy data to DataHub (e.g. from a client on a cluster)
Copy data from DataHub (e.g. to a client on a cluster)
Delete intermeiate data on DataHub after staging has completed
The Flow can be started via the Globus web portal via the link To use the Flow users need to be a member of the “MPCDF Flows Users” group, request access to via the Globus web portal.
The status and results from the flow can be viewed in the Globus web portal either for the whole flow or the individual actions.
A demo video and slidest is available via the following link