Data Movement

Workshop Instructor: Ron Merrill, Information Technology Services – Research Cyberinfrastructure, University of Hawaiʻi System

Description
This workshop will be an introduction to understanding the challenges and options in moving scientific data over the network. In particular, attendees will learn about some of the different network infrastructure and tools available and the use cases to apply them towards and highlight any disadvantages or drawbacks to a particular technology. Lessons will also address what to do if a transfer is experiencing less than expected performance, potential common contributors to a transfer bottleneck and when and who to ask for assistance will also be covered. Different data transfer tools like SFTP, rsync and Globus for moving data will be covered in hands-on exercises. Attendees can expect to transfer data using Globus and Rclone during the workshop.

Prerequisites
See Readings "Preparation" and "Install Rclone".
Participants would also need to have an understanding of Koa and Open On Demand.
https://change-hi.github.io/morea/dat...
https://change-hi.github.io/morea/dat...

Learning objectives
By the end of this workshop attendees will be able to:
Understand appropriate solutions for a data transfer workflow, taking into account current computing, storage and network infrastructure, and end-point sites.
Transfer data using Rclone.
Transfer data using Globus.

Tools Used for this workshop
Koa Open OnDemand