FAQ

How Do I …?

… Transfer Data To/From The Cluster?

Data can be copied to and from the cluster using scp, or rsync. We strongly recommend rsync due to the many quality of life features.

# Copy local to cluster
rsync -r --progress --compress --bwlimit=10240 <local_source_path> <username>@login001.elysium.hpc.rub.de:<remote_destination_path>

# Copy cluster to local
rsync -r --progress --compress --bwlimit=10240 <username>@login001.elysium.hpc.rub.de:<remote_source_path> <local_destination_path>

The paths to the data which shall be copied and the destination, as well as the username need to be adjusted. Note that there is no trailing “/” at the end of the source path. If there was one, the directories contents, not the directory itself would be copied.

Flags:

-r enables recursive copies (directories and their content)
--progress gives you a live update about the amount that has been copied already and an estimate of the remaining time
--compress attempts to compress the data on the fly to speed up the data transfer even more
--bwlimit limits the data transfer rate in order to leave some bandwidth to other people who want to copy data, or work interactively.

If multiple file are to be copied to/from the cluster, the data should be packed into a tar-archive before sending:

# create a tar archive
tar -cvf myfiles.tar dir_or_files.*

# extract a tar archive
tar -xvf myfiles.tar

Note that running multiple instances of rsync, or scp will not speed up the copy process, but slow it down even more!

FAQ

How Do I …?

… Transfer Data To/From The Cluster?

More answers coming soon.