How to transfer data

From RCSWiki
Revision as of 20:52, 22 April 2020 by Dmitri (talk | contribs) (→‎Intro)
Jump to navigation Jump to search

General

Linux and MacOS

While you can find transfer programs for MacOS and Linux that have graphical point-and-click interface, both the operating system come with pre-installed (most of the time) command line transfer tools: scp, rsync, sftp. These are powerful and convenient tools that can handle any practical data transfer to and from our compute clusters.

scp -- secure copy


scp copies files between hosts on a network. It uses ssh for data transfer, and uses the same authentication and provides the same security as ssh.


In practical terms scp is a minimal and also sufficient transfer tool to copy files between network connected Unix based computers in a secure manner.

rsync -- Remote SYNCronizer


Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.


In practice, rsync is scp on steroids. It is designed to synchronize two locations, that is to make them the same. So, if a transfer stops for some reason, if one restarts the transfer, rsync will check the destination and only transfers what is needed. This way, you can conveniently restart the transfer at any moment without loosing the progress. With scp this is not an option.

sftp -- secure file transfer protocol


sftp is a file transfer program, similar to ftp, which performs all operations over an encrypted ssh transport. It may also use many features of ssh, such as public key authentication and compression.

Windows

MobaXterm is the recommended tool for remote access and data transfer in Windows OSes.

MobaXterm

MobaXterm is a one-stop solution for most remote access work on a compute cluster or a Unix / Linux server. Along the remote access SSH client and X11 graphics server it provides graphical interface for data transfer operations.