Tranferring Data

We recommend two approaches for transferring data to and from the cluster.

  1. Secure Copy through transfer node
  2. Globus Connect

Each are most suitable for different situations. We explain the usage of each tool below.

Secure Copy

TODO: Are all network storage filesystems mounted to the transfer nodes?

Secure copy is ideal for transferring small amounts of data and is a bit more straight forward to use than Globus connect. A secure copy can be performed with the scp command. This command can only be used to transfer data to and from a transfer node. There are two transfer nodes.

  1. transfer.cam.uchc.edu This will accept password authentication or ssh keys and is available from anywhere.
  2. transfer.hpc.cam.uchc.edu This is actually three load-balanced servers. it will only accept SSH key authentication and can only be reached through the CAM VPN.

A secure copy can be performed from the terminal with the scp command which has this required syntax:

scp <src> <dest>

The src and dest arguments specify file or directory paths.

Any remote file path must be prefixed with a user ID along with the transfer node address and separated from the path with a colon like so:

<user>@<hostname>:.

In order to secure copy a directory use the -r option.

Examples

Copy a file from your local computer to your home directory on the cluster:

scp <local path> <user>@transfer.cam.uchc.edu:<remote path>

Copy a directory from the cluster to your local computer:

scp -r <user>@transfer.cam.uchc.edu:<remote path> <local path>

Globus Connect

Globus Connect is well suited for large file transfers and for transferring between institutions such as when you need to share files with a collaborator.

Connecting with Globus Connect

  1. Log in to the Globus web app at https://www.globus.org.
  2. Select the University of Connecticut as your organization and enter your login credentials when prompted.
  3. Once logged in, connect to UConn HPC by searching for “UCHC Globus Server” in the collection text field.
  4. Select UCHC Globus Server and log in with your CAM credentials.
Note

Globus Connect also has a desktop client available for installation on your local computer.

This can be downloaded at https://www.globus.org/globus-connect-personal

Finding files in UCHC Globus Server collection

There are two root directories within the globus server collection. You can navigate the server file system by entering paths within these two directories into the path text field.

/globus

This Globus file server directory contains many of the directories located in root directory of the cluster file system. From here you can access the /labs, /projects, or /seqdata cluster file system directories among others.

/home

This Globus file server directory contains the /home cluster file system directory. You would access your home directory by navigating to /home/FCAM/<user> in the Globus file server.