HOWTO: Transfer files using Globus Connect

Globus Connect Logo

Globus Connect is a reliable, high-performance file transfer platform allowing users to transfer large amounts of data seamlessly between systems.  It aims to make transfers a "click-and-forget" process by setting up configuration details in the background and automating fault recovery.  

Globus can be used for both file transfers between OSC and:

  • A computing institution with Globus installed (check with your site provider for availability) or
  • A personal computer (known as a personal endpoint)

Users transferring between OSC and another computing institution with​ Globus installed do not need to install Globus Connect Personal, and can skip to Usage.

More on how Globus works can be found on the Globus "How It Works" page.

If you are looking to transfer smaller sized files you can utilize OnDemand's file transfer capabilities, or use a SFTP client to connect to  sftp.osc.edu . Our general recommendation is that for small files - measured in MB to several hundred MB - to use OnDemand or SFTP. You can continue to use SFTP and get reasonable performance up to file sizes of several GB. For transfers of several GB or larger, you should consider using Globus Online.

Install Globus Connect Personal

To use Globus to transfer from a personal computer, you will need to install the Globus Connect Personal client on your computer following the steps below. Those transferring between OSC and another computing institution can skip to Usage.

  1. Sign up for a free Globus account
  2. Download the Globus Connect Personal Client 
    • Click "Manage Endpoints" under the "Manage Transfers/Data" menu
    • Click "add Globus Connect" on the top-right of the page
    • Choose a unique name for your endpoint and generate the setup key
    • Download the Globus Connect client for your operating system
  3. Install Globus Connect Personal Client
    • Windows
      1. Run the Installer
      2. Copy-Paste the setup key to complete the installation
    • Mac​​
      1. Mount your drives
      2. Copy the Globus Client to your application Folder
      3. Start The Globus Client, and enter the provided setup key
    • Linux
      1. Un-tar the .tgz file with the command tar -zxvf
      2. Run globusconnect , found within the unzipped directory
      3. Copy-Paste the setup key when prompted
  4. (Optional) Changing directories accessible to Globus

By default Globus will only add certain default folders to the list of files and directories accessible by Globus. To change/add/remove files and directories from this list:

Windows

  1. Start Globus Connect Personal
  2. Go to "Options"
  • Add directories/files using the  "+" button
  • Remove directories/files using the "-" button
  • Revert to the default accessible directories/files using the "Reset to Defaults" button
  • Any changes you make are not made permanent until you press the "Save" button

​​Mac

  1. Start Globus Connect Personal
  2. Go to Preferences -> Access
  • Add directories/files using the  "+" button
  • Remove directories/files using the "-" button
  • Revert to the default accessible directories/files using the "Reset to Defaults" button
  • Any changes you make are not made permanent until you press the "Save" button

​Linux​

  1. Configure the   ~/.globusonline/lta/config-paths  file. This file is a plain text file, with each line corresponding to the configuration of a particular directory path you wish to make accessible. Each line consists of 3 comma separated fields as below 
    <path1>,<sharing flag>,<R/W flag>
<path2>,<sharing flag>,<R/W flag>
<path3>,<sharing flag>,<R/W flag>
...

    Path: an absolute directory/path to be permitted. A leading tilde "~" can be used to represent the home directory of the user that runs globusconnectpersonal.

    Sharing Flag: it controls sharing, with a value of "1" allowing sharing for the path and a value of "0" disallowing sharing for the path.

    R/W Flag: it determines whether the path will be accessible read-only or read/write, with a "1" permitting read/write access and a "0" marking the path as read-only. 

  2. After making a change to the ~/.globusonline/lta/config-paths file, you must stop and restart Globus Connect Personal before the changes will take effect as below
$ ./globusconnectpersonal -stop
$ ./globusconnectpersonal -start &

Usage

  1. Login to Globus and navigate to the "Transfer Files" under the "Manage Data" menu
  2. Enter your end point in one of the boxes
    • If transferring to a computer with Globus Connect Personal installed, this will be the unique name chosen during installation. It is also listed under "Administered by Me"
  3. Enter osc#gcs (OSC Globus Connect Server) as the other endpoint
    • Enter your OSC username and password for the authetication
  4. You can now transfer files and directories both ways by selecting them and pressing the arrow indicating which way you'd like to transfer
If you are doing a large transfer you should transfer to/from the parallel file system or project space for best performance.
Once a transfer has begun, you do not need to keep the Globus webpage up, but you will need to make sure the Globus Connect Personal Client is running on your computer until it has completed.  If the transfer is interrupted for any reason, Globus will attempt to re-initiate the transfer automatically.

Further Reading