HOWTO: Transfer files using Globus

Globus is a reliable, high-performance file transfer platform allowing users to transfer large amounts of data seamlessly between systems. It aims to make transfers a "click-and-forget" process by setting up configuration details in the background and automating fault recovery.  

Globus can be used for both file transfers between OSC and:

  • A computing institution with Globus installed (check with your site provider for availability) or
  • A personal computer (known as a personal endpoint)

Users transferring between OSC and another computing institution with​ Globus installed do not need to install Globus Connect Personal, and can skip to Usage.

If you are looking to transfer smaller sized files you can utilize OnDemand's file transfer capabilities, or use an SFTP client to connect to  sftp.osc.edu . Our general recommendation is that for small files - measured in MB to several hundred MB - to use OnDemand or SFTP. You can continue to use SFTP and get reasonable performance up to file sizes of several GB. For transfers of several GB or larger, you should consider using Globus.

Install Globus Connect Personal

To use Globus to transfer from a personal computer, you will need to install the Globus Connect Personal client on your computer following the steps below. Those transferring between OSC and another computing institution can skip to Usage.

  1. Login to Globus. If your institution does not have an organizational login, you may choose to either Sign in with Google or Sign in with ORCiD iD. b.png                                              
  2. Download the Globus Connect Personal Client 
    • In the main Globus web page, click "ENDPOINTS" on the sidebar.
    • Click "Create a personal endpoint" on the top-right of the pageendpoint_new.png
    • Choose a unique name for your endpoint and generate the setup key. Download the Globus Connect client for your operating systemcreate.png
  3. Install Globus Connect Personal Client
    • Windows
      1. Run the Installer
      2. Copy-Paste the setup key to complete the installation
    • Mac​​
      1. Mount your drives
      2. Copy the Globus Client to your Application Folder
      3. Start The Globus Client, and enter the provided setup key
    • Linux
      1. Un-tar the .tgz file with the command tar -zxvf
      2. Run globusconnect , found within the unzipped directory
      3. Copy-Paste the setup key when prompted
  4. (Optional) Changing directories accessible to Globus

By default, Globus will only add certain default folders to the list of files and directories accessible by Globus. To change/add/remove files and directories from this list:

Windows

  1. Start Globus Connect Personal
  2. Go to "Options"
  • Add directories/files using the  "+" button
  • Remove directories/files using the "-" button
  • Revert to the default accessible directories/files using the "Reset to Defaults" button
  • Any changes you make are not made permanent until you press the "Save" button

​​Mac

  1. Start Globus Connect Personal
  2. Go to Preferences -> Access
  • Add directories/files using the  "+" button
  • Remove directories/files using the "-" button
  • Revert to the default accessible directories/files using the "Reset to Defaults" button
  • Any changes you make are not made permanent until you press the "Save" button

​Linux​

  1. Configure the   ~/.globusonline/lta/config-paths  file. This file is a plain text file, with each line corresponding to the configuration of a particular directory path you wish to make accessible. Each line consists of 3 comma-separated fields as below 
    <path1>,<sharing flag>,<R/W flag>
<path2>,<sharing flag>,<R/W flag>
<path3>,<sharing flag>,<R/W flag>
...

    Path: an absolute directory/path to be permitted. A leading tilde "~" can be used to represent the home directory of the user that runs globusconnectpersonal.

    Sharing Flag: it controls sharing, with a value of "1" allowing sharing for the path and a value of "0" disallowing sharing for the path.

    R/W Flag: it determines whether the path will be accessible read-only or read/write, with a "1" permitting read/write access and a "0" marking the path as read-only. 

  2. After making a change to the ~/.globusonline/lta/config-paths file, you must stop and restart Globus Connect Personal before the changes will take effect as below
$ ./globusconnectpersonal -stop
$ ./globusconnectpersonal -start &

Usage

We provide instructions on how to transfer data using Globus Connect Server v5 on this page. If you would like to share your data with your collaborators, please see this page. 

If you are doing a large transfer you should transfer to/from the parallel file system for best performance. 
  1. Login to Globus. If your institution does not have an organizational login, you may choose to either Sign in with Google or Sign in with ORCiD iD.b.png

  2. If you do not log in with your OSC HPC credentials, map your OSC credentials to your Globus account following this page. Select 'Ohio Supercomputer Center (OSC)' as the identity provider. You will be redirected to the page below. Provide your OSC HPC credentials: OSC_login.png

  3. Navigate to the "File Manager" on the sidebar. Select the two-panel view by switching the icon beside "Panels" on the top-right of the page if the view is one panel switch.png
  4. Enter your endpoint in one of the collections
    • If transferring to a computer with Globus Connect Personal installed, make sure to have Globus Connect Personal running. The name will be the unique name chosen during installation. It is also listed under "Administered by You" under the "ENDPOINTS" on the sidebar
  5. Refer to the table below to search the other endpoint (please copy and paste all keywords for accurate search)
    •   Endpoint Keywords to search
      OSC's home directory OSC $HOME OSC,home,netapp
      OSC's project directory OSC /fs/project OSC,project,gpfs
      OSC's scratch directory OSC /fs/scratch OSC,scratch,gpfs
      OSC's ess storage OSC /fs/ess OSC,ess,gpfs
      AWS S3 storage OSC S3 OSC,s3
    • If needed, enter your username and password for the authentication
    • You should see the files in the collection. You can also change the directory by entering the appropriate path in the Path boxtransfer.png
  6. You can now transfer files and directories both ways by selecting them and pressing the arrow indicating which way you'd like to transferstart.png
  7. Once you click 'Start', an activity report will appear as shown below, and you can click on it to see the detailsstatus.png
  8. Click "View details', you will get the activity report about the transfer. You will also receive an email once the transfer is completed (successful or failed)
Once a transfer has begun, you do not need to keep the Globus webpage up, but you will need to make sure the Globus Connect Personal Client is running on your computer until it has completed.  If the transfer is interrupted for any reason, Globus will attempt to re-initiate the transfer automatically.