Globus is a reliable, high-performance file transfer platform allowing users to transfer large amounts of data seamlessly between systems. It aims to make transfers a "click-and-forget" process by setting up configuration details in the background and automating fault recovery.  

Globus can be used for both file transfers between OSC and:

• A computing institution with Globus installed (check with your site provider for availability) or
• A personal computer (known as a personal endpoint)

Users transferring between OSC and another computing institution with​ Globus installed do not need to install Globus Connect Personal, and can skip this page.

Install Globus Connect Personal

To use Globus to transfer from a personal computer, you will need to install the Globus Connect Personal client on your computer following the steps below. Those transferring between OSC and another computing institution can skip to Usage.

Watch How to Install Globus Personal

1. Login to Globus. If your institution does not have an organizational login, you may choose to either Sign in with Google or Sign in with ORCiD iD.                                               
2. Download the Globus Connect Personal Client 
   • In the main Globus web page, click "ENDPOINTS" on the sidebar.
   • Click "Create a personal endpoint" on the top-right of the page
   • Download the Globus Connect client for your operating system
3. Install Globus Connect Personal Client
   • Windows
     1. Run the Installer
     2. Copy-Paste the setup key to complete the installation
   • Mac​​
     1. Mount your drives
     2. Copy the Globus Client to your Application Folder
     3. Start The Globus Client, and enter the provided setup key
   • Linux
     1. Un-tar the .tgz file with the command tar -zxvf
     2. Run globusconnect , found within the unzipped directory
     3. Copy-Paste the setup key when prompted
4. Setup Globus Personal
   • The setup process will ask you to login. Click Login
   • This will bring up a browser window with Globus asking for permissions for Globus Personal to have access to. Click "allow" to continue.
   • This will bring up a Globus Personal window up to finish the setup process. Fill out the required information and click "Save".   
5. (Optional) Changing directories accessible to Globus

By default, Globus will only add certain default folders to the list of files and directories accessible by Globus. To change/add/remove files and directories from this list:

Windows

1. Start Globus Connect Personal
2. Go to "Options"

• Add directories/files using the  "+" button
• Remove directories/files using the "-" button
• Revert to the default accessible directories/files using the "Reset to Defaults" button
• Any changes you make are not made permanent until you press the "Save" button

​​Mac

1. Start Globus Connect Personal
2. Go to Preferences -> Access

• Add directories/files using the  "+" button
• Remove directories/files using the "-" button
• Revert to the default accessible directories/files using the "Reset to Defaults" button
• Any changes you make are not made permanent until you press the "Save" button

​Linux​

1. Configure the   ~/.globusonline/lta/config-paths  file. This file is a plain text file, with each line corresponding to the configuration of a particular directory path you wish to make accessible. Each line consists of 3 comma-separated fields as below

   <path1>,<sharing flag>,<R/W flag>
   <path2>,<sharing flag>,<R/W flag>
   <path3>,<sharing flag>,<R/W flag>
   ...
   

   Path: an absolute directory/path to be permitted. A leading tilde "~" can be used to represent the home directory of the user that runs globusconnectpersonal.

   Sharing Flag: it controls sharing, with a value of "1" allowing sharing for the path and a value of "0" disallowing sharing for the path.

   R/W Flag: it determines whether the path will be accessible read-only or read/write, with a "1" permitting read/write access and a "0" marking the path as read-only. 

2. After making a change to the ~/.globusonline/lta/config-paths file, you must stop and restart Globus Connect Personal before the changes will take effect as below

$ ./globusconnectpersonal -stop
$ ./globusconnectpersonal -start &

Globus is a reliable, high-performance file transfer platform allowing users to transfer large amounts of data seamlessly between systems. It aims to make transfers a "click-and-forget" process by setting up configuration details in the background and automating fault recovery.  

Globus can be used for both file transfers between OSC and:

• A computing institution with Globus installed (check with your site provider for availability) or
• A personal computer (known as a personal endpoint). See this page on how to install Globus Connect Personal

Users transferring between OSC and another computing institution with​ Globus installed do not need to install Globus Connect Personal.

Transfer files using Globus

We provide instructions on how to transfer data on this page. If you would like to share your data with your collaborators, please see this page. 

Watch How to Transfer Files Using Globus

1. Login to Globus. If your institution does not have an organizational login, you may choose to either Sign in with Google or Sign in with ORCiD iD.

2.  If you do not log in with your OSC HPC credentials, map your OSC credentials to your Globus account following this page. Select 'Ohio Supercomputer Center (OSC)' as the identity provider. You will be redirected to the page below. Provide your OSC HPC credentials: 

3. Navigate to the "File Manager" on the sidebar. Select the two-panel view by switching the icon beside "Panels" on the top-right of the page if the view is one panel
4. Enter your endpoint in one of the collections
   • If transferring to a computer with Globus Connect Personal installed, make sure to have Globus Connect Personal running. The name will be the unique name chosen during installation. It is also listed under "Administered by You" under the "ENDPOINTS" on the sidebar
5. Enter 'OSC Globus Connect Server' in the other collection box to search all the collections managed by OSC as below. Choose the collection where you want to do the data transfer:

   • 	Endpoint
     OSC's home directory	OSC $HOME
     OSC's project directory	OSC /fs/project
     OSC's scratch directory	OSC /fs/scratch
     OSC's ess storage	OSC /fs/ess
     AWS S3 storage	OSC S3

   • If needed, enter your username and password for the authentication
   • You should see the files in the collection. You can also change the directory by entering the appropriate path in the Path box
6. You can now transfer files and directories both ways by selecting them and pressing the arrow indicating which way you'd like to transfer
7. Once you click 'Start', an activity report will appear as shown below, and you can click on it to see the details
8. Click "View details', you will get the activity report about the transfer. You will also receive an email once the transfer is completed (successful or failed)

Globus Connect Server allows OSC users to share data with their collaborators who do not have OSC HPC account (the collaborator needs to sign up for a free Globus account though). The advantage of data sharing via Globus is that you do not have to move your data in order to share it. You can select directory paths to be securely shared with your collaborator, and grant them read-only or read-write access.

Sharing data with collaborators 

Watch How to Share Files Using Globus

• Login to Globus. If your institution does not have an organizational login, you may choose to either Sign in with Google or Sign in with ORCiD iD
• Map your OSC credentials to your Globus account following this page if you haven't done this before. Select 'Ohio Supercomputer Center (OSC)' as the identity provider from the list. You will be redirected to the page below. Provide your OSC HPC credentials:
• Navigate to the "ENDPOINTS" on the sidebar. Enter 'OSC Globus Connect Server' in the search box to search all the endpointss managed by OSC as below:

• Choose the endpoint where you want to share the data. You will be redirected to the overview page of this endpoint. Click the 'Collections' menu. You will see all of the collections added under this endpoint before. To add a new collection, click 'Add a Collection'. 
• Create a guest collection. (Note: the default path will be $HOME for home directory, /fs/project for project filesystem, /fs/scratch for scratch filesystem, and /fs/ess for ess storage. You can change to a more specific directory by providing the path in ‘Directory’). Click 'Create Collection' to finish the creation.arcum
• Navigate to the "File Manager" on the sidebar. Select the view to be one-panel by switching the icon beside "Panels" on the top-right of the page
• Enter the collection you want to share by clicking 'select a collection': 
• Click 'Your Collections' to choose the right collection 
• Click 'Share' button in the right side pane
• To share with your collaborators, click 'Add Permissions - Share With'. You can specify the directory you want to share, choose to share with users by selecting their email address or Globus username, or share with a group or all Globus users or everyone. You can also select 'Send Email' to send email to your collaborator. If you want your collaborator to also be able to transfer data into the directory, you can also provide write access by checking the 'write' button. Once the options have been selected, click 'Add Permission'. You should be redirected to the information page on this collection, and see the people you have shared it with.
  • You can repeat this process for any number of collaborators.
• To terminate the sharing, you can click the 'trash can' icon next to the invitee on the screen.
• To see all collections you have shared, go to 'Endpoints' in the left bar, then 'Shareable by You'.

Information for your collaborators 

1. Your collaborator needs to sign up for a free Globus account for data sharing
2. Your collaborator needs to install the Globus Connect Personal client on his/her personal computer if he/she needs to download the data. See this page for more information. 
3. If you select 'Send Email' to send an email to your collaborator, he/she will get an email including the URL to access the share. Your collaborator can click on the link and log in with the credentials
4. Your collaborator can also access to the share by logging into Globus. Navigate to the "ENDPOINTS" on the sidebar and click on 'Shared with You'. 

Further Reading

• https://docs.globus.org/how-to/share-files/

Globus Connect Server v5 allows OSC clients to connect to Amazon S3. Please follow the steps below:

Create a New Bucket

• Log into Amazon AWS
• Navigate to Services -> S3
• Click 'Create Bucket'
• Give the bucket a name then click 'Next'
• Click 'Next' and then 'Next' again then 'Create bucket' to create a new bucket

Create a New IAM

• Log into Amazon AWS
• Navigate to Services -> IAM

Add Policy

• Select Policies and then click 'Create Policy'
• Choose the JSON tab and paste the JSON policy as below (It will be necessary to replace the value for 'Resource' to match bucket name from previous steps):

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllBuckets",
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:GetBucketLocation"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Bucket",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": "arn:aws:s3:::osc-globus-test"
        },
        {
            "Sid": "Objects",
            "Effect": "Allow",
            "Action": [
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::osc-globus-test/*"
       }
    ]
}

• Choose 'Review Policy'
• Fill in the name such as 'AllowGlobusS3_osc-globus-test', click 'Create Policy' to create the new policy

Add Group

• Choose 'Groups' in left menu bar then click 'Create New Group'. Provide the group name such as “s3_osc-globus-test” then click 'Next Step' button
• In search bar for Filter type 'Globus' and check box next to 'AllowGlobusS3_osc-globus-test' then click 'Next Step'
• Click 'Create Group' after reviewing new group information. 

Add User

• Choose 'Users' in left menu bar and then click 'Add user'. Provide the username, check 'Programmatic access' and then click 'Next: Permissions'
• Choose 'Add user to group', filter for the group created previous and check box next to 's3_osc-globus-test', click 'Next: Tags' and then 'Next: Review'
• Click 'Create user'
• Make a copy of 'Access key ID' and click 'Show' for 'Secret access key' and save both for future reference
• Click 'Close'

Create a New Collection

• Login to Globus. If your institution does not have an organizational login, you may choose to either Sign in with Google or Sign in with ORCiD iD
• Navigate to the 'ENDPOINTS' on the sidebar and search 'OSC, s3'. Click 'OSC S3' to go to this gateway
• Click the 'Collections' menu. You will see all of the collections added by you before. To add a new collection, click 'Add a Collection'. Enter your OSC username and password for the authentication
• Provide 'Access key' and "Access secret' from previous steps and associate Globus identity with your OSC identity
• Provide bucket name for 'Base Directory'
• Provide the name of the collection in 'Collection Display Name' field
• Click 'Create Collection' to finish the creation
• Click the logo of 'Globus' and navigate back to the 'ENDPOINTS' on the sidebar. Click the 'Administered by You' and then you can locate the new collection you just created. 

Further Reading

• https://docs.globus.org/premium-storage-connectors/aws-s3-gcsv5/

(OPTIONAL) Adding InCommon Authentication 

Adding InCommon authentication to your Globus account allows you to login to Globus Online using your university credentials.  Using this process you can store your Globus username password for safe keeping, and instead use your university username and password to login.  If your already logged in to your university authentication system, logging in to Globus can be as simple as two clicks away.

To use this feature, your university needs to be a InCommon participant.  Some Ohio universities active in InCommon include: Ohio State University, Case Western University, Columbus State Community College, Miami University, Ohio Northern University, Ohio University, University of Findlay, University of Dayton, and many more.  

For a complete list, visit https://incommon.org/participants/ .

To add InCommon Authentication:

1. Login to Globus Online
2. Go to "Manage Identities" under your username
3. Click "Add External Identity"
   • Choose a name for the Identity Settings. 
   • Choose InCommon / CILogon from the drop down menu
4. On the next page, choose your University / Identity Provider

• Click "Remember this selection"
• Click "Log on"
• You may be prompted to login to your university authentication system if you are not already

When you go to login next, click "alternative login" and then "InCommon / CILogon".  Select your university on the next page, and login using your university credentials.  Globus will remember this preference, and automatically prompt you to login using your university authentication next time.

OSC clients who are affiliated with Ohio State can deploy their own endpoint on a server using OSU subscriptions. Please follow the steps below:

1. Send a request to OSC Help the following information:
   • Name of organization that will be running the endpoint, ie: OSU Arts and Sciences
     • NOTE: if the name already exists, they will have to coordinate with the existing Admin for that project
   • OSU affiliated email address associated with the Globus account, ie: name.#@osu.edu
2. OSC will create a new project at https://developers.globus.org, make the user provided in #1 the administrator, and inform the user to set up the endpoint credentials
3. The user goes to https://developers.globus.org/ and chooses “Register a new Globus Connect Server v5”. Under the project, the user chooses Add dropdown and chooses Add new Globus Connect Server. Provide a display name for the endpoint, ie: datamover02.hpc.osc.edu. Select “Generate New Client Secret” and save that value and Client ID and use those values when configuring the Globus Connect Server install on their local system
4. The user finishes configuring Globus Connect Server and runs the necessary commands to register the new endpoint with Globus. Once the new endpoint is registered, please email OSC Help the endpoint name so we can mark the endpoint as managed under the OSU subscription