HOWTO: Use AWS S3 in Globus

Globus Connect Server v5 allows OSC clients to connect to Amazon S3. Please follow the steps below:

Create a New Bucket

  • Log into Amazon AWS
  • Navigate to Services -> S3
  • Click 'Create Bucket'
  • Give the bucket a name then click 'Next'
  • Click 'Next' and then 'Next' again then 'Create bucket' to create a new bucket

Create a New IAM

  • Log into Amazon AWS
  • Navigate to Services -> IAM

Add Policy

  • Select Policies and then click 'Create Policy'
  • Choose the JSON tab and paste the JSON policy as below (It will be necessary to replace the value for 'Resource' to match bucket name from previous steps):
You may also refer to this page provided by Globus for the JSON policy.
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllBuckets",
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets",
                "s3:GetBucketLocation"
            ],
            "Resource": "*"
        },
        {
            "Sid": "Bucket",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": "arn:aws:s3:::osc-globus-test"
        },
        {
            "Sid": "Objects",
            "Effect": "Allow",
            "Action": [
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::osc-globus-test/*"
       }
    ]
}
  • Choose 'Review Policy'
  • Fill in the name such as 'AllowGlobusS3_osc-globus-test', click 'Create Policy' to create the new policy

Add Group

  • Choose 'Groups' in left menu bar then click 'Create New Group'. Provide the group name such as “s3_osc-globus-test” then click 'Next Step' button
  • In search bar for Filter type 'Globus' and check box next to 'AllowGlobusS3_osc-globus-test' then click 'Next Step'
  • Click 'Create Group' after reviewing new group information. 

Add User

  • Choose 'Users' in left menu bar and then click 'Add user'. Provide the username, check 'Programmatic access' and then click 'Next: Permissions'
  • Choose 'Add user to group', filter for the group created previous and check box next to 's3_osc-globus-test', click 'Next: Tags' and then 'Next: Review'
  • Click 'Create user'
  • Make a copy of 'Access key ID' and click 'Show' for 'Secret access key' and save both for future reference
  • Click 'Close'

Create a New Collection

  • Login to Globus. If your institution does not have an organizational login, you may choose to either Sign in with Google or Sign in with ORCiD iD
  • Navigate to the 'ENDPOINTS' on the sidebar and search 'OSC GCSv5'. Click 'OSC GCSv5' to go to this endpoint
  • Click the 'Collections' menu. You will see all of the collections added by you before. To add a new collection, click 'Add a Collection'. Enter your OSC username and password for the authentication
  • Choose 'S3 Storage Gateway (S3)' from the list of options
  • Provide 'Access key' and "Access secret' from previous steps and associate Globus identity with your OSC identity
  • Provide bucket name for 'Base Directory'
  • Provide the name of the collection in 'Collection Display Name' field
  • Click 'Create Collection' to finish the creation
  • Click the logo of 'Globus' and navigate back to the 'ENDPOINTS' on the sidebar. Click the 'Administered by You' and then you can locate the new collection you just created. 

Further Reading