This topic outlines how to create and configure S3 cloud storage on LabKey Server. Each bucket on S3 that you plan to access will be defined as a Storage Config on LabKey Server, accessed through a Cloud Account. You will then be able to select which storage config to use on a per-folder basis.

Configure LabKey Server to use Cloud Storage

• Step 1: Create Bucket (on AWS)
• Step 2: Create Cloud Account (on LabKey Server)
• Step 3: Create Storage Config (on LabKey Server)
• Optional: Configure Queue Notifications for File Watchers

Create Bucket (on AWS)

Before you can use your Cloud Storage account from within LabKey Server, you must first create the bucket you intend to use and the user account must have "list" as well as "upload/delete" permissions on the bucket.

• It is possible to have multiple cloud store services per account.
• AWS S3 "default" AES encryption is supported and can be configured on the S3 bucket when the bucket is provisioned.
• With "default" encryption S3 transparently encrypts/decrypts the files/objects when they are written to or read from the S3 bucket.
• AWS S3 also supports unique KMS (Key Management System) encryption keys that are managed by the customer within AWS.

Create Cloud Account On LabKey Server

To access the bucket, you create a cloud account on your server, providing a named way to indicate the cloud credentials to use.

• Select (Admin) > Site > Admin Console.
• Under Premium Features, click Cloud Settings.
• If you do not see this option, you do not have the cloud module installed.
• Under Cloud Accounts, click Create Account.

• Enter an Account Name. It must be unique and will represent the login information entered here.
• Select a Provider.
• Enter your Identity and Credential. See AWS Identity for details.
• Click Create.

This feature uses the encrypted property store for credentials and requires an administrator to provide an encryption key in the application.properties file. LabKey will refuse to store credentials if a key is not provided.

Create Storage Config (on LabKey Server)

Next define a Storage Config, effectively a file alias pointing to a bucket available to your account. LabKey can create new subfolders in that location, or if you want to use a pre-existing S3 subdirectory within your bucket, you can specify it using the S3 Path option.

• Click Create Storage Config on the cloud account settings page under Cloud Store Service.
• If you navigated away, select > Site > Admin Console. Under Premium Features, click Cloud Settings.

• Provide a Config Name. This name must be unique and it is good practice to base it on the S3 bucket that it will access.
• Select the Account you just created from the pulldown.
• Provide the S3 Bucket name itself. Do not include "S3://" or other elements of the full URL with the bucket name in this field. Learn more about bucket naming rules here
• Select Enabled.
• If you disable a storage config by unchecking this box, it will not be deleted, but you will be unable to use it from any container until enabling it again.
• S3 Path: (Optional) You can specify a path within the S3 bucket that will be the configuration root of any LabKey folder using this configuration. This enables use of an existing folder within the S3 bucket. If no path is specified, the root is the bucket itself.
• Directory Prefix: (Optional) Select whether to create a directory named <prefix><id> in the bucket or S3 path provided for this folder. The default prefix is "container".
• If you check the Directory Prefix box (default), LabKey will automatically create a subdirectory in the configuration root (the bucket itself or the S3 path provided above) for each LabKey folder using this configuration. For example, a generated directory name would be "container16", where 16 is the id number of the LabKey folder. You can see the id number for a given folder/container by going to Folder > Management > Information, or by querying the core.Containers table through the UI or an API. You may also find the reporting in Admin Console > Files helpful, as it will let you navigate the container tree and see the S3 URLs including the containerX values. Note that using this option means that the subdirectory and its contents will be deleted if the LabKey folder is deleted.
• If you do not check the box, all LabKey folders using this configuration will share the root location and LabKey will not delete the root contents when any folder is deleted.
• SQS Queue URL: If your bucket is configured to queue notifications, provide the URL here. Note that the region (like "us-west-1" in this URL) must match the region for the S3 Bucket specified for this storage config.
• Click Create.

Authorized administrators will be able to use the Edit link for defined storage configs for updating them.

Configure Queue Notifications for File Watchers

If you plan to use file watchers for files uploaded to your bucket, you must first configure the Simple Queue Service within AWS. Then supply the SQS Queue URL in your Storage Config on LabKey Server.

Learn more in this topic:

• Cloud Storage for File Watchers

Related Topics

• S3 Cloud Data Storage
• AWS Identity Credentials
• Use Files from Cloud Storage