Premium Feature — Available in the Professional and Enterprise Editions of LabKey Server. Also available as an Add-on to the Starter Edition.
Learn more or
contact LabKey.
Once you have
configured S3 cloud storage on your LabKey Server, you can enable and use it as described in this topic.
Enable Cloud Storage in Folders and Projects
In each folder or project where you want to access cloud data, configure the filesystem to use the appropriate cloud storage config(s) you defined. Cloud storage at the project level can be inherited by folders within that project, or folders can override a project setting as needed.
Note that if a cloud storage config is disabled at the site-level it will not be possible to enable it within a folder or project.
- Navigate to the folder where you want to enable cloud storage and open (Admin) > Folder > Management.
- If you want to enable cloud storage at the project level, open (Admin) > Folder > Project Settings instead.
- Select the Files tab.
- Under Cloud Stores, the lower section of the page, first enable the desired cloud stores using the checkboxes.
- Note that it's possible to disable a cloud storage config at the site level. If a config is not enabled at the site level, enabling it in a folder or project will have no effect.
- Click Save.
- After saving the selection of cloud stores for this container, you will be able to select one in the file root section higher on this page.
- Under File Root select Use cloud-based file storage and use the dropdown to select the desired cloud store.
- If you select this option before enabling the cloud store, you will see an empty dropdown.
- Existing Files: When you select a new file root for a folder, you will see the option Proposed File Root change from '<prior option>'. Select what you want to happen to any existing files in the root. Note that if you are not using directory containers, you will not be able to move files as they will not be deleted from the shared root. See Migrate Existing Files for details about file migration options.
- Click the Save button a second time.
Configure a Files Web Part
Once you have configured the file root for your folder to point to the cloud storage, you will access it using the usual
Files web part.
- Go to the Files web part in your folder.
- If you don't have one, select (Admin) > Page Admin Mode.
- From the dropdown in the lower left, click <Select Web Part> and choose Files.
- Click Add.
- If desired, you can give the web part a name that signals it will access cloud storage:
- To do so, choose Customize from the (triangle) menu in the Files web part header.
- Enter the Title you would like, such as "Cloud Files".
- Checking the box to make the Folder Tree visible may also be helpful.
- You can also give the webpart a descriptive title while you are in the customize interface.
- Click Submit.
The Files web part will now display the cloud storage files as if they are part of your local filesystem, as in the case of the .fcs file shown here:
The file is actually located in cloud storage as shown here:
- When a download request for a cloud storage file comes through LabKey Server, the handle is passed to the client so the client can download the file directly.
- When a file is dropped into the Files web part, it will be uploaded to the cloud bucket.
- Files uploaded to the S3 bucket independently of LabKey will also appear in the LabKey Files web part.
Run Pipeline Tasks on S3 Files
Once you have configured an S3 pipeline root, folders can be imported or reloaded from folder archives that reside in cloud storage.
- Import folder (from either .folder.zip archive, or folder.xml file)
Use File Watchers with Cloud Files
To use File Watchers, you will need to enable SQS Queue Notifications on your AWS bucket, then include that queue in the storage config. Learn more in this topic:
Once configured, you can
Reload Lists Using Data File from the cloud. Other file watcher types will be added in future releases.
Delete Files from Cloud Storage
If you have
configured cloud storage in LabKey to create a new subdirectory (using
Directory Prefix) for each new folder created on LabKey, the files placed within it will be associated with the LabKey folder. If you delete the LabKey folder, the associated subfolder of your S3 bucket (and all files within it) will also be deleted.
If you instead configured cloud storage to use an existing S3 folder, any files placed there will be visible from within the LabKey folder, but will
NOT be deleted if the LabKey folder is deleted.
Related Topics
