• Sign In
  1. Messages

File Repository Module write location, etc.

LabKey Support Forum

View Message

view list print
File Repository Module write location, etc. Georgia Mayfield  2018-06-20 12:05
Status: Closed
 

Hi all,

My team is considering using the File Repository module for file upload/storage and has some questions on its functionality/implementation.

First off, where does File Repository actually write the uploaded files (by default)? I'm unable to track uploaded test files on either the machine or in the database. Is there an API into the Webdav repository? Also, is there any sort of built-in backup for File Repository?

Is there's no backup in File Repository itself, then we would need to map the write space to a network drive, such as an RDS repository. What would you estimate to be the complexity/requirements of that?

Additionally, we would be interested in keeping metadata about the uploaded files and clickable links to the files in a database table to retain out current functionality. What sort of metadata capture features like this exist for File Repository, if any? Would it be possible to cap file size, if needed?

System Info

  • OS: rhel 6
  • Version: 17.3

Thanks,
Georgia Mayfield
OHSU Computational Biology
 
 
adam responded:  2018-06-21 11:51

Hi Georgia,

LabKey provides many options for file storage, including local file system, network drives, and cloud storage (Amazon S3). All three integrate with the file browser UI, WebDAV clients and APIs, and the data processing pipeline for analysis and import of data. You can configure any number of custom properties to associate meta data with files; values can be provided (and retrieved) via the UI and the LabKey query API. LabKey automatically creates an on-disk folder hierarchy that matches the LabKey folder structure, but you can override this to provide a custom layout and split the repository between different file systems, if desired. LabKey provides no built-in backup facilities, but standard file system backup approaches should work fine. Other capabilities worth mentioning include full-text search over document repositories and the ability to configure file watchers to automatically load assay, study, and other data.

This probably raises even more questions, but a discussion would be the best way to review your specific requirements and use cases. I've been in communication with some of your colleagues recently, so I'll add you to an email thread and provide some suggestions for next steps.

Thanks,
Adam
 
Georgia Mayfield responded:  2018-07-23 15:16

Hi Adam,

If both the local file system and a network drive can be a mappable location for use with File Repository, where and how specifically is that set/managed?

When I say I need to keep metadata about the files uploaded to File Repository in the database, I mean I need to keep file metadata in a table in the underlying relational database. You're suggesting the LabKey query API would be capable of integrating with File Repository like this?

Best, G
 
Jon (LabKey DevOps) responded:  2018-07-27 11:12
Hi Georgia,

The File Repository have have its File Root set on the Site-level, the Project-level, the Folder-level, and many other places for flexibility. The drive would just need to be made accessible to the user that the Tomcat process runs as.

For more details on this, please see the following documentation for more information:

https://www.labkey.org/Documentation/wiki-page.view?name=setRoots

Additionally, file metadata is something that is recorded as well and can be accessed via a Query API. (https://www.labkey.org/Documentation/wiki-page.view?name=fileMetadata)

Regards,

Jon