×
Maintenance Notice: labkey.org will be offline for maintenance tonight @ 7pm Pacific Time. The estimated downtime will be approximately one hour. Sorry for any inconvenience this may cause.

Error when trying to use FileSet

Installation Forum (Inactive)
Error when trying to use FileSet Andy Straw  2013-04-09 14:41
Status: Closed
 
I'm trying to use a FileSet - that is, I want to use a part of my server's file system for storing/accessing files, but that part of the file system is not underneath my default file root. (We are looking into using different storage for large, read-only files like NGS FASTQ and BAM files. But for this experiment, I was simply using a different area of my C: drive.) I followed the instructions here:
https://www.labkey.org/wiki/home/Documentation/page.view?name=setupFileContent
I successfully registered the file set - got a success message: "Directory successfully registered."

I then customized a Files web part to use that file set as its File Root. That seemed to go okay as well.

I then used that Files web part to try to create a new folder, but saw a popup with error message:
"An error occurred trying to load: Conflict"

I also tried using WebDrive to create a folder, but got this error message: "You need permission to perform this action". I noticed that LabKey had created a "@filesets" folder next to my project's "@files", and the "@filesets" folder had a subfolder with name matching the name of file set. It was inside that folder that I tried to create a subfolder using WebDrive.

The folder I'm using for the file set has the same permissions as the default file root (which works fine). In fact, my tomcat instance is running as me, and I created the folder I'm trying to use as a file set.

I'm running a pretty up-to-date version of trunk on Windows 7.

What am I doing wrong? Please help. Thanks.

Andy Straw
University of Rochester.
 
 
Andy Straw responded:  2013-04-10 06:34
Some additional info from doing some further testing.

- First of all, before registering the file set, I made sure there were some files and sub-folders in the folder I used as a File Set. After registering the file set, and setting it as the File Root of a Files web part, I could see the files in the folder in the Files web part, but could NOT see the sub-folders in the Files web part.
- I tried to upload some files using the "Single file" upload feature of the Files web part. Some files would not upload - I tried several times. There was no error reported to the user, but the log file reported the attached stack trace.
- I tried uploading some other files, and they seemed to upload fine, but each time I uploaded one, I got a stack trace in the log file again.
- The "Created By" for all files displays as "Guest" - both those that were already in the folder before I registered it as a file set, and those I uploaded through the Files web part - even though I was logged in when I did the uploads. Note that when I upload the same files via a Files web part that's using the default File Root, they upload with no problems, and their "Created By" is properly displaying my user's display name.

File sets seem to be pretty unusable - or am I doing something wrong?

Andy Straw
University of Rochester
 
Brian Connolly responded:  2013-04-10 09:03
Andy,

File sets are a legacy feature. They should not be used for any new applications like yours. I recommend either

- setting a non-default file root for the folder or
- configuring a pipeline override for the folder.

See https://www.labkey.org/wiki/home/Documentation/page.view?name=fileTerminology for a background on the terminology LabKey uses. And https://www.labkey.org/wiki/home/Documentation/page.view?name=adminFiles for documentation on working with files.

In this situation, I recommend configuring a pipeline override for your project or folder. When using the pipeline override instead of a fileset, the Files webpart will be able to see all files and directories in the pipeline override directory and all sub-directories and their files. In addition, the "Created By" and other file properties will be supported (if the files are uploaded via the File webpart or Data Pipeline webparts).

Instructions for setting up a pipeline override are at https://www.labkey.org/wiki/home/Documentation/page.view?name=pipelineSetup Once you have configured your pipeline, you can access the pipeline override directory from a Files webpart. You will need to customize the Files webpart:

1. click on the down arrow next to the word "Files" in the webpart header
2. in the file root dropdown select "Pipeline Files"


With respect to the behavior you experienced when using the FileSet. This was not a bug, this behavior is by design. Filesets can only see the files at the root of the Fileset directory and cannot see any sub-directories, etc. As I said above, FileSets are a legacy feature and have been replaced by File Roots and Pipeline Overrides (https://www.labkey.org/wiki/home/Documentation/page.view?name=fileTerminology). Filesets are still documented and supported as some of our oldest customers are still using them.


- Brian
 
jeckels responded:  2013-04-10 09:24
Hi Andy,

File Sets are intended to show a single directory, without exposing subdirectories. This is why you're unable to create or view subdirectories. As Brian says, they're essentially a legacy feature that some installations use to expose files, but aren't in active use for accepting uploads.

I'll open a bug on the NullPointerException you're getting when uploading new files.

Thanks,
Josh