Premium Feature — Available with all Premium Editions of LabKey Server. Learn more or contact LabKey.

File Watchers let administrators set up the monitoring of directories on the file system and perform specific actions when desired files appear. This topic covers using file name selection patterns to define which files trigger the watcher. Use file patterns in conjuction with the target container path to take action on specific files.

No File Pattern Provided / Default File Pattern

If no FilePattern is supplied, the default pattern is used:

(^\D*)\.(?:tsv|txt|xls|xlsx)

This pattern matches only file names that contain letters and special characters (for example: Dataset_A.tsv). File names which include digits (for example: Dataset_1.tsv) are not matched, and their data will not be loaded.

Under the default file pattern, the following reloading behavior will occur:

File NameFile Watcher Behavior
DemographicsA.tsvFile matched, data loaded into dataset DemographicsA.
DemographicsB.tsvFile matched, data loaded into dataset DemographicsB.
Demographics1.tsvNo file match, data will not be loaded.
Demographics2.tsvNo file match, data will not be loaded.

User Defined Pattern

You can use any regex pattern to select source files for loading. If you want to target datasets that have digits in their names, use:

(^.*).(?:tsv|txt|xls|xlsx)

You can also use a "name capture group" as the FilePattern. See below for details.

As another example, you can match all or part of the filename string. Suppose you have the following third source files:

FooStudy_Demographics.tsv
FooStudy_LabResults.tsv
BarStudy_Demographics.tsv

The regex file pattern...

FooStudy_(.+).(tsv)

will result in the following behavior...

File NameFile Watcher Behavior
FooStudy_Demographics.tsvFile matched, data loaded into dataset FooStudy_Demographics.
FooStudy_LabResults.tsvFile matched, data loaded into dataset FooStudy_LabResults.
BarStudy_Demographics.tsvNo file match, data will not be loaded.

Name Capture Group Patterns

The file pattern can extract names, IDs, or other token values from the source file name for use in targeting the dataset, container, or other destination for the file watcher.

As one example, you might extract a <name> or <id> from the source file name, then target an existing dataset of the same name or ID. The same type of capture can be used for other scenarios. Additional examples are in this topic:

Suppose you have a source file with the following name:
dataset_Demographics_.xls

Use the following file pattern to extracts the value <name> from the file name that occurs between the underscore characters, in this case the string "Demographics", and loads data into an existing dataset with the same name: "Demographics".

dataset_(?<name>.+)_.(xlsx|tsv|xls)

Using the pattern above, the following behavior will result:

File NameFile Watcher Behavior
dataset_Demographics_.tsvFile matched, data loaded into dataset Demographics.
datasetDemographics.tsvNo file match, data will not be loaded.
dataset_LabResults1_.tsvFile matched, data loaded into dataset LabResults1.
dataset_LabResults2_.tsvFile matched, data loaded into dataset LabResults2.

To target a dataset by its dataset id, rather than its name, then use the following regex, where <id> refers to the dataset id. Note that you can determine a dataset's id by navigating to your study's Manage tab, and clicking Manage Datasets. The table of existing datasets shows the id for each dataset in the first column.

dataset_(?<id>.+)_.(xlsx|tsv|xls)

If you want to capture a name from a file with any arbitrary text (both letters and numbers) before the underscore and desired name, use:

.*_(?<name>.+).(xlsx|tsv|xls)

This would match as follows:

File NameFile Watcher Behavior
20220102_Demographics.tsvFile matched, data loaded into dataset Demographics.
LatestUpdate_LabResults.xlsxFile matched, data loaded into dataset LabResults.
2022Update_Demographics.xlsFile matched, data loaded into dataset Demographics.

Find more examples, including capturing and using multiple elements from a filename, in this topic:

Related Topics

Was this content helpful?

Log in or register an account to provide feedback


previousnext
 
expand allcollapse all