File Watchers for Script Pipelines

 2024-04-20
Premium Feature — Available with all Premium Editions of LabKey Server. Learn more or contact LabKey.

A script pipeline lets you run scripts and commands in a sequence, where the output of one script becomes the input for the next in the series. The pipeline supports any of the scripting languages that can be configured for the server, including R, JavaScript, Perl, Python, SAS, and others.

With Premium Editions of LabKey Server, script pipelines are available as File Watchers, so that they may be run automatically when desired files appear in a watched location.

Define Script Pipelines

Script pipelines are defined in modules, and will be available as File Watchers in any folder where that module is enabled.

Using the instructions in this topic, define the tasks and pipelines in the module of your choice:

For example, you might use a "helloworld" module and define two pipelines, named "Generate Matrix and Import into 'myassay'" and "Use R to create TSV file during import".

Next, enable the module containing your pipeline in any folder where you want to be able to use the script pipelines as File Watchers.

  • Select (Admin) > Folder > Management > Folder Type.
  • Check the module and click Update Folder.

Once the module is enabled, you will see your script pipelines on the folder management Import tab alongside the predefined tasks for your folder type:

Customize Help Text

The user interface for defining File Watchers includes an information box that begins "Fields marked with an asterisk * are required." You can place your own text in that same box after that opening phrase by including a <help> element in your script pipeline definition (*.pipeline.xml file). 

<pipeline xmlns="http://labkey.org/pipeline/xml"
          name="hello" version="0.0">
    <description>Generate Matrix and Import into 'myassay'</description>
    <help>Custom help text to explain this script pipeline.</help>
    <tasks>
        <taskref ref="HelloWorld:task:hello"/>
    </tasks>
</pipeline>

Configure File Watcher

As with other File Watcher, give it a name and provide the Details and Configuration information using the Create Pipeline Trigger wizard. See Create a File Watcher for details.

Prevent Usage of Script Pipeline with File Watcher

If you want to disable usage of a specific script pipeline as a File Watcher entirely, include in your *.pipeline.xml file setting the triggerConfiguration:allow parameter to false. Syntax would look like: 

<pipeline xmlns="http://labkey.org/pipeline/xml"
          name="hello2" version="0.0">
    <description>Use R to create TSV file during import</description>
    <tasks>
        <taskref ref="HelloWorld:task:hello"/>
    </tasks>
    <triggerConfiguration allow="false"/>
</pipeline>

Related Topics