You can call an ETL task from within another ETL process by using a <taskref> that refers to org.labkey.di.steps.QueueJobTask.

Reference the ETL process you wish to queue up by module name and file name, using the pattern "{MODULE_NAME}/FILE_NAME". For example, to queue up the process MaleNC.xml in the module etlmodule, use the following:

<transform id="QueueTail" type="TaskrefTransformStep">
<taskref ref="org.labkey.di.steps.QueueJobTask">
<setting name="transformId" value="{MODULE-NAME}/MaleNC"/>

An ETL process can also queue itself by omitting the <setting> element:

<transform id="requeueNlpTransfer" type="TaskrefTransformStep">
<taskref ref="org.labkey.di.steps.QueueJobTask"/>

Handling Generated Files

If file outputs are involved (for example, if one ETL process outputs a file, and then queues another process that expects to use the file in a pipeline task), all ETL configurations in the chain must have the attribute loadReferencedFile="true” in order for the runs to link up properly.

<etl xmlns="" loadReferencedFiles="true">

Standalone vs. Component ETL Processes

ETL processes can be set as either "standalone" or "sub-component":

  • Standalone ETL processes:
    • Appear in the Data Transforms web part
    • Can be run directly via the user or via another ETL
  • Sub-Component ETL processes or tasks:
    • Not shown in the Data Transforms web part
    • Cannot be run directly by the user, but can be run only by another ETL process, as a sub-component of a wider job.
    • Cannot be enabled or run directly via an API call.
To configure as a sub-component, set the "standalone" attribute to false. By default the standalone attribute is true.

<transform id="MySubComponent" standalone="false">





expand all collapse all