This topic will cover the main 'attributes' applied at the root level of ETL syntax. Some reference other topics, some described here directly.
Standalone (vs. Component)
ETL processes can be set as either "standalone" or "sub-component":
- Standalone ETL processes:
- Appear in the Data Transforms web part
- Can be run directly via the user or via another ETL
- Sub-Component ETL processes or tasks:
- Not shown in the Data Transforms web part
- Cannot be run directly by the user, but can be run only by another ETL process, as a sub-component of a wider job.
- Cannot be enabled or run directly via an API call.
To configure as a sub-component, set the ETL "standalone" attribute to false. By default the standalone attribute is true.
<etl xmlns="http://labkey.org/etl/xml" standalone="false">
...
</etl>
Learn more here:
ETL: Queuing ETL Processes
Load Referenced Files
To handle files generated during or otherwise involved in ETL processing (for example, if one ETL process outputs a file, and then queues another process that expects to use the file in a pipeline task), all ETL configurations in the chain must have the attribute loadReferencedFiles="true” in order for the runs to link up properly.
<etl xmlns="http://labkey.org/etl/xml" loadReferencedFiles="true">
...
</etl>
Site Scope
By default an ETL is container scoped. To let an ETL run at a site scope level, set the attribute
siteScope to true:
<etl xmlns="http://labkey.org/etl/xml" siteScope="true">
Allow Multiple Queuing
By default, if a job for the ETL is already pending (waiting), we block adding another instance of the same ETL to the job queue. Set the
allowMultipleQueuing flag to "true" override and allow multiple instances in the queue.
Transaction Wrapped ETLs
The attributes
transactSourceSchema and
transactDestinationSchema are used for wrapping a multi-step ETL into a single transaction.
Learn more in this topic:
ETL: Transactions
Related Topics