Premium Feature — Available in the Professional, Professional Plus, and Enterprise Editions.
Learn more or
contact LabKey.
ETL modules can access data through a remote connection to an alternate LabKey Server.
To set up a remote connection, see
Manage Remote Connections.
To configure an ETL process to utilize a remote connection, specify the transform
type and the
remoteSource as shown below:
<transform type="RemoteQueryTransformStep" id="step1">
<source remoteSource="EtlTest_RemoteConnection" schemaName="study" queryName="etl source" />
...
</transform>
A sample ETL configuration file is shown below:
<?xml version="1.0" encoding="UTF-8"?>
<etl xmlns="http://labkey.org/etl/xml">
<name>Remote Test</name>
<description>append rows from "remote" etl_source to etl_target</description>
<transforms>
<transform type="RemoteQueryTransformStep" id="step1">
<description>Copy to target</description>
<source remoteSource="EtlTest_RemoteConnection" schemaName="study" queryName="etl source" />
<destination schemaName="study" queryName="etl target" targetOption="truncate"/>
</transform>
</transforms>
<incrementalFilter className="ModifiedSinceFilterStrategy" timestampColumnName="modified" />
</etl>
Note that using <deletedRowSource> in an incremental filter does not support a remote connection.
ModifiedSinceFilterStrategy
The incrementalFilter
ModifiedSinceFilterStrategy with a specified timestampColumnName, as shown in the above sample, will direct this ETL to only update rows that have been modified on the Source. Those that have been modified on the Target but not modified on the Source will be skipped by the ETL and not overwritten.
Related Topics
