This topic describes how to use existing ETL (Extract Transform Load) processes from within the user interface.
ETL User Interface
The web part
Data Transforms lists all of the ETL processes that are available in the current folder.
- You can also access this interface via (Admin) > Go To Module > Data Integration.
- If you don't know how to add a web part, learn how in this topic: Add Web Parts.
Columns:
- Name - This column displays the name of the process.
- Source Module - This column tells you module where the configuration file resides.
- Schedule - This column shows you the reload schedule.
- Enabled - This checkbox controls whether the automated schedule is enabled: when unchecked, the ETL process must be run manually.
- Last Status, Successful Run, Checked - These columns record the latest run of the ETL process.
- Run: Click Run Now to run this ETL.
- Set Range (Available only in devMode): The Set Range column is displayed only in dev mode and is intended for testing purposes during ETL module development.
- The Run button is only displayed for ETL processes with a filter strategy of RunFilterStrategy or ModifiedSinceFilterStrategy; the button is not displayed for the filter strategy SelectAllFilterStrategy.
- Click Run to set a date or row version window range to use for incremental ETL filters, overriding any persisted or initial values.
- Reset: Use the options on the Reset State button to return the ETL process to its original state, deleting its internal history of which records are, and are not, up to date. There are two options:
- Last Transform Run Log Error - Shows the latest error logged, if any exists.
At the bottom of the ETL listing, the
View Processed Jobs button shows you a log of all previously run ETL jobs, and their status.
Run an ETL Process Manually
From the
Data Transforms web part, you can:
- Run jobs manually, even if a schedule is also configured: Click Run Now.
- Reset state: Select Reset State > Reset resets an ETL transform to its initial state, as if it has never been run.
Enable/Disable a Scheduled ETL
In the
Data Transforms web part, you can see in the
Schedule column whether an ETL is configured to run on a schedule, and if so how frequently.
Use the checkbox in the
Enabled column to control whether that ETL will be run on that schedule or not. When the box is unchecked, the ETL will not be run on a schedule, but may still be
run manually if needed.
Learn more about scheduling ETLs in this topic:
ETL: Schedules.
Cancel and Roll Back Jobs
While a job is running you can cancel and roll back the changes made by the current step by pressing the
Cancel button.
The Cancel button is available on the
Job Status panel for a particular job, as shown below:
To roll back a run and delete the rows added to the target by the previous run, view the
Data Transforms web part, then select
Reset State > Truncate and Reset. Note that rolling back an ETL which outputs to a file will have no effect, that is, the file will not be deleted or changed.
See Run History
The
Data Transform Jobs web part provides a detailed history of all executed ETL runs, including the job name, the date and time when it was executed, the number of records processed, the amount of time spent to execute, and links to the log files.
To add this web part to your page, enter
> Page Admin Mode, then scroll down to the bottom of the page and click the dropdown
<Select Web Part>, select
Data Transform Jobs, and click
Add. When added to the page, the web part appears with a different title: "Processed Data Transforms". Click
Exit Admin Mode.
Click
Run Details for fine-grained details about each run, including a graphical representation of the run.
History of ETL Jobs
To view a history of all ETL jobs ever run across the whole site:
- Go to Admin > Site > Admin Console.
- Under Management, click ETL- All Job Histories.
The history includes the name of the job, the folder it was run in, the date and time it was run, and other information. Links to detailed views of each job and run are provided.
Related Topics
