Import, Export, and Reload a Study: /home/

Import, Export, and Reload a Study

Studies can be exported, imported, and reloaded to make it easy to transfer them between servers, duplicate them within a server, and synchronize them with a master database. A few common usage scenarios:

Studies can be reloaded to transfer a study from a staging environment to a live platform.
A snapshot of one study can be exported and imported into several new studies so that they share a common baseline.
A brand new study can be generated with the exported structure (with or without the data) of an existing study. This allows very rapid creation of new studies based on templates.
The structure of an existing set of studies can also be standardized by importing selected structural elements from a common template.
A study can be exported masking all identifying information enabling the sharing of results without sharing PHI or any patient or clinic details.
Studies can be set up to reload nightly from a remote master data depot.

The process of importing and exporting a folder (including a study folder), is described in the topic: Export / Import a Folder. Some specifics which only apply to study archives are covered here.

Export

To export a study folder, go to Admin > Manage Study and click the Export Study button at the bottom of the page. This page is also accessible through the Admin > Folder > Management page on the Export tab and is described in Export / Import a Folder.

Details about the study objects can be found here: Export Study Objects.

Import

Importing a study archive is the same as importing any folder archive with a few additional considerations. Like other imports, you first create a folder of type "Study", then navigate to it before importing. You can import from an exported archive, or from another study folder on the same server. Selecting which study objects to import from either source gives you, for example, the ability to import an existing study's configuration and structure without including the actual dataset data from the archive. If you do want to import dataset data or specimen data to a new study, you must use the archive import option as the folder template method does not support data transfer.

Validate All Queries After Import

By default, queries will be validated upon import of a study archive and any failure to validate will cause the import job to raise an error. To suppress this validation step, uncheck the Validate all queries after import option. If you are using the check-for-reload action in the custom API, there is a suppress query validation parameter that can be used to achieve the same effect as unchecking this box in the check for reload action.

Fail Import for Undefined Visits

By default, new visit rows will be created in the study during import for any dataset or specimen rows which reference a new, undefined visit. If you want the import to instead fail if it would create visits that are not already in the destination study or imported visit map, you can check the box to "Fail Import for Undefined Visits".

Overlapping Visit Ranges

If you are importing a new visit map for a visit-based study, the import will fail if the new map causes overlapping visit.

Study Templates

A study minus the actual dataset data can be used as a template for generating new studies of the same configuration, structure and layout. To generate one, you can either:

Create a specific template study with all the required elements but no data.
Export an existing study, but exclude "Dataset Data" on export.

In a large project with many studies, keeping them all of the same format and layout may be a priority. A template can also be useful if an administrator needs to make a change to the web parts or tabs in all child studies at once. Importing only the necessary portion of a template into an existing study can be used to change, for example, webpart layout, without changing anything else about the study.

When you import any study archive, or import from a study template, you can select only the objects of interest.

To import a study template:

Select Admin > Folder > Management and click the Import tab.
Click Choose File or Browse and select the study template archive.
Check the box for "Use advanced import options".
Click Import from local zip archive.
Check the box for "Select specific objects to import".
Select the elements to import; if you are using an archive as a template, check that "Dataset Data" and other data objects are not checked. Objects not available or not eligible for import are grayed out.

Click Start Import.

It is also possible to import a study into multiple folders at once. More information about these options can be found here: Advanced Folder Import Options.

Reload

A study can be configured to reload study data from the pipeline root, either manually or automatically at pre-set intervals, which can be useful for refreshing studies whose data is managed externally. For example, if the database of record is SAS, a SAS script could automatically generate TSVs nightly to be reloaded into LabKey Server. This simplifies the process of using LabKey tools for administration, analysis, reporting, and data integration without forcing migration of existing storage or data collection frameworks.

Caution: Reloading a study will replace existing data with the data contained in the imported archive.

To set up reload of study data:

Open the Manage tab and select Manage Reloading.
Check the Allow Study Reload box.

Manual Reload: Set the "Reload Interval" to <Never>.
Automatic Reload: Set the "Reload Interval" to a time interval. A reload is attempted automatically each time the specified interval elapses.

Click Update.
Once you have enabled reloading, whether manual or automatic, a reload attempt can be initiated at any time by an administrator clicking the Attempt Reload Now button or by an external script invoking that same URL.

When reload is attempted, the server checks the modification time on a file named studyload.txt in the pipeline root folder. If it has changed since the last reload, the server reloads the study archive from this folder. LabKey Server ignores the contents of studyload.txt, looking only at the file's modification timestamp.

Study reload can be used to copy study data from an external database of record into LabKey Server, often on a nightly basis, to enable analysis and integration. The process typically involves an automated script that follows steps similar to these:

Read dataset, specimen, and other important study data from a master database and/or specimen LIMS system.
Write the data to the file system in the LabKey study archive format.
Touch the studyload.txt file to update the timestamp to the current date/time.
Signal to the LabKey Server that the archive is ready to load.

This last step causes the server to initiate the reload immediately, instead of waiting for the next reload timer interval. A script signals the server by issuing an HTTP GET to the URL associated with the Attempt Reload Now button. An example URL:

http://<servername>/labkey/study/MyStudy/checkForReload.view?skipQueryValidation=1

The skipQueryValidation parameter is optional. If provided, it instructs the server to skip the query validation step that normally runs after the study is reloaded. This process flags query errors that might not otherwise be noticed, but can be time consuming.

LabKey Support

LabKey Support