You can export and import the contents of a folder, both data and configuration elements, in a folder archive format. A folder archive is a .folder.zip file or a collection of individual files that conform to LabKey folder description conventions and formats. In most cases, a folder archive is created via export from the UI. You can also populate a new folder from a template folder on the current server using the "
Create Folder From Template" option on the folder creation page.
Overview
Folder archives are useful for migrating work from one server to another, or for populating one or more new folders with all or part of the contents of the archive. It is important to note that you can only go "forward" to new versions. You cannot import an archive from a newer (higher) version of LabKey Server into an older (lower) version.
This topic helps you understand the options available for folder archive export/import. A few common usage scenarios:
- Create a folder template for standardizing structure.
- Transfer a folder from a staging / testing environment to a production platform.
- Export a selected subset of a folder, such as excluding columns tagged as containing protected health information to enable sharing of results without compromising PHI.
You can choose to include the datasets, views, and reports, as well as much of the original folder's configuration. See the screen shot below for items that can be included in the folder archive.
Not all folder types are supported for export and re-importation into another folder or server. For example, Biologics and Sample Manager folder types are not supported for migration in this manner. Make sure to test re-importation into a safe folder as a precautionary step.
Export
- To export a folder, go to (Admin) > Folder > Management and click the Export tab.
- Select the objects to export.
- You can use the Clear All Objects button to clear all the selections (making it easier to select a minimal set manually).
- You can use Reset to restore the default set of selections, i.e. most typically exported objects (making it easier to select a larger set).
- Choose the options required.
- Select how/where to export the archive file(s).
- Click Export.
There are additional options available when the folder is a study. For more about exporting and importing a
study, see
Export/Import/Reload a Study.
Folder objects to export
This is the complete list of objects that can be exported to a folder archive. When each of the following options is checked, the specified content will be included in the archive. Some options will only be shown when relevant content is present.
- Folder type and active modules: The options set on the "Folder Type" tab.
- Missing value indicators: The settings on the "Missing Values" tab.
- Full-text search settings: The settings on the "Search" tab.
- Webpart properties and layout: Page layouts and settings.
- Container specific module properties: Settings on the "Module Properties" tab.
- Project-level groups and members (Only available for project exports)
- Role assignments for users and groups: See Export and Import Permission Settings.
- Sample Status and QC State Settings: Exports QC state definitions including custom states, descriptions, default states for the different import pathways and the default blank QC state.
- Learn more about sample status export/import in this topic.
- Notification settings: The settings on the "Notifications" tab.
- Queries: Shared queries. Note that private queries are not included in the archive.
- Grid Views: Shared grid views. Note that private grid views are not included in the archive.
- Reports and Charts: Shared reports and charts. Private ones are not included in the archive.
- Categories: This option exports the Categories for report and dataset grouping.
- External schema definitions: External schemas defined via the Schema Browser.
- Experiments, Protocols, and Runs: Check to include experiment data including samples, assay designs and data, and job templates, if any are defined. Configured assay QC states are exported, but assignments of QC states to assay runs are not included.
- Sample Types and Data Classes: Include definitions for these entities. If a Sample Type has a setting for "Auto-Link Data to Study", this setting will be included in the export.
- ETL Definitions (Premium feature): Extract-transform-load definitions.
- Files: The contents of the file repository.
- Inventory locations and items: (Premium feature) If any freezers have been defined in Sample Manager in this container, you will see this checkbox to include freezer definitions and storage information in the export.
- Lists: Both definition and content of lists.
- Wikis and their attachments
- Study (Only present in study folders): Learn more in this topic: Export a Study.
- Panorama QC Folder Settings (Only present in Panorama QC folders): Learn more in this topic: Panorama QC Folders.
Select Export Options
Whether to
Include Subfolders in your archive is optional; this option is only presented when subfolders exist.
You can also choose to exclude protected health information (PHI) at different levels. This exclusion applies to all dataset and list columns, study properties, and specimen data columns that have been
tagged at a specific PHI level. By default, all data is included in the exported folder.
- Include PHI Columns:
- Uncheck to exclude all columns marked as containing any level of PHI.
- Check to include some or all PHI, then select the level(s) of PHI to include.
Similar PHI functionality is available when publishing a study. For more details see
Publish a Study: Protected Health Information / PHI.
To tag a field at a specific PHI level, see
Protecting PHI Data.
Select an Export Destination
Under
Export to:, select one the export destination:
- Pipeline root export directory, as individual files.
- Pipeline root export directory, as a zip file.
- Browser as a zip file.
You can place more than one folder archive in a directory if you give them different names.
Character Encoding
Study, list, and folder archives are all written using UTF-8 character encoding for text files. Imported archives are parsed as UTF-8.
Items Not Included in Archives
The objects listed above in the "Folder objects to export" section are the only items that are exported to a folder archive. Here are some examples of LabKey objects are not included when exporting a folder archive:
- Assay Definitions without Runs: An assay definition will only be included in a folder export when at least one run has been imported to the folder using it.
- File Properties: The definitions and property values will not be included in the folder archive.
- Issue Trackers: Issue tracker definitions and any issue data will not be included in the folder archive.
- Messages: Message settings and message data will not be included in the folder archive.
- Project Settings and Resources: Custom CSS file, logo images, and other project-level settings are not included in the folder archive.
- Query Metadata XML: Metadata XML applied to queries in the Schema Browser is not exported to the folder archive. Metadata applied to fields is exported to the folder archive.
When migrating a folder into another container using an archive, the items above must be migrated manually.
Import
When you import a folder archive, a new subfolder is not created. Instead the configuration and contents are imported into the current folder, so be sure not to import into the parent folder of your intended location.
To create the imported folder as a subfolder, first create a new empty folder, navigate to it, then import the archive there.
- To import a folder archive, go to (Admin) > Folder > Management and click the Import tab.
- You can import from your local machine or from a server accessible location.
Import Folder From Local Source
- Local zip archive: check this option, then Browse or Choose an exported folder archive to import.
- Existing folder: select this option to bypass the step of exporting to an archive and directly import selected objects from an existing folder on the server. Note that this option does not support the import of specimen or dataset data from a study folder.
Both import options offer two further selections:
- Validate All Queries After Import: When selected, and by default, queries will be validated upon import and any failure to validate will cause the import job to raise an error. If you are using the check-for-reload action in the custom API, there is a suppress query validation parameter that can be used to achieve the same effect as unchecking this box in the check for reload action. During import, any error messages generated are noted in the import log file for easy analysis of potential issues.
- Show Advanced Import Options: When this option is checked, after clicking Import Folder, you will have the further opportunity to:
- Select specific objects to import
- Apply the import to multiple folders
If the folder contains a study, you will have an additional option:
- Fail import for undefined visits: when you import a study archive, you can elect to cancel the import if any imported dataset or specimen data belongs to a visit not already defined in the destination study or the visit map included in the imported archive. Otherwise, new visits would be automatically created.
Select Specific Objects to Import
By default, all objects and settings from an import archive will be included. For import from a template folder, all except dataset data and specimen data will be included. If you would like to import a subset instead, check the box to
Select specific objects to import. You will see the full list of folder archive objects (similar to those you saw in the export options above) and use checkboxes to elect which objects to import. Objects not available in the archive or template folder will be disabled and shown in gray for clarity.
This option is particularly helpful if you want to use an existing archive or folder as a structural or procedural template when you create a new empty container for new research.
Apply to Multiple Folders
[
Video Overview: Applying Study Templates Across Multiple Folders - Version 16.3 ]
By default, the imported archive is applied only to the current folder. If you would like to apply this imported archive to multiple folders, check
Apply to multiple folders and you will see the list of all folders in the project. Use checkboxes to select all the folders to which you want the imported archive applied.
Note that if your archive includes subfolders, they will not be applied when multiple folders are selected for the import.
This option is useful when you want to generate a large number of folders with the same objects, and in conjunction with the selection of a subset of folder options above, you can control which objects are applied. For instance, if a change in one study needs to be propagated to a large number of other active studies, this mechanism can allow you to propagate that change. The option "Selecting parent folders selects all children" can make it easier to use a template archive for a large number of child folders.
When you import into multiple folders, a separate pipeline job is started for each selected container.
Import Folder from Server-Accessible Archive
Click
Use Pipeline to select the server-accessible archive to import.
Related Topics
