You can export and import the contents of a folder in an archive format containing a desired set or subset of data and configuration elements. A folder archive is a .folder.zip file or a collection of individual files that conform to LabKey folder description conventions and formats. In most cases, a folder archive is created via export from the UI.
You can also populate a new folder from a template folder on the current server using the "
Create Folder From Template" option on the folder creation page, which offers similar options but does not export an archive.
Overview
Folder archives are useful for migrating work from one server to another, or for populating one or more new folders with all or part of the contents of the archive. It is important to note that you can only go "forward" to new versions. You cannot import an archive from a newer (higher) version of LabKey Server into an older (lower) version.
This topic helps you understand the options available for folder archive export/import. A few common usage scenarios:
- Create a folder template for standardizing structure.
- Transfer a folder from a staging / testing environment to a production platform or vice versa.
- Export a selected subset of a folder, such as excluding columns tagged as containing protected health information to enable sharing of results without compromising PHI.
You can choose to include the data structure definitions, actual data, views, and more, as well as much of the original folder's configuration.
Not all folder types are supported for seamless export and re-import into another folder or server. For example, Biologics and Sample Manager folder types are not supported for migration in this manner.
Be careful to test re-import into a safe folder as a precautionary step before trying this operation with any production system.
Export
- To export a folder, go to (Admin) > Folder > Management and click the Export tab.
- Select the objects to export.
- You can use the Clear All Objects button to clear all the selections (making it easier to select a minimal set manually).
- You can use Reset to restore the default set of selections, i.e. most typically exported objects (making it easier to select a larger set).
- Choose the options required.
- Select how/where to export the archive file(s).
- Click Export.
There are additional options available when the folder is a study. For more about exporting and importing a
study, see
Export/Import/Reload a Study.
Select Folder Objects and Options
This is the complete list of objects that can be exported to a folder archive. When each of the following options is checked, the specified content will be included in the archive. Some options will only be shown when relevant content is present.
- Folder type and active modules: The options set on the "Folder Type" tab.
- Missing value indicators: The settings on the "Missing Values" tab.
- Full-text search settings: The settings on the "Search" tab.
- Webpart properties and layout: Page layouts and settings.
- Container specific module properties: Settings on the "Module Properties" tab.
- Project-level groups and members (Only available for project exports)
- Role assignments for users and groups: See Export and Import Permission Settings.
- Sample Status and QC State Settings: Exports QC state definitions including custom states, descriptions, default states for the different import pathways and the default blank QC state.
- Learn more about sample status export/import in this topic.
- File Browser Settings: Configuration of the Files web part such as which actions buttons are shown. Note that custom file properties and property settings are not included.
- Notification settings: The settings on the Notifications tab.
- Queries: Shared queries. Note that private queries are not included in the archive.
- Grid Views: Shared grid views. Note that private grid views are not included in the archive.
- Reports and Charts: Shared reports and charts. Private ones are not included in the archive.
- Categories: This option exports the Categories for report and dataset grouping.
- External schema definitions: External schemas defined via the Schema Browser.
- Experiments, Protocols, and Runs: Check to include experiment data including samples, assay designs and data, and job templates, if any are defined. Configured assay QC states are exported, but assignments of QC states to assay runs are not included.
- Experiment Runs: Select the above checkbox but deselect this box to skip exporting the runs themselves. Note that you cannot export experiment runs without also exporting the protocols and assay designs included with the above option.
- Sample Type Designs: Include definitions for sample types. If a Sample Type has a setting for "Auto-Link Data to Study", for example, this setting will be included in the export.
- Sample Type Data: Include Sample Type data. Note that you can export this data without the accompanying designs, though it will not be reimportable as is.
- Data Class Designs: Include definitions for data classes, including Sources in Sample Manager, and Registry Sources, and some Media definitions in Biologics.
- Data Class Data: Include Data Class data. Note that you can export this data without the accompanying designs, though it will not be reimportable as is.
- ETL Definitions (Premium feature): Extract-transform-load definitions.
- Files: The contents of the file repository.
- Inventory locations and items: (Premium feature) If any freezers have been defined in Sample Manager in this container, you will see this checkbox to include freezer definitions and storage information in the export.
- List Designs: The definitions of Lists in this container.
- List Data: The data for the above lists. This can be exported without the accompanying designs but may not be reimportable.
- Wikis and their attachments
- Study (Only present in study folders): Learn more in this topic: Export a Study.
- Panorama QC Folder Settings (Only present in Panorama QC folders): Learn more in this topic: Panorama QC Folders.
Select Export Options
Whether to
Include Subfolders in your archive is optional; this option is only presented when subfolders exist.
You can also choose to exclude protected health information (PHI) at different levels. This exclusion applies to all dataset and list columns, study properties, and specimen data columns that have been
tagged at a specific PHI level. By default, all data is included in the exported folder.
- Include PHI Columns:
- Uncheck to exclude columns marked as containing any level of PHI where exclusions apply.
- Check to include some or all PHI, then select the level(s) of PHI to include.
Similar PHI functionality is available when publishing a study. For more details see
Publish a Study: Protected Health Information / PHI.
To tag a field at a specific PHI level, see
Protecting PHI Data.
Select an Export Destination
Under
Export to:, select one the export destination:
- Pipeline root export directory, as individual files.
- Pipeline root export directory, as a zip file.
- Browser as a zip file.
You can place more than one folder archive in a directory if you give them different names.
Character Encoding
Study, list, and folder archives are all written using UTF-8 character encoding for text files. Imported archives are parsed as UTF-8.
Items Not Included in Archives
The objects listed above in the "Folder objects to export" section are the only items that are exported to a folder archive. Here are some examples of LabKey objects are not included when exporting a folder archive:
- Assay Definitions without Runs: An assay definition will only be included in a folder export when at least one run has been imported to the folder using it.
- File Properties: The definitions and property values will not be included in the folder archive, even when the File browser settings option is checked.
- Issue Trackers: Issue tracker definitions and issue data will not be included in the folder archive.
- Messages: Message settings and message data will not be included in the folder archive.
- Project Settings and Resources: Custom CSS file, logo images, and other project-level settings are not included in the folder archive.
- Query Metadata XML: Metadata XML applied to queries in the Schema Browser is not exported to the folder archive. Metadata applied to fields is exported to the folder archive.
When migrating a folder into another container using an archive, the items above must be migrated manually.
Import
When you import a folder archive, a new subfolder is not created. Instead the configuration and contents are imported into the current folder, so be sure not to import into the parent folder of your intended location.
To create the imported folder as a subfolder, first create a new empty folder, navigate to it, then import the archive there.
- To import a folder archive, go to (Admin) > Folder > Management and click the Import tab.
- You can import from your local machine or from a server accessible location.
Import Folder From Local Source
- Local zip archive: check this option, then Browse or Choose an exported folder archive to import.
- Existing folder: select this option to bypass the step of exporting to an archive and directly import selected objects from an existing folder on the server. Note that this option does not support the import of specimen or dataset data from a study folder.
Advanced Import Options
Both import options offer two further selections:
- Validate All Queries After Import: When selected, and by default, queries will be validated upon import and any failure to validate will cause the import job to raise an error. If you are using the check-for-reload action in the custom API, there is a suppress query validation parameter that can be used to achieve the same effect as unchecking this box in the check for reload action. During import, any error messages generated are noted in the import log file for easy analysis of potential issues.
- Show Advanced Import Options: When this option is checked, after clicking Import Folder, you will have the further opportunity to:
If the folder contains a study, you will have an additional option:
Select Specific Objects to Import
By default, all objects and settings from a folder archive will be included. For import from a template folder, all except dataset data and specimen data will be included. If you would like to import a subset instead, check the box to
Select specific objects to import. You will see the full list of folder archive objects (similar to those you saw in the export options above) and use checkboxes to elect which objects to import. Objects not available in the archive (or template folder) will be disabled and shown in gray for clarity.
This option is particularly helpful if you want to use an existing container as a structural or procedural template when you create a new empty container for new research.
Apply to Multiple Folders
By default, the imported archive is applied only to the current folder. If you would like to apply this imported archive to multiple folders, check
Apply to multiple folders and you will see the list of all folders in the project. Use checkboxes to select all the folders to which you want the imported archive applied.
Note that if your archive includes subfolders, they will not be applied when multiple folders are selected for the import.
This option is useful when you want to generate a large number of folders with the same objects, and in conjunction with the selection of a subset of folder options above, you can control which objects are applied. For instance, if a change in one study needs to be propagated to a large number of other active studies, this mechanism can allow you to propagate that change. The option "Selecting parent folders selects all children" can make it easier to use a template archive for a large number of child folders.
When you import into multiple folders, a separate pipeline job is started for each selected container.
Import Folder from Server-Accessible Archive
Click
Use Pipeline to select the server-accessible archive to import.
Related Topics
