Import Samples from File

2024-03-29

When creating new Samples, the Add > Import from File option (or Add Samples > Import from File from the dashboard) lets you upload a spreadsheet of sample data, streamlining the process over entering data into a grid. You can also update data for existing samples via Edit > Update from File from a grid of samples.
Note that fields of type File are not included in file import methods. Values must be individually added as described in this topic: Attach Images and Other Files.

Obtain File Import Template

An import template shows all the expected columns for your data structure, making it easier to ensure your imported data will conform to system expectations. Templates are available for Sample Types, Source Types, and Assays from the main dashboard for the type of structure.

From the main menu, select the target Sample Type. Click the Template button to download the template for this type of sample.

  • Use this template as a guide for ensuring your data matches the expected columns.
  • Note that any "File" fields will be omitted from the downloaded template.
  • When populating the template, you may not need to include all columns. For example, if you have defined import aliases for sources or parents, all possible columns will be included in the template, but you only need to include the ones you want to use.

Import Samples From File

You can start from the home page of the application or the Sample Types dashboard and select Add Samples > Import from File or from the overview page for a specific Sample Type, select Add > Import from File.

  • Set the Sample Type for the Samples you will import.
    • If you started from a specific sample type grid, the type will be prepopulated and you will not see the dropdown.
    • If you want to import samples of multiple types, select Multiple sample types. Learn more below.
  • If you are including storage information for the samples you're importing, select the appropriate radio button:
    • Add to existing storage only: all storage locations must exist already.
    • Create new storage too: a combination of existing and new storage units are allowed.
  • Use the Template button if you didn't already download a template file for your data.
  • Drag and Drop the file into the target area to upload it.
    • You will see the sample information obtained from the file in the panel.
    • Only fields included in the definition of the Sample Type will be imported. In this example, the Sample ID will be generated for you, so is not included in the imported file or shown here. If any fields are unrecognized, they will be ignored and a banner will be shown.
    • Note that you cannot edit the values here. If you see an error, you can use the X to delete the loaded file, make changes offline, and reselect the revised file.
  • Click Import to create the samples from the data in the file.
You will see the sample grid for that type, with a banner message indicating how many were created and offering actions for working with them.

Additional examples and details are available here:

Import Samples Across Multiple Sample Types

To import samples across multiple Sample Types, select the Sample Type option Multiple sample types.

Ensure the file contains a "Sample Type" column that contains the name of the Sample Type (eg. Blood) for every row, in order to map the samples being added to the correct sample type.

It is good practice to check after a file import that the number of samples imported matches your expected total for each type. Confirm by comparing the number of samples in the notification of the completion of the background task with the number shown in the new sample grid.

Note: When there are more than 1000 samples of a given type in a multi-Sample Type file, you should either import one Sample Type at a time, or break the import into smaller multi-Sample Type chunks. This issue will be fixed in a future update.

Import Samples and Create New Storage

The default Storage Option when you import samples from file is to Add to existing storage only, provided the location details are included in the file. Learn more about the details that you need to provide here.

To create any new storage locations while importing samples from file, i.e. to assign new samples to either new or existing storage locations within the same file, a user with the "Storage Designer" role can select the option Create new storage too.

Ensure the file contains a "Storage Unit" column that corresponds to the name of the Storage Unit (eg. "9x9 Box" to use one of the default box types) to create the appropriate type of storage unit for at least the first row placing a sample into that unit. This is in addition to the "Storage Location, Storage Row, and Storage Col" values required (in every row) for adding samples to storage.

The creation of new storage during import is determined by parsing the value provided in the "StorageLocation" field for each row.

  • If any locations in that path do not exist, they will be created.
  • The "StorageLocation" field value should be slash-separated and must end with a terminal storage unit.
  • If any levels in this field value do not exist, they will be created.
  • You can also create an entirely new storage system (freezer) with this process, using a value like "MyNewFreezer/Shelf1/Box1" where "MyNewFreezer" does not already exist.
    • You can also precede new storage with a / to make it clear that a new top level system is to be created, i.e. "/MyNewFreezer/Shelf1/Box1".
  • If the terminal storage unit (final element in the field value) does not already exist, it will be created as the type of unit specified in the "Storage Unit" field.
  • This means that only the first row for a new storage unit needs to provide the correct type of unit to create.
  • If all units in the full "StorageLocation" field already exist, the "Storage Unit" value is not read for that row.
New storage locations are created in the background during the import, and the user will see a notification when the import completes. If the import and creation of samples fails for any reason, the new storage locations will not be created.

Update Existing Samples

To update existing samples, or merge a combined spreadsheet of new and existing samples, select Edit > Update from File. The update page is very similar to the Import from File page, with additional Update Options. The default is Only update existing samples.

  • Any data you provide for the rows representing samples that already exist will replace the previous values.
    • When updating, you only need to provide the columns you wish to update. Existing data for other columns will be left as is.
    • Important: All columns you upload will be updated in the system. If you include an empty column, all data for that data will be deleted.
    • Note that you cannot update an existing sample to make it an aliquot of another sample.
Just as for importing new samples from file, you can make use of import templates to know which fields you want to update. You might start from an existing grid of samples, make adjustments offline and/or add new samples to it, and then Edit > Update from File by first downloading the update template, merging your data to fit, then importing. Be sure to remove all columns you don't plan to update.

A common scenario for using "update" for samples is to add storage information for a group of samples that are already registered in the system. Learn more in this topic: Migrate Storage Data into LabKey

Merge by Allowing New Samples

The Create new samples too update option controls whether you will be able to merge the incoming file or simply update existing rows.

  • When this option is not selected, the default, the update will fail if rows are included that do not match existing samples already in the system.
  • When selected, any rows for samples that do not yet exist will add them as new samples.

Related Topics