Table of Contents

guest
2021-04-12
Experiment / Assay Data
   Describe Assay Data Structure
   Import Assay Data
   Manage Assay Designs
   Manage Assay Data

Experiment / Assay Data


The data you obtain from running your samples through various instruments and experimental procedures can be uploaded into LabKey Sample Manager. To describe how each kind of data should be interpreted, stored, and associated with the samples themselves, you create a framework called an Assay Design. Using this framework, many runs of data can be uploaded and stored in a way that makes it easy to interpret and analyze.

You can create as many different assay designs as you will need to describe the different types of data you will upload. In this section, learn about describing, importing, and updating assay data.

Topics

Tutorial

Get started with assay designs in the Sample Manager tutorial step: Tutorial: Define Assays.




Describe Assay Data Structure


This topic is under construction for the 21.4 (April 2021) release of Sample Manager. For the previous version of this documentation, click here.

The results of experiments and instrument runs can be uploaded and associated with samples that are registered in the system.

This topic covers how to create a "template" describing each different type of experiment data you will upload. LabKey Sample Manager calls these data descriptions Assay Designs. All assay data will be associated with a sample, so a mapping to the correct Sample Type is required.

Create Assay Design

From the main menu, select Assays and then select Create > Assay Design. (Before any assays have been created, you will see a direct link to "Create an assay design" on the menu.)

  • Give the assay design a Name (Required). The name must be unique and cannot be changed after the design is created.
  • Enter a Description to give more information (Optional).
  • Choose Editing Settings using the checkboxes. Either runs, results, or both can be editable.
  • Add the necessary fields and sample mapping described below before clicking Finish Creating Assay Design.

Add Run Fields

Run fields represent information that will be set once per run of data, such as a spreadsheet of individual result rows uploaded together. All rows will have the same value for any run fields you define.

  • Click the Run Fields section to open it.
  • If you have a specially prepared JSON file of field definitions, you can drop it into the Import fields from file panel.
  • Otherwise, click Manually Define Fields.
  • Use Add Field to add each run field you need (one is created for you).
    • Enter a Name (without spaces)
    • Select the Data Type.
    • To set more properties of the field, click the expansion icon.

If you add any extraneous fields, delete them by clicking the .

Add Results Fields

Results Fields represent the data information in the spreadsheet. You can define results fields in several ways:

  1. Upload a sample spreadsheet to infer all the necessary fields. Either drag and drop a file into the upload area, or click within the same area to select a file directly.
  2. Import a specially designed JSON file containing field definitions by dropping or selecting it in the same panel.
  3. Click Manually Define Fields below the panel to define fields in the editor.
In the Sample Manager tutorial, you can try this procedure with our sample data in the step: Tutorial: Define Assays.

Results fields will be inferred from your upload.

  • Note that the data itself will not be imported at this time.
  • Once fields have been inferred, you can make changes if needed.
  • For instance, if your results spreadsheet also includes columns for the run fields you defined, you may need to delete the duplicate fields. Click the to delete a field.
  • If your result spreadsheet contains any reserved fields, they will not be shown on the inferral list, but they will always be created. You will see a blue banner indicating this reason for not seeing specific fields from your file.
There must be a field mapping assay data to the sample it represents. If your fields include one named "SampleID", it will be automatically mapped. Otherwise, you will see a blue notice and need to follow the steps in the next section before clicking Finish Creating Assay Design for your assay design.

Map to Samples

In order to associate all assay data with the sample it represents, every assay design must included a field which maps to samples in the system. The data type Sample is used to represent that mapping as a lookup into the Sample Type containing the samples.

After you infer fields, if one of them is named "SampleID" (such as when you use a naming pattern), it will be mapped automatically. If not, you will see a blue message section asking you to map one of the fields to be the Sample Lookup. The pulldown menu will be populated with the results fields that were inferred. If you need to add a new field to provide the sample linkage, use Add new field. In this example, the SampleID field will be our lookup.

As soon as you select it from the dropdown, the chosen field changes to be of type Sample and opens the properties panel. Select the desired Sample Type from the dropdown.

Click Finish Creating Assay Design in the lower right when finished.

The assay design describing the structure of assay data has now been created. Note that the actual data contained in the spreadsheet you used to infer fields was not imported.

Now you can add experiment data that matches this structure and map it to samples and other associated data.

Related Topics




Import Assay Data


This topic describes how to import assay data about your samples. You should already have defined the assay structure of your data and how it will map to your samples. You can follow this topic to upload our example data to the "Tutorial Assay" once you have created it.
Note that our assay data assumes you have followed the previous steps and already created enough samples in the "Tutorial Samples" Sample Type to have the Sample IDs "Tutorial-003" through "Tutorial-012" available to associate with our data. You can confirm this by selecting Tutorial Samples from the main menu and add more if necessary.

Import Data

Locate the correct assay design and click Import Data. You can select it by name from the assays list on the main menu or using the dropdown in the Recent Assay Data panel on the home page.

Enter Run Details

Enter the Run Details requested. Any fields that are required will be marked with an asterisk (the getting started tutorial does not include any required fields). The Assay Id field will be the name for this run of data. If you don't enter a name here, an Assay ID will be generated for you. If you upload a data file, the filename will be used. Otherwise, it will be a concatenation of the assay design name and the date and time.

For the getting started tutorial, enter:

  • Assay Id: "Run1"
  • Date: "2019-10-01" (the time "00:00" will autopopulate if you don't select a time)
  • Instrument: "INS-01"

Enter Data (Results)

You can enter the result data in one of three ways:

Upload Files

Drag and drop the file(s) containing your result data into the target area or click the region to select a file. If needed, you can first download a template file of the expected format by clicking Template.

The first three rows of data will be shown for a quick verification before you upload.

If everything looks as expected, click Import to import the data. For large files (over 100kb), you may see a notice that your import will be done in the background, freeing you to continue using the app for other use. Learn more about background imports in this topic.

Copy-and-Paste Data

If drag and drop upload is not convenient, you can copy and paste a tab-separated set of values directly into the box on the Copy-and-Paste Data tab. The column headings must be included in what you paste.

Click Import to import the data. If your copy is more than 1000 rows, you may see a notice that your import will be done in the background, freeing you to continue using the app for other use. Learn more about background imports in this topic.

Note that you cannot copy and paste data that uses the "Multi-line Text" data type. To import multiline text fields, use another method.

Enter Data into Grid

If neither method described above is appropriate, you can use the Enter Data Into Grid tab to type directly into the entry window. Start by adding the number of rows you want to add and clicking Add row(s).

Enter values directly into the grid. Note that when you begin typing into the field you have defined as mapping to samples (shown here "SampleID") it will show the set of samples available. Select the correct mapping for each row.

Note that any columns that are required will be marked with an asterisk (*). Before you can import the data, these columns will need a valid value in every row.

Bulk Insert

You can also use the Bulk Insert button to prefill the grid with many rows of data with some or all values in common. Enter the number of rows to add and provide values that those rows should share. You do not need to enter a value for every column.

After bulk inserting rows, you can hand edit as needed in the grid view.

Bulk Update

Once data has been entered into the grid, either directly or using bulk insert, you can select one or more rows and click Bulk Update to assign new values to all the selected rows for one or more columns. In the popup, use the "Disabled/Enabled" slider to activate updating any row and enter the new value. Values in columns which are disabled in the update will remain unchanged.

Delete Rows

If you enter extra rows by mistake, you can select them using the checkboxes and click Delete Rows.

Complete Import and Review Data

Click Import when ready to import. If there are any missing or invalid values, you will need to fix them before the import will complete.

When your data has been imported, you will see the results for the specific run you just entered. Run details are at the top, results in a grid below.

The results grid can be searched, sorted, and filtered. Learn more in the topic: Data Grid Basics

Related Topics




Manage Assay Designs


This topic describes how to edit and manage existing assay designs. To create new designs, see this topic: Describe Assay Data Structure

View All Assay Designs

The assays defined are all listed on the main menu under Assays. To see the list of assays as a heatmap or a grid, click the heading Assays.

By default, you will see the Grid listing the name and description of each defined assay. Select Heatmap from the View menu to see a visual representation of relative count of runs over the past 12 months for each assay. Darker blocks indicate more runs. Hover over any block for details.

Manage Assay Design

Click the name of any assay from the main menu, grid, or heatmap to open the overview page for that assay. You will see the assay description, as well as a grid of runs.

Use the (three bar) menu in the upper right to edit or delete the assay design. Administrators can also view the audit history for this assay.

In addition to the overview tab, tabs for runs and results for the assay are available in the upper left. Click Runs for the grid of runs and Results for the grid of result data.

From any page within the assay interface, you can click Import Data to import a new run.

Edit Assay Design

Click the name of any assay from the main menu or assay grid to open the overview page.

  • To edit the design select > Edit Assay Design.
  • The panels for editing properties and fields in your assay will open.
  • Use the field editor to adjust as needed. Remember that if you delete any fields, all their data will be deleted as well.
  • When finished making changes, click Save.

Note that you have the ability to edit the assay results field mapped as a sample lookup. Use caution if you change the sample lookup for this field, as it may change or remove the links in your existing assay results.

For instance, if you create an assay design that maps result data to "All Samples", import some data mapped to samples of both "Blood" and "Serum" types, and then later edit this sample lookup to map results to only "Serum" samples, the existing run results linked to "Blood" samples will show only the row and not the link.

You can work around this limitation by defining a copy of the assay design mapped to a narrower set of samples instead of changing the mapping lookup in the original assay design.

Delete Assay Design

Click the name of any assay from the main menu or assay grid to open the overview page.

  • To delete the design select > Delete Assay Design.
  • Note that when you delete a design, all runs of data associated with it will also be deleted. Deletion cannot be undone.
  • Confirm in the popup to complete the deletion.

Related Topics




Manage Assay Data


This topic describes how to work with assay data runs and results.

Manage Runs

First open the assay design of interest, by using the main menu and clicking the assay name, then click the Runs tab in the upper left to manage runs. From this page, you can reimport a run, and delete runs if needed.

Edit Run Details

If your assay has editable runs and you have sufficient permissions, you can edit run details.

Click the name of the run, then use the (Edit) icon in the Run Details panel to open them for editing. You will see an entry panel you can use to make changes.

Click Save Run Details when finished.

Re-Import Run

If you have a change of data or metadata after importing a run, and have editable runs and/or results, you may be able to make the change directly. However, if your runs/results are not editable, you can import a revised version of the run as follows. LabKey Sample Manager will track run re-imports and maintain data integrity.

Opening the run details (shown above) and select > Re-Import Run.

You will see the interface from when you originally imported the run, including the values and datafile previously entered. Make changes as needed and click Re-Import.

A note about event logging: When you re-import an assay run, two new assay events are created:
  • Assay Data Re-imported: This event is attached to the "old run" that is being replaced.
  • Assay Data Loaded: This event is for the "new run" you import.

Delete Run

To delete a run, either:

  • Start from the run details page and use > Delete Run
  • OR, select the checkbox for the run on either the Overview or Runs tab and select Manage > Delete Run.

Manage Results

The result data for your assay is available on the Results tab. Results are individual rows within runs. You cannot add results rows within the user interface. To do so, either import a new run containing the results, or add them to an existing run by reimporting the run after adding the additional rows to the run data file.

Edit Selected Results in Grid

If your assay has editable results, and you have sufficient permissions, you can select one or more rows using checkboxes and select Manage > Edit Selected Results in Grid.

A grid will be shown, with a row for each row you selected, allowing you to edit the necessary values. Click Finish Updating # Results to save your changes.

Edit Selected Results in Bulk

If you are editing a number of rows to insert shared values, select the desired rows with checkboxes and select Manage > Edit Selected Results in Bulk.

An editing popup will let you select which field or fields you want to batch update. By default, all fields are disabled; choosing Enabled will let you enter a value to assign for that field in all rows. Shown here, the MCV field will be updated with a shared value, but all other fields left unchanged.

After entering updated values, you can leave the bulk popup using either:

  • Edit with Grid to switch to updating in a grid format (with the bulk changes you just made already applied). Use this option if you want to make individual as well as bulk row changes.
    • Be sure to click Finish Updating # Results when finished with the grid update to save both the bulk changes AND individual changes you made.
  • Update Results if no further editing is needed. The bulk updates will be saved.

Delete Results

To delete one or more rows of results within any run, either open the run from the Runs tab or find the desired rows on the Results tab. Use sorting and filtering to help you isolate rows of interest.

Check the box(es) for the row(s) you want to delete and select Manage > Delete Results.

Note that you can only delete 10,000 assay results in one operation. To delete more than that, perform the deletion in batches.

Related Topics