Example Code: QC Reporting: /Documentation

Example Code: QC Reporting

This topic demonstrates several ways to implement quality control trend reporting on data within LabKey Server. The implementation methods include:

Using the R API and Rlabkey package to generate the report within RStudio
Using integrated R reporting within LabKey
Building a QC Trend Report for the assay through the user interface

The tutorial steps are:

Set Up the Workspace
Option 1: Use Rlabkey Package
Option 2: Use R Within LabKey
Option 3: Create QC Trend Report

This content was originally presented during the 2018 LabKey User Workshop. Premium edition users can watch the presentation here:

2018 LabKey User Workshop: Using Native and Integrated Tools for Reporting

Set Up the Workspace

Prerequisites

1. Premium edition and Developer role: To complete all the steps in this topic, you must be running a premium edition of LabKey Server version 18.3 or later. For example, you might use a local server built with the premium module installed using a developer machine. These steps assume you can create new projects and folders and have the Platform Developer role on your server. If you are not using a local dev machine, you may need to work with an administrator to obtain the necessary access.

2. R Scripting in LabKey: On your LabKey Server, open the Admin Console and confirm that you have configured an R engine. If needed, you can have different R engines configured per container if needed. Note the version of R you are using. It may be included as part of the path on your local machine.

This topic was developed using R version 3.5.1; if you are running an earlier version and run into package installation or other problems, try upgrading.

3. RStudio: To complete all three plot options, you should be running RStudio on your local machine. To install RStudio on your machine, download the free desktop version from the RStudio site.

Start RStudio and install these packages (available on your CRAN mirror):

curl
ggplot2
Rlabkey: Contains the LabKey query APIs. See documentation here.

You want to make sure the version of R used by RStudio is the same as the version configured on your LabKey server (see #2 above). Within RStudio, type "version" in the console. If you need to change it, select Tools > Global Options..., then update the R version path to match the R version your server using. Click Apply and OK to continue.

4. Credentials: RStudio needs login credentials to access LabKey Server via the API. You can either use your own credentials or a session key if enabled on your server. To avoid embedding the credentials directly in your scripts, you can use a netrc file with either your login information or the API session key you generated. You may need to set the %HOME% environment variable to point to the directory where the netrc file resides.

Note that when you generate a session you will see the R script needed to authenticate your scripts. You can also use this within your scripts if you prefer.

Set Up the Folder

In this step, you import a folder archive that includes assay data on which the QC reports are based. The assay data is loosely based on Luminex data and represents results of running a daily check to see if a control sample was measuring within an expected range.

1. Create your working folder.

It is good practice to create a new folder to avoid overwriting anything or interfering with anyone else's work.
Log in to your server and navigate to your "Tutorials" project. Create it if necessary.

If you don't already have a server to work on where you can create projects, start here.
If you don't know how to create projects and folders, review this topic.

Create a new subfolder "QC Report Demo". Accept all defaults.
Note: If you use a different folder name or path, make that substitution below where we refer to your "QC Report Demo" folder.

2. Import the demo folder archive.

Download this archive but do not unzip it: QC_Report_Demo.folder.zip
On LabKey Server, navigate to the "QC Report Demo" folder you created.
Select > Folder > Management.
Click the Import tab.
Click Choose File or Browse and select the archive you downloaded.
Click Import Folder.
When the Status shows "COMPLETE", click the link QC Report Demo near the top of the page to go to the home page of the folder. You may need to refresh the page to see the COMPLETE status.

Option 1: Use the External R API

Data stored in LabKey can be extracted for use in other reporting platforms. Here we pull the assay data out to create visualizations in RStudio. Using the selectRows API shown below works similarly for other scripting platforms.

Obtain selectRows Script

First, we obtain the script for extracting the data from LabKey into RStudio.

On the main page of your "QC Report Demo" folder, click View Results above the QCTrendReportGPAT Runs grid.
Click (Export), then click the Script tab.
Select R and click Create Script.

A new browser tab will open with the necessary script for pulling this data grid into RStudio. Copy it to a new .R script file or you can also use any text file for making the following edit if necessary.
If it is not already present, add a parameter setting to the selectRows call in your saved script: colNameOpt="rname", being sure to include commas between all parameters. This will convert column names to valid R names. See the RlabKey documentation for details.

The script will look something like this:

## R Script generated by LabKey Server on ...

library(Rlabkey)

# Select rows into a data frame called 'labkey.data'

labkey.data <- labkey.selectRows(
    baseUrl="http://localhost:8080/labkey", 
    folderPath="/Tutorials/QC Report Demo", 
    schemaName="assay.General.QCTrendReportGPAT", 
    queryName="Data", 
    viewName="", 
    colFilter=NULL, 
    containerFilter= "Current" , # Added comma
    colNameOpt="rname"           # Added line to transform column names to valid R names
)

Notice that the script includes your own baseURL and folderPath which could differ from the above. The schemaName specifies the assay data included in the demo folder archive. Also note that you could further edit this script to add column filters or sorts if needed. For instance, you might add colSort="Date", to sort results by the date column.

Execute selectRows in RStudio

Open RStudio, then copy and paste the selectRows script you edited above into the RStudio console window, then hit enter. Note that this script runs on the LabKey Server as the user specified in the netrc file.

You will notice the upper right panel now shows the "labkey.data" object is available. Click it to show the grid in the upper left.

Troubleshooting tip: If you are not successful loading data into RStudio, check that your netrc file contains the correct login credentials, or that they were included in your script. For example, you might see this error: Error in processResponse(response) : HTTP request was unsuccessful. Status code = 401, Error message = User does not have permission to perform this operation indicating that the R API was not granted access to the server in the script using the credentials available.

Create Visualizations in RStudio

Next, execute the following R script. Paste it into the RStudio console and hit enter:

library(ggplot2)
theme_set(theme_bw())

# first plot without the guide set lines
ggplot(aes(x = date, y = value), data = labkey.data) +
  ggtitle("MFI") +
  xlab("Experiment Date") + ylab("") +
  geom_line() +
  geom_point()

You will see an MFI (mean fluorescence intensity) plot in the panel on the right.

Next, paste the following additional script into the same console window and hit enter again.

# Levey-Jennings plot with expected range of mean +/- 3 stdDev
mean <- 25.03297
sd <- 3.82473
ggplot(aes(x = date, y = value), data = labkey.data) +
  ggtitle("MFI") +
  xlab("Experiment Date") + ylab("") +
  geom_errorbar(aes(ymin = mean - (3 * sd), ymax = mean + (3 * sd)), alpha = 0.25, colour = "red") +
  geom_errorbar(aes(ymin = mean - (2 * sd), ymax = mean + (2 * sd)), alpha = 0.25, colour = "blue") +
  geom_errorbar(aes(ymin = mean - (1 * sd), ymax = mean + (1 * sd)), alpha = 0.25, colour = "green") +
  geom_hline(yintercept = mean) +
  geom_line() +
  geom_point()

This second script segment creates a Levey-Jennings plot showing expected ranges based on standard deviation. You can see a couple of outliers, one within the first month in this example.

Option 2: Use R Within LabKey

You can also use R directly within LabKey to create similar visualizations from the same data. One advantage of using this method is that you have live access to the data. If it changes, your visualization will automatically update. You can also make this type of report available on a portal page or in a wiki on LabKey Server for viewing by other users. You can learn more about the R Report Builder in this topic.

On your LabKey Server, navigate to your QC Report Demo folder.
In the QCTrendReportGPAT Runs web part, click View Results.
Select (Charts/ Reports) > Create R Report.

Paste this script in place of the default content of the Source tab:

library(ggplot2)
theme_set(theme_bw())

# Levey-Jennings plot with expected range of mean +/- 3 stdDev
mean <- 25.03297
sd <- 3.82473
PLOT <- ggplot(aes(x = specimenid, y = value), data = labkey.data) +
   ggtitle("MFI") + 
   xlab("Experiment") + ylab("") +
   geom_errorbar(aes(ymin = mean - (3 * sd), ymax = mean + (3 * sd)), alpha = 0.25, colour = "red") +
   geom_errorbar(aes(ymin = mean - (2 * sd), ymax = mean + (2 * sd)), alpha = 0.25, colour = "blue") +
   geom_errorbar(aes(ymin = mean - (1 * sd), ymax = mean + (1 * sd)), alpha = 0.25, colour = "green") +
   geom_hline(yintercept = mean) +
   geom_line() + 
   geom_point()

png(filename="${imgout:labkeyl.png}", height=450, width=1500, units="px")
print(PLOT)
dev.off()

Click the Report tab to see the resulting report.

If you see an error that specimenid does not exist, you may have created the report on the runs grid instead of the results. Click View Results and retry creating the report.

The X-axis differs in label and space between dates, but you can see the same low 'outlier' in the first month of experiment dates is shown here.
Return to the the Source tab to save this visualization.

Option 3: Create QC Trend Report

Another method for quality control trend tracking is available through the user interface without writing R code. Created within LabKey, these reports can be shared with others and are updated as the data changes.

Navigate to the main page of your QC Report Demo folder and click View Results. Above the grid, select (Charts/Reports) > Create QC Trend Report to open the wizard.

Enter the details:

Choose a Label for your report.
The Schema, Query, and View fields will be set based on the dataset we were viewing when we opened the creation wizard.
From the Assay dropdown, select the "QCTrendReportGPAT" assay to associate this report with it.
Check the box for Shared if you want to be able to share with others.
Click Next.

On the next page, you select the configuration parameters.

Select a Date column; all the date/time columns in the dataset are available.
You can choose a different X-Axis Label column, or leave it blank to use the Date column on the x-axis.
Select Graph Parameters if you want to be able to show different trends. To match the earlier examples, we will not specify any here.
Select Metric(s) from among the columns marked as measures. Again to match our earlier plot, we choose "MFI".

Click Save. You first see the report grid.
Click View Graph to see the plot.

Notice this is the same as the first RStudio plot we showed.
To add guide set lines, click Create New under the plot.

To match the earlier plots, enter the Start Date of 2018-06-01 to apply this set to all points.
Enter the Mean and Std Dev; again to match the earlier plot enter 25.03 and 3.82 respectively.
Click Submit.
Now the guide set is applied. Notice again we see the same outliers.

LabKey QC Trend Reports include additional features like marking the outliers as red and the ability to apply multiple guide sets.

Once defined, you can access QC trend reports from grids of assay data.

LabKey Support

LabKey Support