Overview

The latest version of LabKey Server brings you next-generation technologies on top of an ever-more stable, more flexible and speedier platform. Significant performance enhancements in v9.3 mean that you’ll see faster page loads and swifter data processing across the product.

New capabilities introduced in this release are summarized below. For an exhaustive list of all improvements made in 9.3, see Items Completed in 9.3. Refer to the 9.3 Upgrade Tips to quickly identify behavioral changes associated with upgrading from v9.2 to v9.3.

Download LabKey Server v 9.3.

Performance

The speed of loading pages and processing data across diverse areas of LabKey Server has improved significantly. You will see speed improvements for:

• Importing lists on PostgreSQL. This now takes one third as much time on v9.3 as it took on v9.2.
• Copying assay results to studies or deleting runs
• Specimen, security and user administration pages
• Certain queries over lists and assays
• Servers with large folder trees
• Large wikis

External Schemas and Data Sources

External Data Sources. You can now define a database other than the LabKey database as a source of external schemas and tables. LabKey currently supports SQLServer, PostgreSQL and SAS data sources. You can define an arbitrary number of data sources in your labkey.xml file.

SAS Data Sources. SAS external schemas and data sources are now supported. Publishing SAS datasets to your LabKey Server provides secure, dynamic access to datasets residing in a SAS repository. Published SAS data sets are dynamic, meaning that LabKey treats the SAS repository as a live database; any modifications to the underlying data set in SAS are immediately viewable on LabKey. The data sets are visible only to those who are authorized to see them. Authorized users can view published data sets using the familiar, easy-to-use grid user interface used throughout LabKey. They can customize their views with filters, sorts, and column lists. They can use the datasets in custom queries and reports. They can export the data in Excel, web query, or TSV formats. They can access the data sets from JavaScript, SAS, R, and Java client libraries.

Study

Time-varying cohort assignments. LabKey Server now provides a sophisticated cohort tracking system that allows cohort assignments to change over time. A given participant may move through more than one of a study's designated cohorts (e.g., Negative, Acute, and/or Established) across their study visits. Documentation: User Guide for Cohorts and Admin Guide for Cohorts. Features:

• Queries and reports can now be time-aware; in other words, they can show results that reflect each participant’s cohort at the time the data was collected.
• The "Cohorts" dropdown above grid views provides options for filtering the view based on the times participants when were assigned to particular cohorts.
• Administrators still have the option to set up a study's cohorts as time-invariant, simplifying UI options for users.

Participant- and participant-visit-level comments. Specimen coordinators can now apply comments at the participant and participant-visit levels instead of just at the vial level. Coordinators can also move or copy comments from vials to participants or participant-visit pairs. All comments can be displayed in a "Comments" column in grid views. Documentation: User Guide for Comments, Admin Guide for Comments, Study Tutorial for Comments.

Study import/export/reload enhancements. Import/export/reload allows you to easily transfer a study from a staging environment to a live LabKey platform. It also allows you to populate a brand new study with the exported contents of an existing study. For similar groups of studies, this helps you leverage your study setup efforts. Features added in v9.3:

• Additional data types included in import/export/reload:
• Lists. All lists in the study folder are now exported if the lists option is checked in the study exporter. Study import will create new lists or replace existing lists with list schema, properties, and data in the study archive.
• QCStateLabel columns in datasets. This allows you to export labels for quality control states.
• Specimen repository settings when you do not have an existing specimen archive. Previously an archive was required.
• Enhanced error checking and logging during import/reload, including:
• Checking of all queries and custom views for errors during import/reload.
• Logging of status and errors to a single pipeline log that can be browsed from the UI. The log also provides counts for imported queries, views, reports, and lists.
• Enhanced pipeline processing, including the following options:
• Browse to a study archive or select a pipeline zip file whose name ends with .study.zip.
• Store multiple study.xml files (e.g., study.xml, foo.study.xml, or study001.study.xml) in the same directory.
• Reload (in addition to import) via pipeline browsing to local .zip file. If the study exists, the “Reload Study” button appears on the manage study page, linking to the Import/Reload Study page.
• Enhanced documentation for the study load file format for programmers generating file-based studies. See:
• XML Schema Reference, which provides documentation for LabKey XML schemas
• Serialized Elements and Attributes of Lists and Datasets.
• Improved UI text on import study page

Data Grids

Field/column designer enhancements. The columns of LabKey datasets, assays and lists are described as lists of fields, each of which has associated properties (e.g., Name, Description, etc). Improvements to the field designer allow you to set additional properties, reorder fields, add import aliases and adjust the visibility of columns. features added:

• Custom URLs for gridview columns. The "URL" property of any list/dataset/assay field can now be customized using substitution parameters that reflect live data in the grid. This allows you to produce URLs that are determined by the data stored in a particular row of a dataset. You might use a custom URL to link to custom details page for a particular participant or subject. Or you might set up an "Image" column that automatically provides links to files named with participant or subject IDs.
• Ability to change the order of fields in list, assay and dataset designs. The order of fields determines the order of columns in default grid views, as well as the insert, update, and detail views.
• Column aliases. You can now specify an arbitrary number of alternative column aliases for fields in the list/dataset/assay design editors. When LabKey Server imports data from files (TSV, Excel, etc) it uses these aliases to match file column headers to field names.
• Visibility/editability in insert, update, and details views. Users can now determine whether columns are hidden from the update and default views, plus whether they are read-only in the update view. Hidden fields are not part of the default grid views and will only be shown in "Customize View" when the user clicks on the checkbox to show hidden columns.
• Ability to export a list of dataset fields and field properties. The new "Export Fields" button on the dataset definition page allows an admin to export a TSV file that contains the header line and meta data in a format matching TSV dataset schema import.

Crosstabs and charts available for any data grid. Charts and crosstabs are now available by default over all query views. The Manage Views page also contains options for configuring the types of views allowed for different types of queries.

Assays

Transform scripts for run properties. Transformation scripts can now modify run- and batch-level properties during data import. As with transformation of uploaded data, a script can only transform run or batch properties that already exist in the assay definition. A script cannot create a new run or batch property.

New versions of TPP and X!Tandem. The newest version of LabKey Server includes the most recent versions of these tools for MS2 proteomics.

New NAb-curve fit algorithm. A third curve fit algorithm, "polynomial," is now available in the list of NAb curve-fit options displayed during NAb data import. This algorithm allows you to quantifying a sample’s neutralization behavior based on the area under a calculated neutralization curve, commonly abbreviated as “AUC”. The assay's run report (accessed through the details link) generates all graph, IC50, IC80 and AUC information using the selected curve fit method.

Viability assay. This new, built-in assay type can collect and organize results for all types of Guava runs, including AQC, EQC, and proficiency testing. The new assay tool enables the submission of cell viability and recovery data directly to a data portal, where lab and program staff can have easy access to all submitted data. This approach provides complete and efficient data uploads while simplifying a lab’s post-Guava assay workflows. The LABKEY.Query.selectRows API can be used to query uploaded assay results a Viability Assay.

Assay folder type. This new folder type makes it easier to set up and organize assays. It displays the "Assay List" web part and provides a list of available assays. It also provide links for setting up the pipeline, creating additional assays and managing existing assays.

JavaScript Client API

Auto-generation of Ext forms. LABKEY.ext.FormPanel can generate simple forms using query or assay metadata. It extends Ext.form.FormPanel and understands various LabKey metadata formats. When a LABKEY.ext.FormPanel is created with additional metadata, it will try to intelligently construct fields of the appropriate type.

Email notifications. LABKEY.Message.sendMessage sends an email notification message through your LabKey Server.

Group renaming. You can now rename groups either through the UI or the API. See: LABKEY.Security#renameGroup.

Effective permissions. Effective permissions are now provided for both users and/or groups as part of the following APIs: LABKEY.Security.getUserPermissions, LABKEY.Security.getGroupPermissions, and LABKEY.Security.getContainers.

Fine-grained control over the display of buttons and links in grid views and the LABKEY.QueryWebPart. For example, you can now hide the Edit and Details columns and the "Insert New" button in the button bar as well. LABKEY.QueryWebPart:

• showUpdateColumn
• showDetailsColumn
• showInsertNewButton
• showDeleteButton
• showExportButtons
• showBorders
• showRecordSelectors
• showPagination
• shadeAlternateRows
• printView

Query Validation. LABKEY.Query.validateQuery validates the specified query by ensuring that it parses and executes without an exception.

Display of LabKey SQL queries in Ext grids. Use the "sql" configuration property on the LABKEY.ext.Store constructor. The resulting Ext grid will be read-only.

LABKEY.Utils.requiresScript and LABKEY.Utils.requiresCSS. These APIs allow you to include a script file or Cascading Style Sheet (CSS) file in a page.

Improved LABKEY.Assay.DomainFieldObject syntax. This API's lookup properties (lookup.container, lookup.displayColumn, lookup.keyColumn, lookup.schema and lookup.table) now match those that are returned by the LABKEY.Query APIs. The older versions of these lookup properties are still supported, but are deprecated.

Developer menu for non-admin developers. Provides quick access to areas handy to developers, such as the schema browser and the client API documentation. If the user is not an admin, but is in the site developers group, that user should see a drop-menu in the upper-right where the Admin menu usually is.

Enhanced documentation in the LabKey JavaScript API Reference:

• Events documented
• Static fields, methods and events now marked static
• Additional examples provided.
• Cross-linking to related documentation topics improved

Developer Toolkit

Schema browser. This interactive schema exploration tool allows developers to easily determine the columns and data types that compose LabKey tables. It replaces the existing Query module exploration/management UI, so it provides links to validate queries, create new user-defined queries in each schema, edit the source or design of user-defined queries and access the "Schema Administration" page. New features:

• Users can view all schemas they have permissions to read in a container and get descriptions about what kind of data are stored in each schema.
• Users can view all tables and queries they have permissions to read in each schema and get descriptions about what kind of data are exposed from each.
• User-defined queries are differentiated from built-in tables/queries.
• Users can view all columns they are allowed to read in each query and get information about those columns, especially the following:
• Programmatic name (the name used in queries and the API).
• Data type.
• Description of the kind of data exposed in that column.
• For foreign keys, information about what table/column it joins to.
• Users can quickly view data in any of the tables/queries by clicking through to a standard grid view.

Custom insert/update/delete links. You can now customize insert, update and delete links for query views to go to their own custom pages. To customize the URLs for the destination page of these actions, you can either set them in your schema.xml file or add them by editing the query source's Metadata XML in the UI. This feature allows insert/update/delete URLs to be customized by file-based modules.

Site Management

Group renaming. You can now rename groups either through the security UI or the API. See: LABKEY.Security#renameGroup.

Folder for adding robots.txt and sitemap files. You can now place site-specific, static content (such as robots.txt and sitemaps) into the new extraWebapp directory, a peer to modules and labkeyWebapp. These files won't be deleted when the site is upgraded.

Easier setup for perl scripting engines. The scripting engine setup dialog now automatically offers the perl path if perl is already on the system path.

New Documentation, Tutorials and Demos

• Advanced List Tutorial and Demo
• LabKey XML Schema Reference
• SAS Integration
• User Guide: Dataset Quality Control
• Admin Guide: Dataset Quality Control Management
• Specimen Quality Control
• Add Screenshots to a Wiki