Table of Contents

guest
2020-07-05
     Install LabKey
       Supported Technologies
       Example Hardware/Software Configurations
       Premium Resource: Reference Architecture / System Requirements
       Installation Checklists
         Install on Linux: Main Components
         Install on Windows: Main Components
         Installation: Tomcat Configuration
         Installation: SMTP, Encryption, LDAP, and File Roots
         Installation: Third-Party Components
         Troubleshoot Server Installation and Configuration
           Installation Error Messages
           Collect Debugging Information
         Common Install Tasks
           Configure Tomcat Webapp Memory
           Creating & Installing SSL/TLS Certificates on Tomcat
           Install Microsoft SQL Server
           Install and Set Up R
             Configure an R Docker Engine
             Determine Available Graphing Functions
             Configure R on Linux
           Install SAS/SHARE for Integration with LabKey Server
           Server Bootstrap Properties
           Set Up Robots.txt and Sitemaps
           GROUP_CONCAT Install
           Sending Email from Non-LabKey Domains
           PremiumStats Install
           Deploying an AWS Web Application Firewall
           Configure the Virtual Frame Buffer on Linux
       Community Edition Modules - Descriptions
         Additional Source Modules

Install LabKey


These topics explain the options and process for installing LabKey Server.

If you are upgrading an existing installation, see Upgrade LabKey.

If you wish to evaluate LabKey Server, consider creating a trial server instance. Both hosted and local trial options are available.

Installation Topics

Troubleshooting

Other Install Topics


Premium Resources Available

Subscribers to premium editions of LabKey Server can learn more with these topics:


Learn more about premium editions




Supported Technologies


Supported Technologies Roadmap

This chart summarizes server-side dependency recommendations for past & current releases, and predictions for upcoming releases.

Do not use: not yet available or tested with this version of LabKey
        Recommended: fully supported and thoroughly tested with this version of LabKey
Upgrade ASAP: deprecated and no longer supported with this version of LabKey
Do not use: incompatible with this version of LabKey and/or past end of life (no longer supported by the organization that develops the component)

 

ComponentVersionLabKey 19.2.x
(Jul 2019)
LabKey 19.3.x
(Nov 2019)
LabKey 20.3.x
(Mar 2020)
LabKey 20.7.x
(Jul 2020)
LabKey 20.11.x
(Nov 2020)
JavaJava 15 
Java 14   
Java 13   
Java 12  
Tomcat9.0.x     
8.5.x     
7.0.x  
PostgreSQL12.x    
11.x     
10.x     
9.6.x     
9.5.x     
9.4.x  
Microsoft SQL Server2019    
2017     
2016     
2014     
2012     

Browsers

LabKey Server requires a modern browser for many advanced features, such as locking data table column headers in place when scrolling, creating charts, or visualizing security group relationships. To take advantage of all features, we recommend upgrading your browser(s) to the latest release. Since this is not always possible, the LabKey team attempts to maintain basic compatibility and fix major issues with the following browsers:

As a general rule, LabKey Server supports the latest version of the browsers listed above. To review and download the latest versions, click the links provided. Note that Internet Explorer has reached end of life and is no longer supported.

If you experience a problem with a supported browser feel free to post the details to the support forum so we're made aware of the issue.

Java

As of LabKey Server 20.3.2, we recommend using the latest point release of AdoptOpenJDK 14 64-bit (x64) with HotSpot JVM (currently 14.0.1+7), the community-supported production-ready distribution of the Java Development Kit. LabKey continues to support AdoptOpenJDK 13 with HotSpot JVM (current version 13.0.2+8) and Oracle OpenJDK, Oracle's open-source build of the JDK.

We strongly recommend upgrading from OpenJDK 12 immediately, since public updates (i.e., patches to address security and reliability issues) for this version are no longer provided. Support for JDK 12 will be removed in LabKey Server 20.7.0.

LabKey Server has not been tested with other Java distributions such as the Oracle commercial Java SE, Amazon Corretto, Red Hat, Zulu, OpenJ9, etc.

Apache Tomcat

We recommend using the latest point release of Apache Tomcat 9.0.x (currently 9.0.36).

LabKey continues to support older version 8.5.x, though we strongly recommend installing the latest point release (currently 8.5.56); older point releases have major compatibility issues and security vulnerabilities. Note that Tomcat 7.0.x has been slated for end-of-life by the Tomcat team; we strongly recommend upgrading to Tomcat 9.0.x immediately. Support for Tomcat 7.0.x will be removed in LabKey Server 20.7.0.

We recommend installing Tomcat using the binary distributions; if a package manager is used, the lib directory may be installed in a different location. Also, the packages sometimes include alternative versions of some components (like JDBC connection pools) that can cause incompatibilities.

We recommend not using the Apache Tomcat Native library; this library can interfere with SSL and prevent server access via LabKey's client libraries.

PostgreSQL

For installations using PostgreSQL as the primary database, we recommend using the latest point release of PostgreSQL 12.x (currently 12.3).

For those who can't transition to 12.x yet, LabKey continues to support PostgreSQL 11.x, 10.x, 9.6.x, and 9.5.x, though here also we strongly recommend installing the latest point release (currently 11.8, 10.13, 9.6.18, and 9.5.22) to ensure you have all the latest security, reliability, and performance fixes.

PostgreSQL provides instructions for how to upgrade your installation, including moving your existing data database.

Microsoft SQL Server

For installations using Microsoft SQL Server as the primary database, we recommend using Microsoft SQL Server 2019, which we've tested on both Windows and Linux.

LabKey continues to support SQL Server 2017, 2016, 2014, and 2012. LabKey does not support SQL Server 2008 R2 or earlier releases.




Example Hardware/Software Configurations


This topic shows example hardware/software configurations for different LabKey Server installations. These are intended as guidelines only -- your own configuration should be adjusted to suit your particular requirements.

Small Laboratory Installation

The following configuration is appropriate for 10-20 users with small file and table sizes. We assume that the server and database are located on the same machine.

CPUs2+ CPUs or Virtual CPUs
RAM4GB (minimum) - 16GB (recommended)
Disk Storage164GB (64GB for File Storage, 100GB for database storage)
Software• OS: Linux or Windows (LabKey Server is supported on both Operating Systems. Select the operating system which is best supported by your organization)
• Java
• Tomcat
• DB software (PostgreSQL or MS SQL Server)
• See Supported Technologies for the specific versions to use.

As usage increases, increase the amount of RAM memory to 8GB at a minimum (and also increase the memory used by Tomcat and the database accordingly).

Large Multi-project Installation

The following configuration is appropriate for a 100s of users working on multiple projects with large files and data tables.

We recommend placing the web server and the database server on different machines in order to optimize maintenance, update, and backup cadences.

Machine #1: Web Server

CPUs4+ CPUs or Virtual CPUs
RAM8GB (minimum) - 16GB (recommended)
Disk Storage64GB (for OS and LabKey binaries), 512GB (for file storage)
Software• OS: Linux or Windows (LabKey Server is supported on both Operating Systems. Select the operating system which is best supported by your organization)
• Java
• Tomcat
• See Supported Technologies for the specific versions to use.
Network1 GB/s

Machine #2: Database Server

CPUs4+ CPUs or Virtual CPUs
RAM8GB (minimum) - 16GB (recommended)
Disk Storage100GB (for database storage)
Software• OS: Linux or Windows (LabKey Server is supported on both Operating Systems. Select the operating system which is best supported by your organization)
• DB software (PostgreSQL or MS SQL Server)
• See Supported Technologies for the specific versions to use.
Network1 GB/s

Related Topics




Premium Resource: Reference Architecture / System Requirements





Installation Checklists


These checklists explain how to install LabKey Server along with its prerequisite components. LabKey Server is a Java web application that runs under Apache Tomcat and accesses a relational database, either PostgreSQL or Microsoft SQL Server. LabKey Server can also reserve a network file share for the data pipeline, and use an outgoing (SMTP) mail server for sending system emails. LabKey Server may optionally connect to an LDAP server to authenticate users within an organization.

If you are upgrading a prior installation of LabKey, follow the steps in this topic:

The following are the main prerequisites for installing LabKey:

  • Java - Specifically the OpenJDK.
  • Tomcat - Provides the web server framework.
  • PostgreSQL or Microsoft SQL Server - Select one of these to serve as the database.
If you wish to evaluate LabKey Server, we recommend that you consider a hosted trial server instead of installing it locally.

Installation Checklist: Linux

To install on a Linux machine, complete the topics below:

  1. Install on Linux: Main Components - Assemble the main components.
  2. Installation: Tomcat Configuration - Configure the Tomcat web server.
  3. Installation: SMTP, Encryption, LDAP, and File Roots - Extend and optimize your installation.
  4. Installation: Third-Party Components - Add functionality via 3rd party applications.

Installation Checklist: Windows

To install on a Windows machine, complete the topics below:

  1. Install on Windows: Main Components - Assemble the main components
  2. Installation: Tomcat Configuration - Configure the Tomcat web server.
  3. Installation: SMTP, Encryption, LDAP, and File Roots - Extend and optimize your installation.
  4. Installation: Third-Party Components - Add functionality via 3rd party applications.

Troubleshooting Topics




Install on Linux: Main Components


This topic applies when installing LabKey Server 20.4 and later.

For installing LabKey Server 20.3, see the topic at:

https://www.labkey.org/Documentation/Archive/20.3/wiki-page.view?name=installComponents

This topic explains how to install LabKey on Linux systems, along with its prerequisites: Java, Apache Tomcat, and a database.

This topic assumes installation onto a new, clean Linux machine. If you are installing into an established environment, you can skip steps, or adjust them, as required to use any prerequisites already in place. We also assume that you have super user access to the machine, and that you are familiar with Linux commands and utilities. The example Linux commands below are for Ubuntu 18.04. Adapt these commands as appropriate for your Linux implementation.

Installation Steps:

Determine Supported Versions

  • Consult the topic Supported Technologies to identify supported and compatible versions of the prerequisite components. Make a list of the latest version numbers (for Java, Tomcat, and a database) that are supported by the version of LabKey Server you wish to install.

Create Folder Structure

  • For simplicity, we recommend using the directory structure described here, particularly whenever you are creating a new LabKey installation from scratch.
  • Use the following commands to create the directory structure for the installation:
    sudo mkdir -p /usr/local/labkey/apps
    sudo mkdir -p /usr/local/labkey/backups
    sudo mkdir -p /usr/local/labkey/labkey
    sudo mkdir -p /usr/local/labkey/labkey/externalModules
    sudo mkdir -p /usr/local/labkey/src
    sudo mkdir -p /usr/local/labkey/tomcat-tmp
  • These directories have the following purposes:
    • /labkey/apps - This directory holds the deployed third party components: Tomcat, PostgreSQL, OpenJDK.
    • /labkey/backups - This directory stores database and file backups as needed during upgrade.
    • /labkey/labkey - This directory is holds the deployed LabKey Server installation, known as LABKEY_HOME.
    • /labkey/labkey/externalModules - Holds modules not included in the server distribution packages.
    • /labkey/src - This directory and its subfolders stores the downloaded components before deployment.
    • /labkey/tomcat-tmp - A log directory for Tomcat.

Install Java

  • Go to OpenJDK (Opens in a new tab.) and get the download link to the JDK binary distribution, a .tar.gz file.
  • Download into the directory /usr/local/labkey/src
  • For example, use the command shown below, where <LINK> is the download link for the JDK binary distribution.
    /usr/local/labkey/src$ sudo wget <LINK>
  • Unzip the .tar.gz file into the directory /usr/local/labkey/apps/ for example:
    sudo tar -xvzf openjdk-##.#.#_linux-x64_bin.tar.gz -C /usr/local/labkey/apps/
  • Creating a symbolic link from /usr/local/java to /usr/local/labkey/apps/jdk-##.#.#. This will make Java easier to update in the future.
    sudo ln -s /usr/local/labkey/apps/jdk-##.#.# /usr/local/java
  • Define JAVA_HOME and add it to the PATH as follows:
  • Edit the file /etc/profile:
    sudo nano /etc/profile
  • Add the following lines to the end of the file:
    export JAVA_HOME="/usr/local/java"
    export PATH=$PATH:$JAVA_HOME/bin
  • Save: Ctrl+O and press the Enter key.
  • Exit: Ctrl+X
  • Apply changes to your current shell:
    source /etc/profile

Install the Apache Tomcat Web Server

  • Identify a compatible version of Apache Tomcat at http://tomcat.apache.org/ (Opens in a new tab.)
  • Download the tar.gz binary distribution to /usr/local/labkey/src, for example:
  • Unzip into the /apps directory, for example:
    sudo tar -xvzf apache-tomcat-#.#.##.tar.gz -C /usr/local/labkey/apps
  • Create a symbolic link from /usr/local/tomcat to /usr/local/labkey/apps/apache-tomcat-#.#.##
    sudo ln -s /usr/local/labkey/apps/apache-tomcat-#.#.## /usr/local/tomcat
  • The directory /usr/local/labkey/apps/apache-tomcat-#.#.## is called <CATALINA_HOME>.
  • Confirm the Java and Tomcat installations by runing the Tomcat startup script:
    sudo /usr/local/labkey/apps/apache-tomcat-#.#.##/bin/startup.sh
  • Enter the following URL in a web browser:
  • For a local test machine, go to:
  • If your Java and Tomcat installations are successful you will see the Tomcat success page.
  • Stop Tomcat before proceeding:
    sudo /usr/local/labkey/apps/apache-tomcat-#.#.##/bin/shutdown.sh

Install the Database Server (PostgreSQL)

  • Below we describe how to install PostgreSQL as the database server. If you wish to use MS SQL Server, complete the topic: Install Microsoft SQL Server
  • Use your native Linux install utility to install PostgreSQL (like apt or yum) or go to http://www.postgresql.org/download/ and download the PostgreSQL binary packages or source code. Follow the instructions in the downloaded package to install PostgreSQL.
  • Create the database and associated user/owner. Using superuser permissions, do the following:
    • Create an empty database named 'labkey'.
    • Create a PostgreSQL user named 'labkey'.
    • Grant the owner role to the labkey user over the database.
    • Revoke public permissions from the database.
  • See the following example PostgreSQL commands:
  • Connect to the DB server as the Postgres Super User using the psql command:
    sudo -u postgres psql
  • Issue the following commands to create the user, database and revoke public permissions from the database. Use the example below, after substituting your chosen PASSWORD. Retain the single quotes around the password value.
    create user labkey password 'PASSWORD';
    create database labkey with owner labkey;
    revoke all on database labkey from public;
    \q
  • The username, password, and db name you choose above will be used to configure Tomcat below.

Download LabKey Server System Components

  • Click Here to Register and Download LabKey Binaries. (Opens in a new tab.)
  • Get the download link to the current binary tar.gz distribution by right-clicking the button Download tar.gz and selecting Copy link address.
  • Download into the /usr/local/labkey/src directory:
    /usr/local/labkey/src$ sudo wget <LINK>
  • Unzip into the same directory:
    sudo tar xfz LabKey#.#.#-#####.#-community-bin.tar.gz
  • After unpacking, the directory contains these files and directories:
    • labkeywebapp - The LabKey Server web application.
    • modules - LabKey Server modules.
    • pipeline-lib- Jars for the data processing pipeline.
    • tomcat-lib - Required server library jars.
    • labkey.xml - LabKey Server configuration file.
    • manual-upgrade.sh - Manual upgrade script.
    • README.txt - A file pointing you to this documentation.
    • VERSION - A file containing the release number and build number.
  • For convenience, export the following:
    export LABKEY_DIST=/usr/local/labkey/src/LabKey##.#.#-#####.#-community-bin
    export LABKEY_HOME=/usr/local/labkey/labkey
    export CATALINA_HOME=/usr/local/labkey/apps/apache-tomcat-#.#.##
  • Confirm the paths are correct:
    echo $LABKEY_DIST
    echo $LABKEY_HOME
    echo $CATALINA_HOME

Deploy Tomcat Libraries

  • Ensure that Tomcat is stopped by running the shutdown script:
    sudo sh $CATALINA_HOME/bin/shutdown.sh
  • Copy all the JAR files from the /tomcat-lib directory to the <CATALINA_HOME>/lib directory, namely these six files:
    • javax.activation.jar
    • labkeyBootstrap.jar
    • mail.jar
  • For example:
    sudo cp $LABKEY_DIST/tomcat-lib/javax.activation.jar $CATALINA_HOME/lib
    sudo cp $LABKEY_DIST/tomcat-lib/labkeyBootstrap.jar $CATALINA_HOME/lib
    sudo cp $LABKEY_DIST/tomcat-lib/mail.jar $CATALINA_HOME/lib

Deploy the Main LabKey Binaries

  • Copy the following directories to /usr/local/labkey/labkey (LABKEY_HOME).
    • labkeywebapp
    • modules
    • pipeline-lib
  • For example:
    sudo cp -R $LABKEY_DIST/labkeywebapp $LABKEY_HOME
    sudo cp -R $LABKEY_DIST/modules $LABKEY_HOME
    sudo cp -R $LABKEY_DIST/pipeline-lib $LABKEY_HOME

Deploy the LabKey Server Configuration File

Find the file labkey.xml in the LabKey Server distribution files. This is the main LabKey Server configuration file and contains a number of settings required by LabKey Server to run.

  • Open labkey.xml in a text editor.
    sudo nano $LABKEY_DIST/labkey.xml
  • The parameter values you need to change are bracketed by @@...@@:
    • @@appDocBase@@ - Replace with <LABKEY_HOME>/labkeywebapp. Use the full path: /usr/local/labkey/labkey/labkeywebapp
    • @@jdbcUser@@ - Replace with the database user created above: 'labkey'.
    • @@jdbcPassword@@ - Replace with the password create above.
  • Create the intermediate "Catalina" and "localhost" subdirectories to make the following path:
    sudo mkdir -p $CATALINA_HOME/conf/Catalina/localhost
  • Copy labkey.xml into the new directory path:
    sudo cp $LABKEY_DIST/labkey.xml $CATALINA_HOME/conf/Catalina/localhost

Configure the Tomcat User and Service

To ensure that the server is not running as root, we configure Tomcat and LabKey to run under a new user named 'tomcat'.

We also configure Tomcat to start automatically using a .service unit file, so that the operating system manages the server's availability.

The instructions and .service file below are written for Ubuntu 18.04, but can be adapted to other environments.

  • Create the tomcat user (and associated group):
    sudo useradd -r -m -U -s /bin/false tomcat
  • Give this user ownership over CATALINA_HOME, LABKEY_HOME, and the logging directory. For example:
    sudo chown -R tomcat: $CATALINA_HOME
    sudo chown -R tomcat: $LABKEY_HOME
    sudo chown -R tomcat: /usr/local/labkey/tomcat-tmp
  • Create the tomcat.service file:
    sudo nano /etc/systemd/system/tomcat.service
  • Paste in the file contents. Start with the template below. Substitute any paths, version numbers, and other parameters to fit your environment.
    # Systemd unit file for tomcat_lk

    [Unit]
    Description=lk Apache Tomcat Application
    After=syslog.target network.target

    [Service]
    Type=forking
    Environment="JAVA_HOME=/usr/local/labkey/apps/jdk-13.0.2"
    Environment="CATALINA_BASE=/usr/local/labkey/apps/apache-tomcat-9.0.30"
    Environment="CATALINA_OPTS=-Djava.library.path=/usr/local/labkey/apps/jdk-13.0.2 -Djava.awt.headless=true -Duser.timezone=America/Los_Angeles -Xms512M -Xmx4096M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/usr/local/labkey/tomcat-tmp -Djava.net.preferIPv4Stack=true"
    Environment="CATALINA_TMPDIR=/usr/local/labkey/tomcat-tmp"

    ExecStart=/usr/local/labkey/apps/apache-tomcat-9.0.30/bin/catalina.sh start
    ExecStop=/usr/local/labkey/apps/apache-tomcat-9.0.30/bin/catalina.sh stop
    SuccessExitStatus=0 143
    Restart=on-failure
    RestartSec=2

    User=tomcat
    Group=tomcat

    [Install]
    WantedBy=multi-user.target
  • Save and close the file.
  • Notify the system of the new unit file:
    sudo systemctl daemon-reload
  • Enable automatic startup on system reboot:
    sudo systemctl enable tomcat

Start the Server

Installation Checklist

  1. (Current Topic) Install on Linux: Main Components
  2. Installation: Tomcat Configuration
  3. Installation: SMTP, Encryption, LDAP, and File Roots
  4. Installation: Third-Party Components



Install on Windows: Main Components


This topic is under construction for the 20.7 release of LabKey Server.

This topic explains how to install LabKey on a Windows machine, along with its prerequisites: Java, Apache Tomcat, and a database.

This topic assumes installation onto a clean Windows machine which contains none of the prerequisite components. If you are installing into an established environment, you can skip steps, or adjust them, to use any prerequisites already in place. We also assume that you have super-user/administrator access to the machine.

Directory Set Up

  • Setup the directory structure as below. This is the structure that we use internally for our LabKey servers:
C:\labkey\apps - This is where the prerequisite apps live.
C:\labkey\apps\tomcat
C:\labkey\apps\java
C:\labkey\apps\lib - This holds any special scripts or files, like SSL certificates.
C:\labkey\labkey - This is where Labkey gets installed to.
C:\labkey\src\labkey - This is where the downloaded distribution files go before installing.
  • In a command line shell, use:
cd C:\
mkdir labkey\apps\tomcat
mkdir labkey\apps\java
mkdir labkey\apps\lib
mkdir labkey\labkey
mkdir labkey\src

Download Components

Java

  • Unzip and deploy as follows:
labkey
apps
java
jdk-##.#.#
  • Set the JAVA_HOME environmental variable and add it to the PATH as follows:
  • Press the Start key, type "env", and click Edit the system environment variables.
  • In the System Properties dialog, on the Advanced tab, click Environment Variables.
  • Under System Variables click New....
  • For Variable name, enter "JAVA_HOME"
  • For Variable value, enter the full path to your jdk, for example, "C:\labkey\apps\jdk-##.#.#"
  • Click OK to confirm your changes.
  • Locate the System Variable named Path.
  • Select it, and click Edit...
  • Add the following entry: %JAVA_HOME%\bin
  • Click OK three times to confirm the stacked dialog boxes.

Tomcat

  • Unzip and deploy as follows:
labkey
apps
tomcat
apache-tomcat-#.#.##
  • Set the environment variable CATALINA_HOME to C:\labkey\apps\tomcat-#.#.##
  • Install Tomcat as a service by running the script service.bat:
C:\labkey\apps\tomcat-#.#.##\bin\service.bat install
  • Open the Services panel. Right-click Apache Tomcat 9.0 Tomcat9 and select Properties.
  • In the popup dialog, click the tab Log On. If necessary, select Local System account. Click OK.
  • On the Services panel, start Tomcat.
  • Confirm the Tomcat install, by visting the URL:
  • On the Services panel, stop Tomcat before continuing.

PostgreSQL

This topic explains how to install PostgreSQL as part of an installation of LabKey Server.

If you already have PostgreSQL installed, LabKey Server can use that installed instance, provided the version is supported.

To install PostgreSQL on Windows:

  • Download and run the Windows PostgreSQL one click installer. When the wizard prompts you to choose where to install PostgreSQL, point it to the apps subdirectory, i.e. C:\labkey\apps\
  • Keep track of the PostgreSQL Windows Service account name and password. LabKey Server needs to ask for it so that we can pass it along to the PostgreSQL installer.
  • Keep track of the database superuser name and password. You'll need these to initially create the LabKey database, the LabKey database user, and grant that user the owner role.
  • We recommend that you install the graphical tool pgAdmin 4.x for easy database administration. Leave the default settings as they are on the "Installation Options" page to include pgAdmin.
  • Create the database. Using superuser permissions, do the following:
    • Create an empty database.
    • Create a PostgreSQL user named 'labkey'.
    • Grant the owner role to the labkey user over the database.
    • Revoke public permissions from the database.
See the following example PostgreSQL commands:
  • Connect to the DB server as the Postgres Super User using the psql command:
sudo -u postgres psql
  • Issue the following commands to create the user, database and revoke public permissions from the database. Use the example below, after substituting your chosen values for LABKEY_USERNAME and PASSWORD_HERE. Retain the single quotes around the password value.
create user labkey password 'PASSWORD_HERE' ;
create database labkey with owner labkey;
revoke all on database labkey from public;
\q
  • The username and password you choose above will be used in the main configuration file below.

LabKey Server

  • Unzip LabKey####-####-community-bin.zip
  • The zip file contains:
bin                Windows-specific binary files required by LabKey Server.
labkeywebapp The LabKey Server web application.
modules LabKey Server modules.
pipeline-lib Jars for the data processing pipeline.
tomcat-lib Server library jars.
labkey.xml LabKey Server configuration file.
manual-upgrade.sh For use with existing installations.
README.txt A file pointing you to this documentation.
VERSION A file containing the release number and build number.

Main LabKey Binaries and Libraries

  • Copy the following directories:
bin
labkeywebapp
modules
pipeline-lib
  • into LABKEY_HOME:
C:\labkey\labkey

Tomcat Libraries

  • Go to the tomcat-lib dir in the distribution:
\LabKey####-####-community-bin\tomcat-lib
  • Copy the contents of tomcat-lib...
javax.activation.jar
labkeyBootstrap.jar
mail.jar
  • ...into CATALINA_HOME\lib:
C:\labkey\apps\apache-tomcat-#.#.##\lib

Main Configuration File: labkey.xml

  • Create the directory path
C:\labkey\apps\apache-tomcat-#.#.##\conf\Catalina\localhost
  • Copy the file labkey.xml. Find it in the unzipped distribution, and copy this file into the dir path just created, as follows:
C:\labkey\apps\apache-tomcat-#.#.##\conf\Catalina\localhost\labkey.xml

docbase, user, and password

  • In labkey.xml, substitute the values for @@...@@ tokens as below:
    • @@appDocBase@@ - replace with the full path: C:\labkey\labkey\labkeywebapp
    • @@jdbcUser@@ - the same username specified when creating the PosgreSQL db above. In this example 'labkey'.
    • @@jdbcPassword@@ - the same password specified when creating the PosgreSQL db above.

Start LabKey Server

  • On the Services panel, restart Apache Tomcat
  • If you are installing on a local machine, visit the LabKey home page at:

or, if you are installing on a remote server:

  • For a new server, you are invited to create the initial user profile, which is a Site Administrator.

Installation Checklist

  1. (Current Topic) Install on Windows: Main Components
  2. Installation: Tomcat Configuration
  3. Installation: SMTP, Encryption, LDAP, and File Roots
  4. Installation: Third-Party Components



Installation: Tomcat Configuration


This topic assumes you have completed the installation steps in either Install on Linux: Main Components or Install on Windows: Main Components, which explain how to install the core LabKey Server components.

The following tasks continue the installation process by optimizing the performance and behavior of your server. None of the tasks below are strictly required, but some are recommended, especially for production servers.

Increase Webapp Memory (Recommended)

Administrators will see a banner alerting them if the memory allocation for the server is too low.

To increase the memory allocation, follow these instructions: Configure Tomcat Webapp Memory.

Configure the Tomcat Default Port Number (Recommended)

Tomcat uses port 8080 by default, and to load any page served by Tomcat, you must either specify the port number as shown above, or you must configure the Tomcat installation to use a different port number. To configure the Tomcat HTTP connector port, edit the server.xml file at:

<CATALINA_HOME>/conf/server.xml

Find the entry that begins with <Connector port="8080" .../> and change the value of the port attribute to the desired number. In most cases, you'll want to change this value to "80", which is the default port number used by web browsers. If you change this value to "80", users will not need to include the port number in the URL to access LabKey Server.

Existing Installations of Tomcat

You can run two web servers on the same machine only if they use different port numbers, so if you have a web server running you may need to reconfigure one to avoid conflicts.

If you have an existing installation of Tomcat, you can configure LabKey Server to run on that installation. Alternately, you can install a separate instance of Tomcat for LabKey Server; in that case, you will need to configure each instance of Tomcat to use a different port. If you have another web server running on your computer that uses Tomcat's default port of 8080, you will also need to configure Tomcat to use a different port.

If you receive a JVM_BIND error when you attempt to start Tomcat, it means that the port Tomcat is trying to use is in use by another application. The other application could be another instance of Tomcat, another web server, or some other application. You'll need to configure one of the conflicting applications to use a different port. Note that you may need to reconfigure more than one port setting. For example, in addition to the default HTTP port defined on port 8080, Tomcat also defines a shutdown port at 8005. If you are running more than one instance of Tomcat, you'll need to change the value of the shutdown port for one of them as well.

Configure LabKey Server to Run Under SSL/TLS (Recommended)

You can configure LabKey Server to run under TLS (Transport Layer Security), or it's predecessor, SSL (Secure Socket Layer). We recommend that you take this step if you are setting up a production server to run over a network or over the Internet, so that your passwords and data are not passed over the network in clear text.

To configure Tomcat to run LabKey Server under SSL/TLS:

To require that users connect to LabKey Server using a secure (https) connection:
  • Select (Admin) > Site > Admin Console.
  • Under Configuration, click Site Settings.
  • Check Require SSL connections.
  • Enter the SSL/TLS port number that you configured in the previous step in the SSL Port field.
An example server.xml file is available at: https://github.com/LabKey/samples/blob/master/ops/config-examples/server-SSL.xml

Configure Tomcat Session Timeout (Optional)

Tomcat's session timeout specifies how long a user remains logged in after their last session activity, 30 minutes by default. To increase session timeout, edit <CATALINA_HOME>/conf/web.xml . Locate the <session-timeout> tag and set the value to the desired number of minutes.

Configure Tomcat to Use Gzip (Optional)

You may be able to improve the responsiveness of your server by configuring Tomcat to use gzip compression when it streams data back to the browser.

You can enable gzip in <CATALINA_HOME>/conf/server.xml by adding a few extra attributes to the active <Connector> elements:

compression="on" compressionMinSize="2048"
noCompressionUserAgents="gozilla, traviata"
compressableMimeType="text/html, text/xml, text/javascript, text/plain, text/css, application/json"

Note that there is a comment in the default file that provides basic instructions for enabling gzip. The snippet above improves on the comment's recommendation by enabling compression on a few extra MIME types.

Installation Checklist Topics

  1. Install on Linux: Main Components OR Install on Windows: Main Components
  2. (Current Topic) Installation: Tomcat Configuration
  3. Installation: SMTP, Encryption, LDAP, and File Roots
  4. Installation: Third-Party Components



Installation: SMTP, Encryption, LDAP, and File Roots


The LabKey Server configuration file, named "labkey.xml" by default, contains settings required for LabKey Server to run on Tomcat. This topic describes enhancements and extensions of the server that involve modifications to labkey.xml.

Configuration File Name

The name of the configuration file, in our case "labkey.xml", controls the context path in the URL:

To use a simplified URL without "labkey" appended as a context path, rename the file labkey.xml to "ROOT.xml". You can then access the LabKey Server application with the address:

Securing the LabKey Configuration File

The LabKey configuration file contains user name and password information for your database server, mail server, and network share. For this reason you should secure this file within the file system, so that only designated network administrators can view or change this file.

Modifying Configuration File Settings

You can edit the configuration file with your favorite text or XML editor. You will need to modify the LabKey Server configuration file if you are manually installing or upgrading LabKey Server, or if you want to change any of the following settings.

  • The appdocbase attribute, which indicates the location of the web application in the file system
  • Database settings, including server type, server location, username, and password for the database superuser.
  • SMTP settings, for specifying the mail server LabKey Server should use to send email to users.
  • Encryption Key - Configure an encryption key for the encrypted property set.
Note that in our template labkey.xml file, the placeholder values for these settings are highlighted with "@@", i.e "@@jdbcPassword@@", to make them easier to find in the file. Replace the full placeholder with the correct password along with the bracketing "@@". The final result should resemble:

<?xml version='1.0' encoding='utf-8'?>
<Context docBase="C:labkeylabkeylabkeyWebapp" reloadable="true" crossContext="true">
<Resource name="jdbc/labkeyDataSource" auth="Container"
type="javax.sql.DataSource"
username="labkey"
password="!2zYVgEZWuBXb!DeE"
...

The appDocBase Attribute

The appDocBase attribute of the Context tag must be set to point to the directory where you have extracted or copied the labkeywebapp directory. For example, if the directory where you've copied labkeywebapp is /usr/local/labkey/labkey, you would change the initial value to "/usr/local/labkey/labkey/labkeywebapp".

Database Settings

The username and password attributes must be set to a user name and password with admin rights on your database server. Both the name and password attribute are found in the Resource tag named "jdbc/labkeyDataSource". If you are running a local version of PostgreSQL as your database server, you don't need to make any other changes to the database settings in labkey.xml, since PostgreSQL is the default database choice.

The following is a template resource tag for PostgreSQL:

<Resource name="jdbc/labkeyDataSource" auth="Container"
type="javax.sql.DataSource"
username="postgres"
password="sa"
driverClassName="org.postgresql.Driver"
url="jdbc:postgresql://localhost:5433/labkey162"
maxTotal="20"
maxIdle="10"
maxWaitMillis="120000"
accessToUnderlyingConnectionAllowed="true"
validationQuery="SELECT 1"
/>

If you are running LabKey Server against Microsoft SQL Server, you should comment out the Resource tag that specifies the PostgreSQL configuration, and add a Resource tag for the Microsoft SQL Server configuration. A template Resource tag for MS SQL Server is available in this topic: Install Microsoft SQL Server.

If you are running LabKey Server against a remote installation of a database server, you will also need to change the url attribute to point to the remote server; by default it refers to localhost.

The maxWaitMillis parameter is provided to prevent server deadlocks. Waiting threads will time out when no connections are available rather than hang the server indefinitely.

GUID Settings

By default, LabKey Servers periodically communicate back to LabKey developers whenever the server has experienced an exception. LabKey rolls up this data and groups it by the GUID of each server. You can override the Server GUID stored in the database with the one specified in labkey.xml. This ensures that the exception reports sent to LabKey Server developers are accurately attributed to the server (staging vs. production) that produced the errors, allowing swift delivery of fixes. For details, see Tips for Configuring a Staging Server.

SMTP Settings

LabKey Server uses an SMTP mail server to send messages from the system, including email to new users when they are given accounts on LabKey. Configuring LabKey Server to connect to the SMTP server is optional; if you don't provide a valid SMTP server, LabKey Server will function normally, except it will not be able to send mail to users.

At installation, you will be prompted to specify an SMTP host, port number, user name, and password, and an address from which automated emails are sent. Note that if you are running Windows and you don't have an SMTP server available, you can set one up on your local computer.

The SMTP settings are found in the Resource tag named "mail/Session".

  • mail.smtp.host Set to the name of your organization's SMTP mail server.
  • mail.smtp.user Specifies the user account to use to log onto the SMTP server.
  • mail.smtp.port Set to the SMTP port reserved by your mail server; the standard mail port is 25. SMTP servers accepting a secure connection may use port 465 instead.
SMTP Authentication and Secure Connections: Many LabKey installations run an SMTP server on the same machine as the LabKey web server, which is configured for anonymous access from the local host only. Since only local applications can send mail, this ensures some amount of security without the hassle of using a central, authenticated mail server. If you choose instead to use an external authenticated server, you'll need to add the following:
  • mail.smtp.from This is the full email-address that you are would like to send the mail from. It can be the same as mail.smtp.user, but it doesn't need to be.
  • mail.smtp.password This is the password.
  • mail.smtp.starttls.enable When set to "true", configures the connection to use Transport Level Security (TLS).
  • mail.smtp.socketFactory.class When set to "javax.net.ssl.SSLSocketFactory", configures the connection to use an implementation that supports SSL.
  • mail.smtp.auth= When set to "true", forces the connection to attempt to authenticate using the user/password credentials.
When LabKey Server sends administrative emails, as when new users are added or a user's password is reset, the email is sent with the address of the logged-in user who made the administrative change in the From header. The system also sends emails from the Issue Tracker and Announcements modules, and these you can configure using the mail.from attribute so that the sender is an aliased address. The mail.from attribute should be set to the email address from which you want these emails to appear to the user; this value does not need to correspond to an existing user account. For example, you could set this value to "labkey@mylab.org".

Notes and Alternatives

  • If you do not configure an SMTP server for LabKey Server to use to send system emails, you can still add users to the site, but they won't receive an email from the system. You'll see an error indicating that the email could not be sent that includes a link to an HTML version of the email that the system attempted to send. You can copy and send this text to the user directly if you would like them to be able to log into the system.
  • If you are running on Windows and you don't have a mail server available, you can configure the SMTP service. This service is included with Internet Information Server to act as your local SMTP server. Follow these steps:
    • From the Start menu, navigate to Control Panel | Add or Remove Programs, and click the Add/Remove Windows Components button on the left toolbar.
    • Install Internet Information Services (IIS).
    • From Start | Programs | Administrative Tools, open the Windows Services utility, select World Wide Web Publishing (the name for the IIS service), display the properties for the service, stop the service if it is running, and set it to start manually.
    • From Start | Programs | Administrative Tools, open the Internet Information Services utility.
    • Navigate to the Default SMTP Virtual Server on the local computer and display its properties.
    • Navigate to the Access tab, click Relay, and add the address for the local machine (127.0.0.1) to the list of computers which may relay through the virtual server.

Troubleshooting SMTP

  • Begin by double checking that your labkey.xml file entries for the SMTP parameters (host, user, port, password) are correct. You can validate this by using them to log into the SMTP server directly.
  • If you are using gmail as your own email host, you may need to configure your gmail security settings to allow other applications to use it to access the server.
  • LabKey Server development source code includes a module named Dumbster to be used as a dummy SMTP server for testing purposes. See the testAutomation GitHub repository. It provides a Mail Record web part, which you can use to test outgoing email without sending actual email. If you are seeing unexpected SMTP configuration values, such as mail.smtp.port set to 53937 when you go to the Test Email Configuration link on the Admin Console, check the Admin Console's list of deployed modules for the Dumbster module. If it is present, you may need to disable the capture of email via the Mail Record web part to enable SMTP to send email. Learn more here.

Master Encryption Key

LabKey Server deployments can be configured to authenticate and connect to external systems to retrieve data or initiate analyses. In these cases, LabKey must store credentials (user names and passwords) in the primary LabKey database. While your database should be accessible only to authorized users, as an additional precaution, LabKey encrypts these credentials before storing them and decrypts them just before use. This encryption/decryption process uses a "master encryption key" that administrations set in labkey.xml; LabKey will refuse to save credentials if an encryption key is not configured.

Specify a master encryption key as follows:

<Parameter name="MasterEncryptionKey" value="@@masterEncryptionKey@@" />

Replace @@masterEncryptionKey@@ with a randomly generated, strong password, for example, a string of 32 random ASCII characters or 64 random hexadecimal digits. Once a key is specified and used, the key can't be changed, otherwise the existing values will be irretrievable. Different LabKey Server deployments should use different encryption keys, however, servers that use copies of the same database (for example, most test, staging, and production server combinations) need to use the same encryption key.

LDAP Synchronization (Premium Feature)

If you wish to synchronize with the user and groups on an LDAP server, add an appropriate <Resource> tag to the labkey.xml file.

An example configuration to a public LDAP test service:

<Resource name="ldap/ConfigFactory" auth="Container"
type="org.labkey.premium.ldap.LdapConnectionConfigFactory"
factory="org.labkey.premium.ldap.LdapConnectionConfigFactory"
host="ldap.forumsys.com"
port="389"
principal="cn=read-only-admin,dc=example,dc=com"
credentials="password"
/>

Since this is an XML file, you will need to follow XML escaping for special characters such as using "&lt;" for less than.

The following optional properties are also supported:

  • useTls: true/false, defaults to false.
  • useSsl: true/false, defaults to false. You may only opt into TLS or SSL, not both.
  • sslProtocol: SSL, SSLv2, or SSLv3 to restrict which versions of the SSL protocol is to be used.
Learn more about LDAP synchronization in this topic: LDAP User/Group Synchronization

Set File Roots

If you want to update the server so the files directory is pointing to a SAN, NFS, or another mounted drive, see the following topics:

Installation Checklist

  1. Install on Linux: Main Components OR Install on Windows: Main Components
  2. Installation: Tomcat Configuration
  3. (Current Topic) Installation: SMTP, Encryption, LDAP, and File Roots
  4. Installation: Third-Party Components



Installation: Third-Party Components


This topic covers third-party components which are often used in conjunction with LabKey Server to enhance its functionality. All of these components are optional and are not required for the server to work in general, but some are required for specific use cases.

R

Graphviz (All)

Trans Proteomic Pipeline (MS2)

  • Information on TPP: http://tools.proteomecenter.org/wiki/index.php?title=Software:TPP
  • Install Instructions:
    • Download the source distribution of v4.6.3 of the tools and unzip or obtain directly from Subversion at https://svn.code.sf.net/p/sashimi/code/tags/release_4-6-3.
    • Build the tools
      • Edit trans_proteomic_pipeline/src/Makefile.incl
      • Add a line with XML_ONLY=1
      • Modify TPP_ROOT to point to the location you intend to install the binaries. NOTE: This location must be on your server path (ie the path of the user running the Tomcat server process).
    • Run 'make configure all install' from trans_proteomic_pipeline/src
    • Copy the binaries to the directory you specified in TPP_ROOT above.
  • License: LGPL
  • Notes:
    • If you are using XPRESS for quantitation as part of your analysis, you will also need to have Perl installed and available on your path.

X!Tandem (MS2)

Proteowizard (MS2)

Installation Checklist

  1. Install on Linux: Main Components OR Install on Windows: Main Components
  2. Installation: Tomcat Configuration
  3. Installation: SMTP, Encryption, LDAP, and File Roots
  4. (Current Topic) Third-Party Components and Licenses



Troubleshoot Server Installation and Configuration


This topic is under construction for the July 2020 release of LabKey Server. For current documentation of this feature, click here.

In case of errors or other problems when installing and running LabKey Server, first review installation basics and options linked in the topic: Install LabKey. This topic provides additional troubleshooting suggestions if the instructions in that topic do not resolve the issue.

Installation Logs

From time to time Administrators may need to review the LabKey Tomcat logs to troubleshoot system startup issues. The LabKey Tomcat logs are located at:

/labkey/apps/tomcat/logs

The logs of interest are catalina.out and labkey.log. catalina.out contains the log file entries from Tomcat as it is starting the LabKey application. Once the LabKey application starts, logging is recorded in labkey.log.

Useful commands:

# monitor the catalina.out log file while LabKey/Tomcat starts
sudo tail -f /usr/local/labkey/apps/tomcat/logs/catalina.out

# monitor the labkey.log file while LabKey/Tomcat starts
sudo tail -f /usr/local/labkey/apps/tomcat/logs/labkey.log

Developer Mode

Running a server in "devmode" provides enhanced logging and enables the MiniProfiler, and has the server prevent the browser from caching resources like JavaScript and CSS files as aggresively. To run the server in devmode, follow these steps. The supported version of tomcat will change over time; replace the # in our examples with the actual version number.

  • Open a command prompt.
  • Go to the <CATALINA_HOME>/bin directory, for example:
C:\labkey\apps\apache\apache-tomcat-#.#.##\bin
  • Execute the tomcat#w.exe program:
tomcat#w.exe //ES//LabKeyTomcat#
  • The command will open a program window. Click the Java tab.
  • In the Java Option box, scroll to the bottom of the properties. Add the following property at the bottom of the list:
-Ddevmode=true
  • Close the program window and restart the server.
  • To confirm that the server is running in devmode:
    • Go to (Admin) > Site > Admin Console.
    • Under Diagnostics, click System Properties.
    • Locate the devmode property, and confirm that its value is 'true'.
Note that the "caching" JVM system property can also be used to control just the caching behavior, without all of the other devmode behaviors. To disallow the normal caching, perhaps because files are being updated directly on the file system, add -Dcaching=false to the JVM arguments.

Diagnostic Information

Which Version of LabKey Server Is Running?

  • Find your version number at (Admin) > Site > Admin Console.
  • At the top of the Server Information panel, you will see the release version.

Learn more about other diagnostic information on the admin console in this topic:

Common Issues

Conflicting Applications

Before installing LabKey Server, you should shut down all other running applications. If you have problems during the installation, try additionally shutting down any virus scanning application, internet security applications, or other applications that run in the background.

Compatible Component Versions

Confirm that you are using the supported versions of the required components, as detailed in the Supported Technologies Roadmap. It is possible to have multiple versions of some software, like Java, installed at the same time. Check that LabKey and other applications, such as Tomcat, are configured to use the correct versions.

PostgreSQL Installation

You may need to remove references to Cygwin from your Windows system path before installing LabKey, due to conflicts with the PostgreSQL installer. The PostgreSQL installer also conflicts with some antivirus or firewall software programs. (see http://wiki.postgresql.org/wiki/Running_%26_Installing_PostgreSQL_On_Native_Windows for more information).

Tomcat Failure to Start

If installation fails to start tomcat (such as with an error like "The specified service already exists."), you may need to manually stop or delete a failing Tomcat service.

To stop the service on Windows, open Control Panel > Administrative Tools > Services. Select the relevant service, such as LabKey Server Apache Tomcat #.0 and click Stop.

To delete, run the following from the command line as an administrator, substituting the major version number for the #:

sc delete LabKeyTomcat#

Connection Pool Size

If your server becomes unresponsive, it could be due to the depletion of available connections to the database. Watch for a Connection Pool Size of 8, which is the Tomcat connection pool default size and insufficient for a production server. To see the connection pool size for the LabKey data source, select (Admin) > Site > Admin Console and check the setting of Connection Pool Size on the Server Information panel. The connection pool size for every data source is also logged at server startup.

To set the connection pool size, edit your labkey.xml configuration file and change the "maxTotal" setting for your LabKey data source to at least 20. Depending on the number of simultaneous users and the complexity of their requests, your deployment may require a larger connection pool size.

You should also consider changing this setting for external data sources to match the usage you expect. Learn more in this topic: External Schemas and Data Sources.

Restart Installation from Scratch

If you have encountered prior failed installations, don't have any stored data you need to keep, and want to clean up and start completely from scratch, the following process may be useful:

  • Delete the Tomcat service (if it seems to be causing the failure).
  • Uninstall PostgresSQL using their uninstaller.
    • Control Panel > Programs and Features
    • Select PostgreSQL program.
    • Click Uninstall.
  • Delete the entire LabKey installation directory.
  • Install LabKey again.

Installation Forum

You can search for problems resolved through community support in the LabKey Installation Forum.

If you don't see your issue listed there, you can post a new question. If the install seems successful, it is often helpful to submit debugging logs for diagnosis.

If the install failed to complete, please include the install.log and install-debug.log from your selected LabKey install directory.

PostgreSQL logs its installation process separately. If PostgreSQL installation/upgrage fails, please locate and include the PostgreSQL install logs as well.

Related Topics




Installation Error Messages


If you have encountered errors or other problems when installing and starting LabKey Server, first review the topic Troubleshooting: Common Issues. If you're still encountering problems, please review the list below for common errors, messages, and problems.

You can also search the LabKey Community Support Forums for guidance. If you don't already see your issue listed there, please post a new question.


1.

Error

Error on startup, "Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections."

ProblemTomcat cannot connect to the database.
Likely causes
  • The database is not running
  • The database connection URL or user credentials in the Tomcat configuration files are wrong
  • Tomcat was started before the database finished starting up
SolutionMake sure that database is started and fully operational before starting Tomcat. Check the database connection URL, user name, and password in the <tomcat>/conf/Catalina/localhost/labkey.xml file.

2.

ProblemError when connecting to LabKey server on Linux: Can't connect to X11 window server or Could not initialize class ButtonServlet.
SolutionRun tomcat headless. Edit tomcat's catalina.sh file, and add the following line near the top of the file:
CATALINA_OPTS="-Djava.awt.headless=true"
Then restart tomcat.

3.

Problem

Viewing certain pages results in a specific NoSuchMethodError.

Error

java.lang.NoSuchMethodError: 
                org.apache.jasper.runtime.JspRuntimeLibrary.releaseTag(Ljavax/servlet/jsp/tagext/Tag;Lorg/apache/tomcat/InstanceManager;Z)V
       at org.labkey.jsp.compiled.org.labkey.core.admin.maintenance_jsp._jspx_meth_labkey_005ferrors_005f0(maintenance_jsp.java:159)
       at org.labkey.jsp.compiled.org.labkey.core.admin.maintenance_jsp._jspService(maintenance_jsp.java:110)
       at org.labkey.api.view.JspView.renderView(JspView.java:170)
       at org.labkey.api.view.WebPartView.renderInternal(WebPartView.java:372)

Solution

Upgrade your Apache Tomcat deployment to a supported version.


4.

Problem

After upgrading LabKey Server the following error is shown.

Error

java.lang.NoSuchMethodError: org.labkey.api.settings.HeaderProperties: method 'void <init>()' not found

Solution

This may be the result of a partial upgrade, where some but not all of the binaries get upgraded.
1. Shut down the Tomcat service
2. Delete your modules and labkeyWebapp directories
3. Recopy those directories form the new version you downloaded
4. Restart the Tomcat service


5.

Error

You receive a message "The requested resource () is not available." OR "500: Unexpected server error" and see something like one of the following in the log file:

Problem
SEVERE: Error deploying configuration descriptor labkey.xml 
java.lang.IllegalStateException: ContainerBase.addChild: start: LifecycleException: start: : java.lang.UnsupportedClassVersionError: org/labkey/bootstrap/LabkeyServerBootstrapClassLoader : Unsupported ...
A failure occurred during LabKey Server startup.
java.lang.NoClassDefFoundError: javax/script/ScriptEngineFactory....
A failure occurred during LabKey Server startup.
java.lang.UnsupportedClassVersionError: Bad version number in .class file
       at java.lang.ClassLoader.defineClass1(Native Method)
       at java.lang.ClassLoader.defineClass(ClassLoader.java:675) ...
Solution

You may need to use a newer version of the JDK. See Install Required Components.

Confirm that Tomcat is configured to use the correct version of Java, as it is possible to have multiple versions installed simultaneously.


6.

Problem

Fatal Error in Java Runtime Environment

Error
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x0000000000000000, pid=23893, tid=39779
#
# JRE version: Java(TM) SE Runtime Environment (8.0_45-b14) (build 1.8.0_45-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.45-b02 mixed mode bsd-amd64 compressed oops)
# Problematic frame:
# C 0x0000000000000000
#
# Failed to write core dump. Core dumps have been disabled.
# To enable core dumping, try "ulimit -c unlimited" before starting Java again
Cause

These are typically bugs in the Java Virtual Machine itself.

Solution

Ensuring that you are on the latest patched release for your preferred Java version is best practice for avoiding these errors. If you have multiple versions of Java installed, be sure that JAVA_HOME and other configuration is pointing at the correct location. If you are running through the debugger in IntelliJ, check the JDK configuration: under Project Structure > SDKs check the JDK home path and confirm it points to the newer version.

7.

Problem

If you see the following error when running complex queries on PostgreSQL:

Error
org.postgresql.util.PSQLException: ERROR: failed to build any 8-way joins
Solution

Increase the join collapse limit. Edit postgresql.conf and change the following line:

# join_collapse_limit = 8

to

join_collapse_limit = 10




Collect Debugging Information


This topic is under construction for the July 2020 release of LabKey Server. For current documentation of this feature, click here.

To assist in debugging errors, an administrator can force the LabKey Server process to dump the list of threads and memory to disk. This information can then be reviewed to determine the source of the error.

A thread dump is useful for diagnosing issues where LabKey Server is hung or some requests spin forever in the web browser.

A memory/heap dump is useful for diagnosing issues where LabKey Server is running out of memory, or in some cases where the process is sluggish and consuming a lot of CPU performing garbage collection.

Definitions

  • <LABKEY_HOME>: Installation location of the LabKey Server, for example:
    • Linux: /usr/local/labkey/labkey
    • Windows: C:\labkey\labkey\
  • <CATALINA_HOME>: Installation location of the Apache Tomcat Web Server (where x.x.xx is the specific version installed, for example:
    • Linux: /usr/local/labkey/apps/apache/apache-tomcat-x.x.xx
    • Windows: C:\labkey\apps\apache\apache-tomcat-x.x.xx

Thread Dump

To dump the list of running threads, you can use either the UI or the command line.

Using the UI

  • Go to (Admin) > Site > Admin Console.
  • Under Diagnostics, click Running Threads
  • This will show the state of all active threads in the browser, as well as writing the thread dump to the server log file.

Manually via the Command Line

The server monitors the timestamp of a threadDumpRequest file and when it notices a change, it initiates the thread dump process. The content of the file does not matter.

Unix. On Linux and OSX, use the command line to execute the following commands:

  • Change to the <LABKEY_HOME> directory. For example, if your <LABKEY_HOME> directory is located at /usr/local/labkey/labkey, then the command will be:
    cd /usr/local/labkey/labkey
  • Force the server to dump its memory:
    touch threadDumpRequest
Windows. On a Windows Server, do the following:
  • Open a Command Prompt.
  • Change to the <LABKEY_HOME> directory. For example, if your <LABKEY_HOME> directory is located at C:\labkey\labkey\, then the command will be:
    cd "C:\labkey\labkey"
  • Force the server to dump its memory. The command will open the heapDumpRequest file in the notepad program.
    notepad threadDumpRequest
  • Place the cursor at the top of the file and hit the "Enter" key twice.
    • Save the file.
    • Close notepad.

Location of the File Containing the Thread Dump

The list of threads is dumped the "labkey.log" file, which is located in the <CATALINA_HOME>/logs directory.

Memory Dump

To request LabKey Server dump its memory, you can use either the UI or the command line. By default, it will write the file to the LABKEY_HOME directory. Java can also be configured to dump the heap if the virtual machine runs out of memory.

Heap dumps can be large: several gigabytes in size is typical. You can tell the server to write to a different location by using a JVM startup argument:

-XX:HeapDumpPath=/path/to/desired/target

Using the UI

  • Go to (Admin) > Site > Admin Console.
  • Under Diagnostics, click Dump Heap.
  • You will see a full path to where the heap was dumped.

Manually via the Command Line

The server monitors the timestamp of a heapDumpRequest file and when it notices a change, it initiates the heap dump process. The content of the file does not matter.

Unix. On Linux and OSX, execute the following commands on the command line:

  • Navigate to the <LABKEY_HOME> directory.
  • Force the server to dump its memory:
    touch heapDumpRequest
Windows. On a Windows Server, do the following:
  • Open a Command Prompt
  • Navigate to the <LABKEY_HOME> directory.
  • Force the server to dump its memory. This command will open the heapDumpRequest file in the notepad:
    notepad heapDumpRequest
  • Place the cursor at the top of the file and hit the Enter key twice.
    • Save the file
    • Close notepad

Location of the File Containing the Memory Dump

The file will be located in the <LABKEY_HOME> directory. The file will have the ".hprof" extension.

Automatically Capturing Heap Dump when Server Runs Out of Memory

The Java virtual machine supports an option that will automatically trigger a heap dump when the server runs out of memory. This is accomplished via this JVM startup parameter:

-XX:+HeapDumpOnOutOfMemoryError

For more information, see Oracle's documentation.

Debugging PostgreSQL Errors

LabKey server is unable to log errors thrown by PostgreSQL, so in the case of diagnosing some installation and startup errors, it may be helpful to to view the event log.

On Windows:
  • Launch eventvwr.msc
  • Navigate to Windows Logs > Application.
  • Search for errors there corresponding to the installation failures, which may assist LabKey support in diagnosing the problem.
  • If you can't find relevant messages, you may be able to trigger the error to occur again by running net start LabKey_pgsql-9.2 from the command line.

In addition, you may find assistance debugging PostgreSQL on their troubleshooting installation page. Included there are instructions to "Collect the installer log file."

Collect SQL Server Information

To get a full picture of some problems, it's also useful to have information about running queries and locks. For this, you want both a thread dump (described above) and information about the state of database connections. The latter needs to be obtained directly from SQL Server.

  • Launch SQL Server Management Console or a similar tool
  • Open a connection to the LabKey Server database, often named "labkey."
  • Run and capture the output from the following queries/stored procedures:
    • sp_lock
    • sp_who2
    • SELECT t1.resource_type, t1.resource_database_id, t1.resource_associated_entity_id, t1.request_mode, t1.request_session_id, t2.blocking_session_id FROM sys.dm_tran_locks as t1 INNER JOIN sys.dm_os_waiting_tasks as t2 ON t1.lock_owner_address = t2.resource_address;

Related Topics




Common Install Tasks





Configure Tomcat Webapp Memory


This topic provides instructions for setting the Java Virtual Machine (JVM) memory configuration on Windows, Linux, and OSX operating systems. What do the JVM memory settings do? LabKey Server is Java web application that runs on Tomcat. The Tomcat server is run within a Java Virtual Machinex (JVM). This JVM that controls the amount of memory available to LabKey Server. LabKey recommends that the Tomcat web application be configured to have a maximum Java Heap size of at least 2GB for a test server, and at least 4GB for a production server.

Definitions

  • <CATALINA_HOME>: Installation location of the Apache Tomcat Web Server. If you are following our recommended folder configuration, the location will be (where #.#.## is the specific version installed):
    <LABKEY_ROOT>/apps/apache-tomcat-#.#.##
    • On Linux or OSX, you may also have created a symbolic link /usr/local/tomcat to this location.

Change the JVM Memory Configuration on Linux and OSX

Method 1: If Tomcat is running as a service

  • Locate the tomcat.service file, typically located at /etc/systemd/system/tomcat.service
  • Open the file and edit the CATALINA_OPTS parameter.
    • For a production server, use:
      -Xms4g -Xmx4g -XX:-HeapDumpOnOutOfMemoryError
    • For a test server, use:
      -Xms2g -Xmx2g -XX:-HeapDumpOnOutOfMemoryError

Method 2: If you use JSVC to start/stop LabKey Server

  • Find the JSVC service script.
    • On Linux servers, this is usually in the /etc/init.d directory and named either "tomcat" or "tomcat#"
    • On OSX servers this might be in /usr/local/jsvc/Tomcat#.sh
  • Open the JSVC service script using your favorite editor
  • Find the line for setting CATALINA_OPTS and add one of the following settings inside the double quotes.
    • For a production server, use:
      -Xms4g -Xmx4g -XX:-HeapDumpOnOutOfMemoryError
    • For a test server, use:
      -Xms2g -Xmx2g -XX:-HeapDumpOnOutOfMemoryError
  • Save the file
  • Restart LabKey Server

Method 3: If you use startup.sh and shutdown.sh to start/stop LabKey Server

The start script is located in <CATALINA_HOME>/bin/catalina.sh

  • Open the Start script using your favorite text editor
  • Above the line reading "# OS specific support. $var _must_ be set to either true or false.: add one of the following:
    • For a production server, use:
      JAVA_OPTS="$JAVA_OPTS -Xms4g -Xmx4g -XX:-HeapDumpOnOutOfMemoryError"
    • For a test server, use:
      JAVA_OPTS="$JAVA_OPTS -Xms2g -Xmx2g -XX:-HeapDumpOnOutOfMemoryError"
  • Save the file
  • Restart LabKey Server

Change the JVM Memory Configuration on Windows

Tomcat is usually started as a service on Windows, and it includes a dialog for configuring the JVM. The maximum total memory allocation is configured in its own text box, but other settings are configured in the general JVM options box using the Java command line parameter syntax.

If you changed the name of the LabKey Windows Service, you must use Method 2.

Method 1:

  • Open Windows Explorer.
  • Go to the <CATALINA_HOME>/bin directory.
  • Locate and run the file tomcat#w.exe (where # is the Tomcat version number). Run this as an administrator by right-clicking the .exe file and selecting Run as administrator.
  • The command will open a program window.
  • If this produces an error that says "The specified service does not exist on the server", then please go to Method 2.
  • Go to the Java tab in the new window.
  • In the Java Options box, scroll to the bottom of the properties, and set the following property:
    -XX:-HeapDumpOnOutOfMemoryError
  • Change the Initial memory pool to 2000 MB for a test server or 4000 MB for a production server.
  • Change the Maximum memory pool to the same value.
  • Click OK
  • Restart LabKey Server.

Method 2:

You will need to use this method if you customized the name of the LabKey Windows Service.
  • Open a Command Prompt. (Click Start, type "cmd", and press the Enter key.)
  • Navigate to the <CATALINA_HOME>bin directory.
  • Execute the following command, making appropriate changes if you are running a different version of Tomcat. This example is for Tomcat 9:
tomcat9w.exe //ES//LabKeyTomcat9
    • The command will open a program window.
    • If this produces an error that says "The specified service does not exist on the server", then see the note below.
  • Go to the Java tab in the new window.
  • In the Java Options box, scroll to the bottom of the properties, and set the following property:
    -XX:-HeapDumpOnOutOfMemoryError
  • Change the Initial memory pool to 2GB for a test server or 4GB for a production server.
  • Change the Maximum memory pool to the same value.
  • Click the OK button
  • Restart LabKey Server
NOTE: The text after the //ES// must exactly match the name of the Windows Service that is being used to start/stop your LabKey Server. You can determine the name of your Windows Service by taking the following actions:
  • Open the Windows Services panel. (Click Start, type "Services", and press the Enter key.)
  • In the Services panel, find the entry for LabKey Server. It might be called something like Apache Tomcat or LabKey
  • Double-click on the service to open the properties dialog.
In the command above, replace the text "LabKeyTomcat#" with the text shown next to ServiceName in the Properties dialog.

Related Topics

Back to Installation: Tomcat Configuration




Creating & Installing SSL/TLS Certificates on Tomcat


This topic is under construction for the July 2020 release of LabKey Server. For current documentation of this feature, click here.

This topic describes how to create and install an SSL/TLS certificate on a Tomcat server. First we cover the process for creating a self-signed certificate and then an actual signed certificate from a Certificate Authority (CA).

Tomcat uses a Java KeyStore (JKS) repository to hold all of the security certificates and their corresponding private keys. This requires the use of the keytool utility that comes with the Java Development Kit (JDK) or the Java Runtime Environment (JRE). We highly recommend using OpenJDK 11 (or later) for better security of self-signed certificates. Review this topic for current version recommendations: Supported Technologies

Two things to understand:
  • The alias is simply a "label" used by Java to identify a specific certificate in the keystore (a keystore can hold multiple certificates). It has nothing to do with the server name, or the domain name of the Tomcat service. A lot of examples show "tomcat" as the alias when creating the keystore, but it really doesn’t matter what you call it. Just remember when you do use it, you stick with it.
  • The common name (CN) is an attribute of the SSL/TLS certificate. Your browser will usually complain if the CN of the certificate and the domain in the URI do not match (but since you’re using a self-signed certificate, your browser will probably complain anyway). HOWEVER, when generating the certificate, the keytool will ask for "your first and last name" when asking for the CN, so keep that in mind. The rest of the attributes are not really that important.

Create a Self-Signed Certificate

Why create a self-signed certificate?

  • It allows you to learn to create a keystore and certificate, which is good practice for getting an actual SSL/TLS certificate provided by a Certificate Authority.
  • It allows you to use a certificate right away and make sure it works successfully.
  • It's free.
How to get started creating your self-signed certificate:

Step 1. Locate the keytool application within your OpenJDK installation. Confirm that your <JAVA_HOME> environment variable points to the current supported version of OpenJDK and not to another JDK or JRE you may have installed previously.

The keytool will be in the bin directory. For example (where ## is the specific version number):

/usr/local/labkey/apps/Java/openjdk-##.jdk/Contents/Home/bin/keytool

Step 2. Create the keystore and the certificate. When you use the following commands, be sure to change <your password> to a password of your choosing and add a <keystore location> and <certificate location> that you will remember, likely the same location.

Use the following syntax to build your keystore and your self-signed certificate. Some examples follow.

  • The path to the keytool
  • The -genkeypair flag to indicate you are creating a key pair
  • The -exportcert flag to generate the certificate
  • The -alias flag and the alias you want to use for the keystore
  • The -keyalg flag and the algorithm type you want to use for the keystore
  • The -keysize flag and the value of the certificate encryption size
  • The -validity flag and the number of days for which you want the certificate to be valid
  • The -keystore flag and the path to where you want your keystore located
  • The -file flag and the path where you want the certificate located
  • The -storepass flag and your password
  • The -keypass flag and your password
  • The -ext flag to generate the SAN entry that is required by some modern browsers
For example:
$JAVA_HOME/bin/keytool -genkeypair -alias my_selfsigned -keyalg RSA -keysize 4096 -validity 720 -keystore <keystore location>/my_selfsigned.p12 -storepass <your password> -keypass <your password> -ext SAN=dns:localhost,ip:127.0.0.1

$JAVA_HOME/bin/keytool -exportcert -alias my_selfsigned -file <certificate location>/my_selfsigned.cer -keystore <keystore location>/my_selfsigned.p12 -storepass <your password>

Step 3. The string will then create a series of prompts, asking for you to supply a password for the keystore and create what is known as a Certificate Server Request (CSR):

  • The keystore password and to confirm the password.
  • The domain you want your SSL/TLS certificate made for.
NOTE: This prompt will literally ask the question “What is your first and last name? [Unknown]:” This is NOT your first and last name. This should be the domain you want to make the SSL/TLS certificate for. Since this section is about creating a self-signed certificate, please enter localhost as the domain and press Enter.
  • The name of your Organizational Unit. This is optional, but most will put down the name of the department the certificate is being used for or the department the certificate is being requested by.
  • The name of your Organization. This is also optional, but most will put in the name of their company.
  • The name of your city. This is also optional, but most will put in the city the company or department is located in.
  • The name of your state/province. This is also optional, but if you do choose to enter in this information DO NOT abbreviate the information. If you choose to enter data here, you must spell out the state/province. For example: California is acceptable, but not CA.
  • The two-letter country code. This is optional, but if you’ve already entered the rest above, you should enter the two-letter code. For example: US for United States, UK for the United Kingdom.
  • Confirmation that the information you entered is correct.
  • A final prompt to enter the certificate, but just press Enter to use the same one as the Keystore from earlier.
The steps above will create the new keystore and add the new self-signed certificate to the keystore.

Step 4. Configure the server.xml file to use the new keystore & self-signed certificate.

Now that the keystore and certificate are ready, you now will need to configure the server.xml file to use them.

  • Access the server.xml located in the Tomcat directory, under the conf directory.
  • Activate the HTTPS-Connector listed in the server.xml file. It should look something like the example below. Note: Be sure to change the certificateKeyAlias, certificateKeystoreFile, and certificateKeystorePassword to the ones you used earlier (especially the password).
<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="200" SSLEnabled="true" scheme="https" secure="true"
>
<SSLHostConfig
sslProtocol="TLSv1.2" protocols="TLSv1.2"
ciphers="TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384:TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384:
TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384:TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384:
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256:TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256:
TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256:TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256:
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384:TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384:
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA:TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA:
TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384:TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384:
TLS_ECDH_RSA_WITH_AES_256_CBC_SHA:TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA:
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256:TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256:
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA:TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA:
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256:TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256:
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA:TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA"

honorCipherOrder="true"
>
<Certificate
certificateKeyAlias="my_selfsigned"
certificateKeystoreFile="$/labkey/apps/lib/my_selfsigned.p12"
certificateKeystorePassword="password_here"
certificateKeystoreType="PKCS12"
type="RSA"/>
</SSLHostConfig>
</Connector>

Step 5. (Optional) You can create a working certificate without this step, but for some uses (like running LabKey automated tests in HTTPS) you'll need to add the certificate manually to your OS and to Java so that they know to trust it. To add the certificate to the Trusted Root Certificate Authorities and Trusted Publishers, follow the instructions for your operating system below.

  • If you're trying to run LabKey's automated tests, change, in the test.properties file, labkey.port to "8443" and labkey.server to "https://localhost".
For OSX:
  • Find your my_selfsigned.cer file from step #2, go to Applications > Utilities > Keychain Access, and drag the file there.
  • Now we need to make this certificate trusted. Go to the Certificates line in the Category section in the lower-left corner of this window, find the "localhost" entry, and double-click it. Then expand the Trust section, and change "When using this certificate" to "Always Trust". Click the close button in the upper left and type your password to finish this operation.
  • Import the same my_selfsigned.cer file into your Java cacerts (which assumes you have not changed your Java's default keystore password of "changeit") by executing this command on the command line:
    $JAVA_HOME/bin/keytool -importcert -cacerts -alias my_selfsigned -file <certificate location>/my_selfsigned.cer -storepass changeit
For Windows:
  • Go to Windows command line and run the command, certmgr. This will open certificate manager for current user. Perform the following in certificate manager.
    • Right-click on Personal/Certificates. Select All Tasks > Import. This should open Certificate Import Wizard.
    • Current User should be selected. Click Next.
    • Enter the path to the certificate you created, <certificate location>/my_selfsigned.cer. Click Next.
    • Select "Place all certificates in the following store". Certificate store should be "Personal".
    • Review and click Finish.
    • You should now see your certificate under Personal/Certificates.
    • Right-click and Copy the certificate.
    • Paste the certificate into Trusted Root Certification Authorities/Certificates.
    • Paste the certificate into Trusted Publishers/Certificates.
    • Close certmgr.
    • Before running LabKey automated tests, edit the test.properties file: change labkey.port to "8443" and labkey.server to "https://localhost".
  • Execute the following to add your certificate to Java cacerts:
    "%JAVA_HOME%/bin/keytool" -importcert -cacerts -alias my_selfsigned -file <certificate location>/my_selfsigned.cer -storepass changeit
Windows may not pick up the certificate right away. A reboot should pick up the new certificate.

Step 6: Restart Tomcat and try to connect to your local server as https://localhost:8443/labkey and see if you are able to connect to it via HTTPS. If you did everything correctly, you should see a grey padlock to the left of the URL, and not red "https" with a line through it.

Configure LabKey Server to use HTTPS

Be sure to change the LabKey Server settings in your admin console to support HTTPS better. Go to (Admin) > Site > Admin Console > Settings > Configuration > Site Settings, and change the following values (which assume localhost as your server and port 8443):

In particular, you'll want to do that second item ("Require SSL connections"), because you will no longer be able to sign in to LabKey properly via HTTP.

Create a Real Certificate

Once you’ve successfully created your own self-signed certificate, the steps to requesting and adding an actual certificate will be significantly easier.

To create your actual certificate, do the following:

1. Create the new keystore

  • Repeat step 2 in the self-signed section, but this time enter the domain you want the SSL/TLS certificate to be assigned to. For example, if your domain was “mylab.sciencelab.com”, you can run the following:
/usr/local/Java/bin/keytool 
-genkey
-keystore /labkey/apps/lib/mylab.sciencelab.com.jks
-alias tomcat
-keyalg RSA
-keysize 4096

This would create a new dedicated keystore for your new domain. You can opt to use the existing keystore you created, but I prefer to keep that self-signed one separate from the legitimate one.

  • Go through the same steps as step 3 in the self-signed section, enter in your actual domain name when prompted for “What is your first and last name? [Unknown]:”. Everything else stays the same as before.
2. Create the Certificate Signing Request (CSR): Now that the keystore is made, you can now make the CSR that you will then send to the certificate provider (such as GoDaddy.com, Comodo.com, or SSLShopper.com)

Run the following command:

/usr/local/Java/bin/keytool 
-certreq
-alias tomcat
-keyalg RSA
-keysize 4096
-keystore /labkey/apps/lib/mylab.sciencelab.com.jks
-file /labkey/apps/lib/mylab.sciencelab.com.csr

(Note: You can technically give the CSR any name, but I prefer to use the name of the domain and the extension .csr to keep things orderly)

When you run through the CSR, you’ll be prompted similarly like you were when you created the keystore. Enter in all the same respective values.

Once you are finished, open the .csr file using your favorite plain-text editor. You will see a long hash contained within two lines. This is the CSR that you will provide to the certificate provider to order your SSL/TLS certificate.

3. Applying your SSL/TLS certificate:

Once you receive the SSL/TLS certificate from your certificate provider, they may provide you with a few certificates, either 2 certificates (a Root certificate and the certificate for your domain) or 3 certificates (a Root certificate, an intermediate certificate, and the certificate for your domain). Sometimes, you may just get one certificate that has all of those certificates combined. Your certificate provider will provide you with an explanation on what they issued you and instructions on how to use them as well if you are in doubt.

Place the certificate(s) you’ve received in the same directory as the keystore.

If you are provided with a root and/or an intermediate certificate, run the following command:

/usr/local/Java/bin/keytool 
-import
-alias root
-keystore /labkey/apps/lib/mylab.sciencelab.com.jks
-trustcacerts
-file /labkey/apps/lib/certificate_file_name.crt

Take note that the alias is “root” and not the alias you’re using from before. This is intentional. Do not use the alias you used for the CSR or the keystore for this.

Otherwise, if you only received a single certificate, run the following:

/usr/local/Java/bin/keytool 
-import
-alias tomcat
-keystore /labkey/apps/lib/mylab.sciencelab.com.jks
-file /labkey/apps/lib/certificate_file_name.crt

Once the certificate is loaded into the keystore, the final step is to reconfigure the SSL/TLS connector in the server.xml file.

4. Update your server.xml with the new SSL/TLS certificate info:

Repeat step 4 from the self-signed section. Update the connector to point to your new keystore. For example:

<Connector port="443" protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS"
keyAlias="tomcat" keystoreFile="/labkey/apps/lib/mylab.sciencelab.com.jks"
keystorePass="THEPASSWORD" />

Once this is done, restart Tomcat for the changes to take effect.

Note: We recommend using the server.xml examples we provider here within our GitHub repository: https://github.com/LabKey/samples/tree/master/ops/config-examples

Create Special Wildcard and Subject Alternative Names (SAN) Certificates

  • Sometimes you may need to create a certificate that covers multiple domains. There are two types of additional certificates that can be created:
    • Wildcard certificates that would cover any subdomain under the main one.
    • Subject Alternative Name certificates (SAN) that would cover multiple domains.
  • To create a wildcard certificate, you would simply use an asterisk in lieu of a subdomain when creating your keystore and CSR. So the example of mylabs.sciencelab.com, you would use *.sciencelab.com instead and then when requesting your certificate from the provider, you would specifically indicate that you want a wildcard certificate.
  • To create a SAN certificate, you would insert the additional domains and IPs you wish the certificate to apply to when you run the keytool command.
For example:

/usr/local/Java/bin/keytool 
-genkey
-alias tomcat
-keyalg RSA
-keysize 4096
-keystore /labkey/apps/lib/mylab.sciencelab.com.jks
-file /labkey/apps/lib/mylab.sciencelab.com.csr
-ext SAN=dns:mylab.sciencelab.com,dns:mylab.ultimatescience.net,dns:mylab.researcherscience.org,ip:33.33.33.33

Related Topics




Install Microsoft SQL Server


As part of installing required components, you need to install a database server. You can use either PostgreSQL or Microsoft SQL Server as the primary LabKey database. This topic covers the process of setting up a Windows machine to use Microsoft SQL Server as the primary database. Information about other environments is provided below.

Install and Configure Microsoft SQL Server

For current supported versions of Microsoft SQL Server, see Supported Technologies.

Basic Installation Topics:

Additional Topics:

Install Microsoft SQL Server

If you already have a licensed version of Microsoft SQL Server in a supported version, follow the installation instructions noting the requirements outlined for installing an express edition below.

If you don't have a licensed version of Microsoft SQL Server, you can download a free Express Edition. Note that the Express Edition has database size limitations that generally make it inappropriate for production deployments. You can also use the Developer Edition; details may differ slightly from instructions in this topic.

Configure During Installation

During installation, configure Microsoft SQL Server to accept both Windows Authentication and SQL Server Authentication, ("Mixed Mode"), and specify a user name and password for the administrative account.

To enable this feature on SQL Server 2017 Express, select the Custom installation option and proceed through the wizard accepting defaults until the Database Engine Configuration step.

Keep track of the user name and password; LabKey Server uses it to authenticate to SQL Server. It must be provided in plaintext in labkey.xml or in your mssql.properties file later.

If you've already installed SQL Server without enabling SQL Server Authentication then see How to: Change Server Authentication Mode in the Microsoft SQL Server documentation.

Configure TCP/IP

After you've installed SQL Server, you'll need to configure it to use TCP/IP. Follow these steps:

  • Launch the SQL Server Configuration Manager.
  • Under the SQL Server Network Configuration node, select Protocols for <servername>.
  • In the right pane, right-click on TCP/IP and choose Enable.
  • Right-click on TCP/IP and choose Properties.
  • Switch to the IP Addresses tab.
  • Scroll down to the IPAll section, clear the value next to TCP Dynamic Ports and set the value for TCP Port to 1433 and click OK. By default, SQL Server will choose a random port number each time it starts, but the JDBC driver expects SQL Server to be listening on port 1433.
  • Click OK
  • Restart the service by selecting the SQL Server Services node in the left pane, selecting SQL Server <edition name> in the right pane, and choosing Restart from the Action menu (or use the Restart button on the toolbar).

SQL Server Management Studio

Download the SQL Server Management Studio graphical database management tool.

Set Up a Login

You may want to set up a new login (in addition to the "sa" system administrator) for LabKey Server to use to connect to SQL Server:

  • Run SQL Server Management Studio.
  • Connect to the database.
  • Under Security > Logins, add a new login, using SQL Server authentication.
  • Enter the user name and password.
  • Use this password to configure the data source below.

LabKey Configuration File

Next edit the LabKey Server configuration file, by default named labkey.xml, to configure the jTDS JDBC driver for Microsoft SQL Server, which is included in the LabKey Server installation. You should comment out the Resource tag that specifies the PostgreSQL configuration. This Resource tag can be identified by the driverClassName "org.postgresql.Driver". After commenting out this resource tag, it should look similar to the following:

Comment out PostgreSQL Resource Tag

<!--
<Resource name="jdbc/labkeyDataSource" auth="Container"
type="javax.sql.DataSource"
username="postgres"
password="sa"
driverClassName="org.postgresql.Driver"
url="jdbc:postgresql://localhost:5433/labkey162"
maxTotal="20"
maxIdle="10"
maxWaitMillis="120000"
accessToUnderlyingConnectionAllowed="true"
validationQuery="SELECT 1"
/>
-->

Add MS SQL Server Resource Tag

Use the following template for configuring a MS SQL Server data source. Replace USERNAME, PASSWORD, and DATABASE_NAME to fit the particulars of your target data source.

<Resource name="jdbc/labkeyDataSource" auth="Container"
type="javax.sql.DataSource"
username="USERNAME"
password="PASSWORD"
driverClassName="net.sourceforge.jtds.jdbc.Driver"
url="jdbc:jtds:sqlserver://localhost:1433/DATABASE_NAME"
maxTotal="20"
maxIdle="10"
maxWaitMillis="120000"
accessToUnderlyingConnectionAllowed="true"
validationQuery="SELECT 1"/>

Documentation for this driver is available on SourceForge. Other JDBC drivers for Microsoft SQL Server have not been tested.

Note: The maxWaitMillis parameter is provided to prevent server deadlocks. Waiting threads will time out when no connections are available rather than hang the server indefinitely.

Install CLR Functions (GROUP_CONCAT)

When setting up SQL Server, you may need to separately install the GROUP_CONCAT Common Language Runtime (CLR) function. For details see GROUP_CONCAT Install.

For Premium Editions, you may also need to install the PremiumStats CLR functions separately. For details see PremiumStats Install.

Premium EHR users may also need to install LDKNaturalize, following similar methods.

SQL Server Synonyms

LabKey Server supports the use of SQL Server Synonyms. These alternative names function like shortcuts or symlinks, allowing you to "mount" tables and views which actually exist in another schema or database. For more information, see SQL Synonyms.

Installation on Other Platforms

VirtualBox Deployment

On all operating systems, you can set up a VirtualBox VM running Windows and install SQL Server there. These Mac-specific instructions, combined with the documentation above should get you running.

Linux Deployment

Starting with SQL Server 2017, Microsoft has started distributing a native Linux version:

Docker Deployment

The Linux edition of SQL Server is also available within a Docker container. This will work on Mac, Linux, or Windows.
You will need to install a separate visual database management tool:

Related Topics




Install and Set Up R


Set Up Steps: Additional Steps for Linux/OSX: Once an administrator has set up the R environment, your users can create R Reports.

Install R

  • On the R site, choose a CRAN mirror near you.
  • Click Download R for the OS you are using (Linux, OSX, or Windows).
  • Click the subcategory base.
  • Download the installer using the link provided for your platform, for example Download R #.#.# for Windows.
  • Install using the downloaded file.

Tips:

  • You don’t need to download the “contrib” folder on the Install site. It’s easy to obtain additional R packages individually from within R.
  • Details of R installation/admin can be found here.
OS-Specific Instructions:
  • Linux. An example of installing R on Linux is included on the Configure R on Linux page.
  • Windows. On Windows, install R in a directory whose path does not include a space character. The R FAQ warns to avoid spaces if you are building packages from sources.

Configure LabKey Server to Work with R

Configure Authentication and Permissions

Authentication. If you wish to modify a password-protected LabKey Server database through the Rlabkey macros, you will need to set up authentication. See: Create a netrc file.

Permissions. Refer to Configure Permissions for information on how to adjust the permissions necessary to create and edit R Views. Note that only users who have the "Editor" role (or higher) plus either one of the developer roles "Platform Developer" or "Trusted Analyst" can create and edit R reports. Learn more here: Developer Roles.

Batch Mode. Scripts are executed in batch mode, so a new instance of R is started up each time a script is executed. The instance of R is run using the same privileges as the LabKey Server, so care must be taken to ensure that security settings (see above) are set accordingly. Packages must be re-loaded at the start of every script because each script is run in a new instance of R.

Install & Load Additional R Packages

You will likely need additional packages to flesh out functionality that basic install does not include. Additional details on CRAN packages are available here. Packages only need to be installed once on your LabKey Server. However, they will need to be loaded at the start of every script when running in batch mode.

How to Install R Packages

Use the R command line or a script (including a LabKey R script) to install packages. For example, use the following to install two useful packages, "GDD" and "Cairo":

install.packages(c("GDD", "Cairo"), repos="http://cran.r-project.org" )

To install knitr:

install.packages('knitr', dependencies=TRUE)

You can also use the R GUI (Packages > Install Packages) to select and install packages.

How to Load

Each package needs to be installed AND loaded. If the installed package is not set up as part of your native R environment (check ‘R_HOME/site-library’), it needs to be loaded every time you start an R session. Typically, when running R from the LabKey interface, you will need to load (but not install) packages at the start of every script because each script is run in a new instance of R.

To load an installed package (e.g., Cairo), call:

library(Cairo)

Recommended Packages

GDD &/or Cairo: If R runs on a headless Linux server, you will likely need at least one extra graphics package. When LabKey R runs on a headless Linux server, it may not have access to the X11 device drivers (and thus fonts) required by the basic graphics functions jpeg() and png(). Installing the Cairo and/or GDD packages will allow your users to output .jpeg and .png formats without using the jpeg() and png() functions. More details on these packages are provided on the Determine Available Graphing Functions page.

You can avoid the use of Cairo and/or GDD by installing a display buffer for your headless server (see below for more info).

Lattice: Optional. This package is the commonly used, sophisticated graphing package for R. It is particularly useful for creating Participant Charts.

Headless Linux Servers Only: Install the X Virtual Frame Buffer

On Linux servers, the png() and jpg() functions use the device drivers provided by the X-windows display system to do rendering. This is a problem on a headless server where there is generally no display running at all.

As a workaround, you can install the X Virtual Frame Buffer. This allows applications to connect to an X Windows server that renders to memory rather than a display.

For instructions on how to install and configure the X Virtual Frame Buffer on Linux, see Configure the Virtual Frame Buffer on Linux.

If you do not install the X Virtual Frame Buffer, your users may need to use graphics packages such as GDD or Cairo to replace the png() and jpeg() functions. See Determine Available Graphing Functions for further details.

OSX Only: Fix Hostname Resolution Problems

When a LabKey Server runs on OSX, R views can only resolve hostnames that are stored in the server's host file.

It appears that OSX security policy blocks spawned processes from accessing the name service daemon for the operating system.

You can use one of two work-arounds for this problem:

  1. Add the hostname (e.g., www.labkey.org) to the hosts file (/etc/hosts). Testing has shown that the spawned process is able to resolve hostnames that are in the host file.
  2. Use the name "localhost" instead of DNS name in the R script.

Related Topics




Configure an R Docker Engine


Premium Feature — Available in the Professional and Enterprise Editions of LabKey Server. Learn more or contact LabKey.

As an alternative to installing a local instance of R, or using an Rserver server, LabKey Server can make use of an R engine inside a Docker image.

Multiple Docker-based R engines can be configured, each pointing to different Docker images, each of which may have different versions of R and packages installed.

Docker-based R engines are good candidates for "sandboxed" R engines.

To set up LabKey Server to use Docker-based R engines, complete the following steps:

  • Modify a Docker image to fit your needs.
  • Configure Docker and Startup the Image.
  • Configure LabKey Server to use the R engine inside the Docker image.
See details on each step below:

Install Docker

  • Install Docker appropriate your operating system.
    • On a Windows test/development machine, go to Docker Settings. Under General, place a checkmark next to Expose daemon on tcp://localhost:2375. Do not do this on a production server, as this is not a secure configuration. Only appropriate for local development and testing.

(On Development Machines) Set the Base server URL

  • Set the Base Server URL to something other than localhost:8080. For details, see Site Settings. Example value "http://my.server.com".

Make a Docker Image with R Included

LabKey provides two template images to start from. One template is for building a Dockerized R engine using the latest released version of R. The other template is for building an older version of R.

Startup the Docker Image

When the docker image build process finishes, look for the name of your image in the console, for example:

Successfully tagged labkey/rsandbox-base:3.5.1

Start the image:

docker run labkey/rsandbox-base:3.5.1

Configure Docker and Startup the Image

Configure LabKey Server to Use the Docker R Engine

  • In the Edit Engine Configuration dialog, enter the Docker Image Name (at a minimum).
  • The Mount Paths let you (1) map in-container paths to host machine paths and (2) to write script outputs the mapped path on the host machine. For example, the following settings will write the script.R and script.Rout generated files to your host file system at C:scriptsRreports:
Mount (rw): host directory: C:\scripts\Rreports
Mount (rw): container directory: /Users/someuser/cache

Mount (rw) paths refer to Read-Write directories, Mount (ro) refer to Read-Only paths.

  • The Site Default refers to the default sandboxed R engine. If you have only one sandboxed R engine, this cannot be unchecked. If you have more than one sandboxed R engine, you can choose one of them to be the Site Default.

Related Topics




Determine Available Graphing Functions


Determine Available Graphing Functions

Test Machine Capabilities. Before reading this section further, figure out whether you need to worry about its contents. Execute the following script in the R script builder:

if(!capabilities(what = "jpeg") || !capabilities(what="X11"))
warning("You cannot use the jpeg() function on your LabKey Server");
if(!capabilities(what = "png") || !capabilities(what="X11"))
warning("You cannot use the png() function on your LabKey Server");
If this script outputs both warnings, you’ll need to avoid both jpeg() and png() functions. If you do not receive warnings, you can ignore the rest of this section.

Why Don't png() and jpeg() Work? On Unix, jpeg() and png() rely on the x11() device drivers. These are unavailable when R is installed on a "headless" Unix server.

If png() and jpeg() Don't Work, What Are My Options?. You have two categories of options:

  1. Ask your admin to install a display buffer on the server such that it can access the appropriate device drivers.
  2. Avoid jpeg() and png(). There are currently three choices for doing so: Cairo(), GDD() and bitmap().
Which Graphics Function Should I Use? If you are working on a headless server without an installed display buffer, you will need to use Cairo(), GDD() or bitmap(). There are trade-offs for all options. If you use Cairo or GDD, your admin will need to install an additional graphics package. The Cairo package is based upon libraries undergoing continued development and maintenance, unlike the GDD package. Cairo does not require the use of Ghostscript to produce graphics, as does the bitmap() function. However, Cairo() fails to provide all graphics functions on all machines, so you will need to test its capabilities. GDD may provide functions unavailable in Cairo, depending on your machine setup.

Warning: LabKey R usually runs in batch mode, so any call to plot() must be preceded by a call to open the appropriate device (e.g., jpeg() or pdf()) for output. When R runs in its ordinary, interpreted/interactive mode, it opens an appropriate output device for graphics for you automatically. LabKey R does not do this, so you will need to open an output device for graphics yourself. Identifying appropriate devices and function calls is tricky and covered in this section.

Strategy #1: Use the Cairo and/or GDD Packages

You can use graphics functions from the GDD or Cairo packages instead of the typical jpeg() and png() functions.

There are trade-offs between GDD and Cairo. Cairo is being maintained, while GDD is not. GDD enables creation of .gif files, a feature unavailable in Cairo. You will want to check which image formats are supported under your installation of Cairo (this writer's Windows machine can not create .jpeg images in Cairo). Execute the following function call in the script-builder window to determine formats supported by Cairo on your machine:

Cairo.capabilities();
The syntax for using these packages is simple. Just identify the “type” of graphics output you desire when calling GDD or Cairo. The substitution parameters used for file variables are not unique to Cairo/GDD and are explained in subsequent sections.

#   Load the Cairo package, assuming your Admin has installed it:
library(Cairo);
# Identify which "types" of images Cairo can output on your machine:
Cairo.capabilities();
# Open a Cairo device to take your plotting output:
Cairo(file="${imgout:labkeyl_cairo.png}", type="png");
# Plot a LabKey L:
plot(c(rep(25,100), 26:75), c(1:100, rep(1, 50)), ylab= "L", xlab="LabKey",
xlim= c(0, 100), ylim=c(0, 100), main="LabKey in R");
dev.off();

# Load the GDD package, assuming your Admin has installed it:
library(GDD);
# Open a GDD device to take your plotting output:
GDD(file="${imgout:labkeyl_gdd.jpg}", type="jpeg");
# Plot a LabKey L:
plot(c(rep(25,100), 26:75), c(1:100, rep(1, 50)), ylab= "L", xlab="LabKey",
xlim= c(0, 100), ylim=c(0, 100), main="LabKey in R");
dev.off();

Strategy #2: Use bitmap()

It is possible to avoid using either GDD or Cairo for graphics by using bitmap(). Unfortunately, this strategy relies on Ghostscript, reportedly making it slower and lower fidelity than other options. Instructions for installing Ghostscript are available here.

NB: This method of creating jpegs has not been thoroughly tested.

Calls to bitmap will specify the type of graphics format to use:

bitmap(file="${imgout:p2.jpeg}", type = "jpeg");



Configure R on Linux


Steps

The following example shows how to install and configure R on a Linux machine.

> wget http://cran.r-project.org/src/base/R-2/R-2.6.2.tar.gz 
> tar xzf R-2.6.2.tar.gz
> cd R-2.6.2
> ./configure
...
> make
...
> make install
...

Additional Notes

  • These instructions install R under /usr/local (with the executable installed at /usr/local/bin/R
  • Support for the X11 device (including png() and jpeg()) is compiled in R by default.
  • In order to use the X11, png and jpeg devices, an Xdisplay must be available. Thus you may still need to Configure the Virtual Frame Buffer on Linux.

Related Topics




Install SAS/SHARE for Integration with LabKey Server


Overview

Publishing SAS datasets to your LabKey Server provides secure, dynamic access to datasets residing in a SAS repository. Published SAS data sets appear on the LabKey Server as directly accessible datasets. They are dynamic, meaning that LabKey treats the SAS repository as a live database; any modifications to the underlying data set in SAS are immediately viewable on LabKey. The data sets are visible only to those who are authorized to see them.

Authorized users view published data sets using the familiar, easy-to-use grid user interface used throughout LabKey. They can customize their views with filters, sorts, and column lists. They can use the data sets in custom queries and reports. They can export the data in Excel, web query, or TSV formats. They can access the data sets from JavaScript, SAS, R, and Java client libraries.

Several layers keep the data secure. SAS administrators expose selected SAS libraries to LabKey. LabKey administrators then selectively expose SAS libraries as schemas available within a specific folder. The folder is protected using standard LabKey security; only users who have been granted permission to that folder can view the published data sets.

SAS Setup

Before SAS datasets can be published to LabKey, an administrator needs to do three things:

  • Set up the SAS/SHARE service on the SAS installation
  • Set up the SAS/SHARE JDBC driver on the LabKey web server
  • Define SAS libraries as external schemas within LabKey
Set up the SAS/SHARE server. This server runs as part of the SAS installation (it does not run on the LabKey server itself). SAS/SHARE allows LabKey to retrieve SAS data sets over an internal corporate network. The SAS/SHARE server must be configured and maintained as part of the SAS installation. The LabKey installation must be able to connect to SAS/SHARE; it requires high-speed network connectivity and authentication credentials. SAS/SHARE must be configured to predefine all data set libraries that the LabKey installation needs to access.

Set up the SAS/SHARE JDBC driver. This driver allows LabKey to connect to SAS/SHARE and treat SAS data sets as if they were tables in a relational database. The SAS/SHARE JDBC driver must be installed on the LabKey installation. This requires copying two .jar files into the tomcat/lib directory on LabKey. It also requires adding a new DataSource entry in the labkey.xml file on LabKey containing several connection settings (e.g., SAS/SHARE URL and credentials). See External SAS Data Sources.

Define SAS libraries as external schemas within LabKey. A folder administrator chooses which SAS libraries to publish in a LabKey Server folder via the Schema Administration user interface. If a SAS data source is defined in the labkey.xml file, the "Data Source" drop-down list contains the name of this data source as an option. After selecting the data source, the administrator selects the schema (library) name to publish. After clicking the “Create” button, all data sets in that library are published; in other words, they can be viewed by anyone with read permissions in the folder.

For further instructions on how to choose an external schema, see Set Up an External Schema.

Usage

Once defined via the Schema Administration page, a SAS library can be treated like any other database schema (with a couple important exceptions listed below). The query schema browser lists all its data sets as “built-in tables.” A query web part can be added to the folder’s home page to display links to a library’s data sets. Links to key data sets can be added to wiki pages, posted on message boards, or published via email. Clicking any of these links displays the data set in the standard LabKey grid with filtering, sorting, exporting, paging, customizing views, etc. all enabled. Queries that operate on these datasets can be written. The data sets can be retrieved using client APIs (Java, JavaScript, R, and SAS).

Limitations

The two major limitations with SAS data sets are currently:
  • SAS data sets can be joined to each other but not joined to data in the LabKey database. Any attempt to join between different databases (e.g., a table in PostgreSQL with a data set in SAS) results in an error message.
  • The SAS/SHARE JDBC driver provides read-only access to SAS data sets. You cannot insert, update, or delete data in SAS data sets from LabKey.



Server Bootstrap Properties


Some of the initial configuration and startup steps of bootstrapping LabKey Server, i.e. launching a new server on a new database, can be skipped by creating a properties file that is applied at startup.
Configuration steps which can be provided via the bootstrap properties file include, but are not limited to:
  • Some site settings, such as file root and base server URL
  • Script engine definitions, such as automatically enabling R scripting
  • User groups and permission roles
  • Custom content for the home page

Using Bootstrap Property Files

Bootstrap property files are named with a .properties extention and placed in the /trunk/build/deploy/startup directory. Create this directory in your enlistment if it does not exist.

One or more properties files can be defined and placed in this startup directory. The files are applied in reverse-alphabetical order. Any property defined in two such files will retain the "last" setting applied, i.e the one in the file that is "alphabetically" first.

To control specific ordering among files, you could name multiple .properties files with sequential numbers, ordered starting with the one you want to take precedence. For instance, you might have:

  • 01_application.properties: applied last
  • 02_other.properties
  • ...
  • 99_default.properties: applied first
In this example, anything specified in "99_default.properties" will be overridden if the same property is also defined in any lower-numbered file. All values set for properties in the file "01_application.properties" will override any settings included in other files.

Bootstrap Properties File Format

The properties file is a list of lines setting various properties. The format of a given line is:

<optional scope>.<name>;<optional modifier> = <value>

For example, to emulate going to the "Look and Feel Settings", then setting the "initial" "systemEmailAddress" to be "support@labkey.com", you would include this line:

LookAndFeelSettings.systemEmailAddress;bootstrap=support@labkey.com

Modifier Options

The "<optional modifier>" can be one of the following:

  • bootstrap: (default) Apply the properties only when the server first starts on a new database.
  • startup: Apply the properties every time the server starts.
  • immutable: Set and never change.

Create and Apply Property Files

As an example scenario, you might want to bypass the default "first login" UI by preconfiguring your login and password.

  • If it does not exist, create the directory: /trunk/build/deploy/startup
  • Create a .properties file, such as 01default.properties
  • Paste this content, substituting your own email address:
    • "LookAndFeelSettings.systemEmailAddress:initial=MYEMAIL@YOUR_DOMAIN.com"
  • Save the file, ensuring it maintains a .properties extension.
  • Restart your server on a new database.
  • Instead of the traditional new log in steps, the example property is prefilled.

Minimal Set of Properties to Bootstrap LabKey Server

SiteSettings.systemEmailAddress = {systemEmail} SiteSettings.defaultDomain = {defaultDomain} SiteSettings.baseServerURL = {baseServerUrl} SiteSettings.exceptionReportingLevel = HIGH SiteSettings.usageReportingLevel = MEDIUM SiteSettings.sslRequired = true

#Initial User, we may also want to define a site admin user UserRoles.{email} = org.labkey.api.security.roles.ApplicationAdminRole

#Script Engine Definition, not required for bootstrap but required for scenario: ScriptEngineDefinition.R.name;bootstrap = R Scripting Engine ScriptEngineDefinition.R.languageName = R ScriptEngineDefinition.R.languageVersion = {version} ScriptEngineDefinition.R.extensions = R,r ScriptEngineDefinition.R.exePath = {exePath} ScriptEngineDefinition.R.exeCommand = CMD BATCH --slave ScriptEngineDefinition.R.outputFileName = ${scriptName}.Rout

Related Topics

For more about options that can be controlled with a startup properties file:




Set Up Robots.txt and Sitemaps


If your server allows public access, you may wish to customize how external search engines crawl and index your site. Usually this is done through robots.txt and sitemap files.

You can place robots.txt and sitemap files (or other site-specific, static content) into the extraWebapp directory. This directory is located inside the <LabKey_Root>\build\deploy directory on development machines, or under the main LabKey deployment directory on production servers. It is a peer to the labkeyWebapp and modules directories.

Files in the extraWebapp directory will not be deleted when your site is upgraded to a new version of LabKey Server.




GROUP_CONCAT Install


This topic explains how to install the Microsoft SQL Server GROUP_CONCAT CLR (Common Language Runtime) functions. You may need to install these functions as part of setting up a shared SQL Server installation.

GROUP_CONCAT is a SQL aggregate function (similar to SUM, MIN, or MAX) that combines values from multiple rows into a single string value. For example, executing GROUP_CONCAT on a column with row values "First", "Second", and "Third" produces a single value "First, Second, Third". Some databases, such as MySQL, include this as a built-in function. Microsoft SQL Server does not, so LabKey requires a CLR function that implements the capability.

This function is typically installed automatically as part of the regular LabKey installation process. However, the process can fail if, for example, the database user does not have permission to install CLR functions. In these cases, a database administrator needs to install the function manually.

Note on permissions: To install CLR functions, the database user must have sysadmin permissions. When installing on SQL Server in an Amazon AWS RDS instance, this permission is not available. Instead, you need to create a DB Parameter group in RDS. This advanced feature enables installation of CLRs.

On a workstation with a connection to the Microsoft SQL Server Database Server:

  • If the automatic installation has failed, site administrators will see a banner message on the running server reading "The GROUP_CONCAT aggregate function is not installed. This function is required for optimal operation of this server." with two links:
    • Click Download installation script in the banner message to download the required script, named "groupConcatInstall.sql"
    • Click View installation instructions in the banner message to open to this topic.
  • Connect to the Microsoft SQL Server using an account with membership in the sysadmin role.
  • Execute the downloaded SQL script in the database.
  • Confirm that group_concat is installed in the core schema.
  • Restart Tomcat. The changes to the database will be recognized by the server only after a restart.

The Microsoft SQL Server GROUP_CONCAT CLR functions were developed by opcthree. Code is published at http://groupconcat.codeplex.com/ and licensed under the Microsoft Public License (Ms-PL)

Related Topics




Sending Email from Non-LabKey Domains


This topic describes the steps required for a LabKey Cloud Server to be able to send emails from a non-LabKey email address, such as from a client's "home" email domain. In the legacy hosted environment, clients could configure such emails themselves without this process.

Background

To send email from the client email domain, clients must authorize LabKey to send email on their behalf by creating a new text record in their DNS system known as a DKIM (DomainKeys Identified Mail) record.

What is DKIM?

DKIM is an email authentication method designed to detect email spoofing and prevent forged sender email addresses. Learn more on Wikipedia.

Why has LabKey implemented this new requirement?

LabKey takes client security seriously. LabKey Cloud Servers typically do not use client email servers to send email. Further, many clients use LabKey to manage PHI data and thus need to meet strict compliance guidelines. With LabKey using DKIM authorization, clients can be assured that email originating from LabKey systems has been authorized by their organization thus increasing the level of trust that the content of the email is legitimate.

PostMark

How does mail get sent from a LabKey Cloud Server?

To prevent mail from our servers being filtered as spam and generating support calls when clients can't find messages, LabKey uses a mail service called PostMark.

PostMark confirms through various methods that mail being sent by its servers isn't spam, and can therefore be trusted by recipients.

One part of the configuration requires that every "FROM" email domain being sent by through the LabKey account has a DKIM record. A DKIM record is like a password that tells PostMark LabKey has permission to send mail from that domain. This prevents domain-spoofing in emails coming from LabKey and being sent through PostMark, thus ensuring the integrity of both LabKey and PostMark's reputation.

When LabKey sends a message from one of our cloud servers, it is sent to a specific PostMark email server via a password-protected account. PostMark then confirms the domain is one LabKey has a DKIM record for.

Because PostMark's model is to protect domains, LabKey cannot assign DKIM records to specific hosts, only to domains like labkey.com. As such, mail is sent from our cloud servers as username@domain, as opposed to username@host.domain.

If there's no DKIM for the domain in the email address, PostMark bounces the email from its server and never sends it. If the domain is DKIM approved, the mail is then sent on to the recipient.

Configure DNS Records

To configure DNS records so mail from client email address goes through, the following steps must be completed by both LabKey and the client:

  1. The client tells LabKey which domain they want to send email from.
  2. LabKey's DevOps team then configures PostMark to accept mail with that domain in the from address. At this point, PostMark gives LabKey a DKIM record.
  3. LabKey passes the DKIM records to the client for the client to add to their DNS provider.
  4. The client tells LabKey when they've done this and the LabKey DevOps team confirms that the DKIM record is properly configured.
  5. LabKey sends a test message from that domain to ensure the mail is being accepted and sent.
  6. LabKey informs the client that they can then send from their domain.
This entire process can be done in less than a day, provided the client is able to add the DKIM record with quick turnaround.

DKIM Records

What are the ramifications of adding a DKIM record for the client?

Because DKIM records are TXT records specific to PostMark, these records have no impact on the client apart from authorizing their LabKey Cloud Server to send email with their domain name. DKIM records do not impact existing mail configurations that the client is already using. They do not supplant MX records or add to them. For all intents and purposes, this record is invisible to the client -- it is only used by PostMark when mail is sent from a LabKey server with the client's domain in the from field.

Workarounds

Is there any way around needing the client to add a DKIM record?

  • If the client wants to send mail from their domain from a LabKey Cloud Server, they must add the DKIM record.
  • If they do not add this record, clients can configure their LabKey Cloud Server to send email from a LabKey domain (e.g. do_not_reply@labkey.com). LabKey has already created DKIM records for its email domains.



PremiumStats Install


Premium Feature — An enhanced set of summary statistics is available in all Premium Editions of LabKey Server. Learn more or contact LabKey.

This topic explains how to install the Microsoft SQL Server PremiumStats CLR functions. You may need to install these functions as part of setting up a SQL Server installation with a Premium Edition of LabKey Server.

PremiumStats is a CLR (Common Language Runtime) assembly with aggregate functions supporting LabKey Premium summary statistics, including median, median absolute deviation, quartiles, and interquartile ranges. Microsoft SQL Server does not support these natively, so LabKey requires a CLR assembly to implement these capabilities.

This assembly and functions are typically installed automatically as part of the regular LabKey installation process. However, the process can fail if, for example, the database user does not have permission to install CLR assemblies. In these cases, a database administrator needs to install the assembly manually.

On a workstation with a connection to the Microsoft SQL Server Database Server:

  • If the automatic installation has failed, site administrators will see a banner message on the running server reading "The premium aggregate functions are not installed. These functions are required for premium feature summary statistics." with two links.
    • Click Download installation script in the banner message to download the required script: "premiumAggregatesInstall.sql"
    • Click View installation instructions in the banner message to open to this topic.
  • Connect to the Microsoft SQL Server using an account with membership in the sysadmin role.
  • Execute the downloaded SQL script in the database.
  • Confirm that PremiumStats is installed in the core schema.
  • Restart Tomcat. The changes to the database will be recognized by the server only after a restart.

Related Topics




Deploying an AWS Web Application Firewall


This topic outlines the process for deploying an AWS Web Application Firewall (WAF) to protect LabKey instances from DDoS and Malicious Attacks.

Purpose

Public facing LabKey instances are subject to “internet background radiation” by nefarious miscreants who seek to comprise systems to gain access to protected data. Typically motivated by financial extortion, these individuals use DDoS and bot networks to attack victims. Fortunately there are some easy and low cost tools to protect against many attacks. This document describes how to deploy an AWS Web Application Firewall (WAF) to protect against the OWASP top 10 vulnerabilities and many malicious bot networks.

Related Documentation

Tutorial: Implementing a DDoS-resistant Website Using AWS Services:

Requirements

  • LabKey Instance Deployed in AWS on EC2
  • Configured Elastic Load Balancer with target group routing to LabKey EC2 Instance
  • Required AWS Permissions to use CloudFormation, WAF, IAM Policies, S3, Lambda etc.

Considerations

Many LabKey core features require uploading and downloading of files. These types of activities are difficult to distinguish from malicious activities as the methods used to upload malicious code is indistinguishable from normal workflows. To address possible false positives, clients have the following options:

  • Use the IP Address whitelist function of the WAF to whitelist specific IP addresses or IP Address ranges of users originating from within the clients network. (e.g. whitelist the Public NAT gateway of the client’s network).
  • If whitelisting is not feasible due to expectation of random user IP addresses from various internet locations; consider setting the XSS rule to count vs block. (See information below). While this may reduce the effectiveness of the WAF to protect against XSS attacks, clients still gain the benefit of other WAF features which block known malicious attacker source IP’s.

Deployment

Architecture

Deployment Steps

Follow the AWS Tutorial for detailed steps to deploy the WAF using the CloudFormation Template: High level steps:
  1. Deploy the WAF in the same region as your LabKey Instance.
  2. Configure the WAF to protect your Elastic Load Balancer
  3. Optional: Configuring IP Address Whitelists
  4. Login to AWS and go to the WAF Console.
  5. From the WAF Console, choose WebACL’s.
  6. In the WebACL Conditions, choose IP Addresses.
  7. Edit the "Whitelist Set" rule to include the IP addresses or address ranges you wish to whitelist.
  8. Test.

Optional: Configuring XSS Rule from Block to Count

Edit the WebACL following the steps below:

  1. Login to AWS and go to the WAF Console.
  2. From the WAF Console, choose WebACL’s.
  3. Click the WAF Name in the WebACL’s list. In the resulting dialog that opens, click the Rules tab to see the list of the ACL rules.
  4. Click the Edit web ACL button.
  5. For the XSS Rule, change the rule from "Block" to "Count".
  6. Click the Update button to save changes.
  7. Test.



Configure the Virtual Frame Buffer on Linux


You may need to configure the X virtual frame buffer in order for graphics functions such as png() to work properly in R. This page walks you through an example installation and configuration of the X virtual frame buffer on Linux. For further information on when and why you would need to configure the virtual frame buffer, see Install and Set Up R.

Example Configuration

  • Linux Distro: Fedora 7
  • Kernel: 2.6.20-2936.fc7xen
  • Processor Type: x86_64

Install R

Make sure you have completed the steps to install and configure R. See Install and Set Up R for general setup steps. For Linux-specific instructions, see Configure R on Linux.

Install Xvfb

If the name of your machine is <YourServerName>, use the following:

[root@<YourServerName> R-2.6.1]# yum update xorg-x11-server-Xorg 
[root@<YourServerName> R-2.6.1]# yum install xorg-x11-server-Xvfb.x86_64

Start and Test Xvfb

To start Xvfb, use the following command:

[root@<YourServerName> R-2.6.1]# /usr/bin/Xvfb :2 -nolisten tcp -shmem

This starts a Display on servernumber = 2 and screen number = 0.

To test whether the X11, PNG and JPEG devices are available in R:

[root@<YourServerName> R-2.6.1]# export DISPLAY=:2.0 
[root@<YourServerName> R-2.6.1]# bin/R

You will see many lines of output. At the ">" prompt, run the capabilities() command. It will tell you whether the X11, JPEG and PNG devices are functioning. The following example output shows success:

> capabilities() 
jpeg png tcltk X11 http/ftp sockets libxml fifo
TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
cledit iconv NLS profmem
TRUE TRUE TRUE FALSE

Make configuration changes to ensure that Xvfb is started at boot-time

You need to make sure that Xvfb runs at all times on the machine or R will not function as needed. There are many ways to do this. This example uses a simple start/stop script and treats it as a service.

The script:

[root@<YourServerName> R-2.6.1]# cd /etc/init.d 
[root@<YourServerName> init.d]# vi xvfb
#!/bin/bash
#
# /etc/rc.d/init.d/xvfb
#
# Author: Brian Connolly (LabKey.org)
#
# chkconfig: 345 98 90
# description: Starts Virtual Framebuffer process to enable the
# LabKey server to use R.
#
#

XVFB_OUTPUT=/usr/local/labkey/Xvfb.out
XVFB=/usr/bin/Xvfb
XVFB_OPTIONS=":2 -nolisten tcp -shmem"

# Source function library.
. /etc/init.d/functions


start() {
echo -n "Starting : X Virtual Frame Buffer "
$XVFB $XVFB_OPTIONS >>$XVFB_OUTPUT 2>&1&
RETVAL=$?
echo
return $RETVAL
}

stop() {
echo -n "Shutting down : X Virtual Frame Buffer"
echo
killproc Xvfb
echo
return 0
}

case "$1" in
start)
start
;;
stop)
stop
;;
*)
echo "Usage: xvfb {start|stop}"
exit 1
;;
esac
exit $?

Now test the script with the standard:

[root@<YourServerName> etc]# /etc/init.d/xvfb start 
[root@<YourServerName> etc]# /etc/init.d/xvfb stop
[root@<YourServerName> etc]# /etc/init.d/xvfb
This should work without a hitch.

Note: Any error messages produced by Xvfb will be sent to the file set in

$XVFB_OUTPUT.
If you experience problems, these messages can provide further guidance.

The last thing to do is to run chkconfig to finish off the configuration. This creates the appropriate start and kills links in the rc#.d directories. The script above contains a line in the header comments that says "# chkconfig: 345 98 90". This tells the chkconfig tool that xvfb script should be executed at runlevels 3,4,5. It also specifies the start and stop priority (98 for start and 90 for stop). You should change these appropriately.

[root@<YourServerName> init.d]# chkconfig --add xvfb
Check the results:
[root@<YourServerName> init.d]# chkconfig --list xvfb 
xvfb 0:off 1:off 2:off 3:on 4:on 5:on 6:off

Verify that the appropriate soft links have been created:

[root@<YourServerName> init.d]# ls -la /etc/rc5.d/ | grep xvfb 
lrwxrwxrwx 1 root root 14 2008-01-22 18:05 S98xvfb -> ../init.d/xvfb

Start the Xvfb Process and Setup the DISPLAY Env Variable

Start the process using:
[root@<YourServerName> init.d]# /etc/init.d/xvfb start

Now you will need to the set the DISPLAY env variable for the user. This is the DISPLAY variable that is used to run the TOMCAT server. Add the following the .bash_profile for this user. On this serer, the TOMCAT process is run by the user tomcat

[root@<YourServerName> ~]# vi ~tomcat/.bash_profile 
[added]
# Set DISPLAY variable for using LabKey and R.
DISPLAY=:2.0
export DISPLAY

Restart the LabKey Server or it will not have the DISPLAY variable set

On this server, we have created a start/stop script for TOMCAT within /etc/init.d. So I will use that to start and stop the server

[root@<YourServerName> ~]# /etc/init.d/tomcat restart

Test the configuration

The last step is to test that when R is run inside of the LabKey server, the X11,JPEG and PNG devices are available

Example:

The following steps enable R in a folder configured to track Issue/Bug Tracking:

  1. Log into the Labkey Server with an account with Administrator privs
  2. In any Project, create a new SubFolder
  3. Choose a "Custom"-type folder
  4. Uncheck all boxes on the right side of the screen except "Issues."
  5. Hit Next
  6. Click on the button "Views" and a drop-down will appear
  7. Select "Create R View"
  8. In the text box, enter "capabilities()" and hit the "Execute Script" button.
You should see the following output:
jpeg png tcltk X11 http/ftp sockets libxml fifo 
TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
cledit iconv NLS profmem
FALSE TRUE TRUE FALSE
> proc.time()
user system elapsed
0.600 0.040 0.631

The important thing to see here is that X11, png and jpeg all say "TRUE." If they do not, something is wrong.




Community Edition Modules - Descriptions


This topic is under construction for the July 2020 release of LabKey Server. For current documentation of this feature, click here.

Modules are the functional building blocks of LabKey Server. A folder's functionality is determined by the set of modules that are enabled in that folder. The Folder Type of a project or folder determines an initial set of enabled modules, and additional modules included in the deployment can be enabled as necessary. For details see Enable a Module in a Folder.

To learn which modules are included in the various editions of LabKey Server, see Modules in LabKey Server Editions.

This topic provides a few details about some modules included in the standard distribution of LabKey Server Community Edition. Other open source modules not included in the Community Edition can be downloaded and built locally with sufficient expertise and are listed in Additional Source Modules.

Announcements

Provides a ready-to-use message board where users can post announcements and files, and participate in threaded discussions. See Messages.

Audit

Records user activity on the server. See Audit Site Activity.

Core

The Core module provides central services such administration, folder management, user management, module upgrade, file attachments, analytics, and portal page management.

Demo

The Demo module helps you get started building your own LabKey Server module. It demonstrates all the basic concepts you need to understand to extend LabKey Server with your own module.

Elisa

Implements the ELISA assay.

ELISpotAssay

Implements the ELISpot assay.

Experiment

The Experiment module provides annotation of experiments based on FuGE-OM standards. This module defines the XAR (eXperimental ARchive) file format for importing and exporting experiment data and annotations, and allows user-defined custom annotations for specialized protocols and data.

Web Parts included:

  • Experiment Runs
  • Experiments
  • Lists
  • Sample Types
  • Single List
  • Experiments -> Narrow
  • Protocols -> Narrow
  • Sample Types -> Narrow

FCSExpress

Supports importation and analysis of flow cytometry data from FCS Express.

FileContent

The FileContent module lets you share files on your LabKey Server via the web.

Web parts included:

  • Files

Flow

The Flow module supplies flow-specific services to the flow application.

Web Parts included:

  • Flow Analysis (Flow Analysis Folders)
  • Flow Analysis Scripts
  • Flow Overview (Experiment Management)

Issues

The Issues module provides a ready-to-use workflow system for tracking tasks and problems across a group.

List

Lists are light-weight data tables, often used to hold utility data that supports an application or project, such as a list of instrument configurations.

Luminex

Supports Luminex data import and analysis.

MS2

The MS2 module supplies MS2-specific services to the MS2/CPAS application.

Web Parts included:

  • MS2 Runs
  • MS2 Runs, Enhanced
  • MS2 Sample Preparation Runs
  • Protein Search
  • MS2 Statistics -> Narrow
  • Protein Search -> Narrow

NAb

The NAb module provides tools for planning, analyzing and organizing experiments that address Neutralizing Antibodies. No web parts are provided. Access NAb services via a custom tab in a custom folder.

Pipeline

The Data Pipeline module uploads experiment data files to LabKey Server. You can track the progress of uploads and view log and output files. These provide further details on the progress of data files through the pipeline, from file conversion to the final location of the analyzed runs.

Query

The Query module allows you to create customized views by filtering and sorting data.

Search

The Search module offers full-text search of server contents, implemented by Lucene.

Study

The Study module provides a variety of tools for integration of heterogeneous data types, such as demographic, clinical, and experimental data. Cohorts and participant groups are also supported by this module.

Survey

The Survey module supports custom user surveys for collecting user information, feedback, or participant data.

TargetedMS

Supports targeted mass spectrometry proteomics experiments, better known as Panorama.

Visualization

Implements the core data visualization features, including box plots, scatter plots, time charts, etc.

Wiki

The Wiki module provides a simple publishing tool for creating and editing web pages on the LabKey site. It includes the Wiki, Narrow Wiki, and Wiki TOC web parts.

Related Topics




Additional Source Modules


LabKey Server modules requiring significant customization by a developer are not included in LabKey distributions. Developers can build these modules from source code in the LabKey repository. Please contact LabKey to inquire about support options.

Additional Open Source Modules

ModuleDescriptionDocumentationSource
Reagent InventoryOrganize and track lab reagents.Reagent ModuleSVN Source
File TransferTransfer files from a Globus endpoint.File Transfer Module / Globus File SharingGitHub Source
SNDStructured Narrative DatasetsStructured Narrative DatasetsContact LabKey
GitHub ProjectsMany more modules are available as GitHub projects, including Signal Data and Workflow.For documentation, see the README.md file in each project.GitHub Module List

If you wish to engage professional services to help you adopt these modules, please contact LabKey.

Related Topics