Table of Contents

guest
2019-08-19
     Install LabKey
       Premium Resource: System Requirements for Installing LabKey Server
       Premium Resource: Install LabKey on Linux Systems
       Install LabKey Manually
         Install Required Components
         Configure the LabKey Web Application
         labkey.xml Configuration File
         Third-Party Components and Licenses (Optional Install)
         Creating & Installing SSL/TLS Certificates on Tomcat
       Example Hardware/Software Configurations
       Common Install Tasks
         Install Microsoft SQL Server
         Install PostgreSQL (Windows)
         Install PostgreSQL (Linux or OSX)
         Notes on Installing PostgreSQL on All Platforms
         Install and Set Up R
           Configure an R Docker Engine
           Determine Available Graphing Functions
           Configure R on Linux
         Install SAS/SHARE for Integration with LabKey Server
         Configure Tomcat Webapp Memory
         Server Bootstrap Properties
         Set Up Robots.txt and Sitemaps
         GROUP_CONCAT Install
         Sending Email from Non-LabKey Domains
         PremiumStats Install
         Deploying an AWS Windows Application Firewall
         Configure the Virtual Frame Buffer on Linux
       Supported Technologies
       Troubleshoot Server Installation and Configuration
         Installation Error Messages
         Dump Memory and Thread Usage Debugging Information
       Community Edition Modules - Descriptions
         Additional Source Modules

Install LabKey


These topics explain the options and process for installing LabKey Server.

If you are upgrading an existing installation, see Upgrade LabKey.

If you wish to evaluate LabKey Server, consider creating a trial server instance. Both hosted and local (VM) trial options are available.

Installation Options

  • Install LabKey Server
    • Windows: install required components independently. Suitable for evaluation or for more complex, enterprise-scale installations.
    • Linux, Unix, or Mac: install required components independently.
  • Enterprise Pipeline - Install the enterprise pipeline.

Troubleshooting

Other Install Topics


Premium Resources Available

Subscribers to premium editions of LabKey Server can learn more with these topics:


Learn more about premium editions




Premium Resource: System Requirements for Installing LabKey Server





Premium Resource: Install LabKey on Linux Systems





Install LabKey Manually


LabKey Server is a Java web application that runs under Apache Tomcat and accesses a relational database, either PostgreSQL or Microsoft SQL Server. LabKey Server can also reserve a network file share for the data pipeline, and use an outgoing (SMTP) mail server for sending system emails. LabKey Server may optionally connect to an LDAP server to authenticate users within an organization.

If you wish to evaluate LabKey Server, we recommend that you consider a trial server instance.

Step-by-Step Manual Install

If you are manually installing LabKey Server, you need to download, install, and configure all of its components yourself. The following topics explain how to do this in a step-by-step fashion:

If you are installing manually on Linux or OSX, the instructions assume that you have super-user access to the machine, and that you are familiar with linux commands and utilities such as wget, tar, chmod, and ln.

Related Topics




Install Required Components


Before manually installing or upgrading LabKey Server, you'll need to install the correct versions of all of the required components. We recommend only using the versions we have tested and approved, even if they are not the most recent editions. Find details in the topic: Supported Technologies. This topic outlines the installation process for the components, using # to represent the specific version numbers.

Create Folder Structure

For simplicity, we recommend using the directory structure described here, particularly whenever you are creating a new LabKey installation from scratch.

First create a "labkey" directory at the root level. On Windows, "C:\labkey"; on Mac or Linux, "/usr/local/labkey". This directory is known as <LABKEY_ROOT>. Within this directory, create the following subdirectories. For simplicity, our documentation uses the / slash directory separator; keep in mind that your machine may use \ backslashes:

  • apps: This subfolder will hold the third party components: Tomcat, PostgreSQL, OpenJDK.
  • labkey: This subfolder (i.e. C:\labkey\labkey) is where your actual LabKey installation will go. This location is known as <LABKEY_HOME>.
  • backups: This subfolder will store database and file backups as needed.
  • src: This subfolder will house binaries and source code.
  • src/labkey: Within the "src" directory, create another subdirectory "labkey". This where you will place the downloaded binaries for LabKey Server.

Install Java

Install the latest supported release of OpenJDK.

  1. Download the binary to the <LABKEY_ROOT>/src directory you created.
  2. From this directory, unzip the contents to the <LABKEY_ROOT>/apps/ directory. On Windows, for example: C:\labkey\apps\jdk-##. Our documentation refers to this location as <JAVA_HOME>.
  3. On Windows, set the environment variable JAVA_HOME (no angle brackets) to this location.
  4. Add the /bin subdirectory of this new location to your PATH. Ex: C:\labkey\apps\jdk-##\bin
  5. To test this installation on Windows, open a new command window and type:
    java -version
  6. It should print the version number of the JDK you just installed. If it does not, check your path and JAVA_HOME variable setting.
On Linux, create a symbolic link from /usr/local/java to your <JAVA_HOME> location (for example, /usr/local/labkey/apps/jdk-##). This will make it easier to upgrade in the future.

Install the Apache Tomcat Web Server

We recommend running LabKey Server with the latest approved version of Tomcat. For the specific version to download for your platform, see Supported Tomcat Versions.

Install Tomcat

1. Download the binaries to <LABKEY_ROOT>/src

2. Install to <LABKEY_ROOT>/apps/apache-tomcat-#.#.##. We will call this directory <CATALINA_HOME>. Note that if you were to install Tomcat using their package manager, files and directories such as /lib may be placed in different locations.

  • On Linux, create a symbolic link from /usr/local/tomcat to this location (for example, /usr/local/labkey/apps/apache-tomcat-#.#.##).
3. Configure Tomcat to use the JDK you installed above: You can do this either by creating a JAVA_HOME environment variable under the user account that will be starting tomcat, or by adding that variable to the tomcat startup scripts, <CATALINA_HOME>/bin/startup.sh on Linux or startup.bat on Windows. For example, on Linux add this line to the beginning of the tomcat's startup.sh file:
Export JAVA_HOME=/usr/local/java

4. Start Tomcat. On Linux run <CATALINA_HOME>/bin/startup.sh. If you want Tomcat to start up automatically when you restart your computer see the Tomcat documentation.

5. Test your Tomcat installation by entering "http://<machine_name or localhost or IP_address>:8080" in a web browser. If your Java and Tomcat installations are successful you will see the Tomcat success page.

Install the Database Server

Install one of the following database servers:

Platform-specific installation instructions:

Install the LabKey Server System Components

1. Click Here to Register and Download LabKey Binaries. Choose either:

  • The current binary zip distribution (LabKey##.#.#-#####-bin.zip)
    • OR
  • The current binary tar.gz distribution (LabKey##.#.#-#####-bin.tar.gz)
Place the binaries in the <LABKEY_ROOT>/src/labkey directory.

2. Unzip the LabKey Server components in this same folder location. You will move these components later, but unpacking them in a subdirectory of the <LABKEY_ROOT>/src/labkey directory will help you find them later.

    • On Linux or OSX systems, the following command will unzip and untar the archive.
tar xfz LabKey##.#.#-#####-bin.tar.gz

After unpacking the directory should contain these files and directories:

  • bin: Binary files required by LabKey Server (only in Windows distributions).
  • labkeywebapp: The LabKey Server web application.
  • modules: LabKey Server modules.
  • pipeline-lib: Jars for the data processing pipeline.
  • tomcat-lib: Required server library jars.
  • labkey.xml: LabKey Server configuration file.
  • manual-upgrade.sh: Manual upgrade script for OSX and non-supported Linux installations.
  • README.txt: A file pointing you to this documentation.
  • VERSION: A file containing the release number and build number.

Next Steps

After you've downloaded and installed all components, you'll configure the LabKey Server web application to run on Tomcat. See Configure the LabKey Web Application.

Related Topics




Configure the LabKey Web Application


After you've installed all of the required components, you need to follow some additional steps to configure LabKey Server to run on Tomcat. These steps apply to either a new or an existing Tomcat installation.

These instructions assume you have followed our guidance for creating the LabKey folder structure. The <LABKEY_ROOT>, <LABKEY_HOME>, and <CATALINA_HOME> are defined there and used in this topic. <CATALINA_HOME> is where Tomcat is installed.

Configure Tomcat to Run the LabKey Server Web Application

Follow these steps to run LabKey Server on Tomcat:

  1. Move the LabKey Server Libraries
  2. Configure your LabKey Server home directory
  3. Move the LabKey Server Binary Files and Add a Path Reference
  4. Move the LabKey Server Configuration File
  5. Replace Values in the LabKey Server Configuration File
  6. Configure Webapp Memory
  7. Configure LabKey Server to Run Under TLS/SSL (Recommended)
  8. Configure Tomcat Session Timeout (Optional)
  9. Configure Tomcat to Support Extended Characters in URLs (Optional)
  10. Configure Tomcat to Use Gzip (Optional)
  11. Start the Server

1. Move the LabKey Server Libraries

If you did not download and unzip the LabKey Server binaries in the previous step, do so now.

Copy all the JAR files from the unzipped package's tomcat-lib directory to the <CATALINA_HOME>/lib directory. These include:

  • jtds.jar
  • mail.jar
  • mysql.jar
  • postgresql.jar
  • labkeyBootstrap.jar
  • javax.activation.jar

2. Configure your LabKey Home directory

Navigate to the <LABKEY_HOME> directory you created earlier, i.e. the <LABKEY_ROOT>/labkey subdirectory.

  • On Windows the default <LABKEY_HOME> is C:\labkey\labkey
  • On Unix the default <LABKEY_HOME> is /usr/local/labkey/labkey
On Unix systems, ensure that you have ownership over <LABKEY_HOME> by calling "sudo chown". (Failing to do this can cause problems with the pipeline file root.) For example, the user "steveh" assumes ownership of <LABKEY_HOME> using the following command:

sudo chown steveh /usr/local/labkey/labkey

Copy the following directories from the unzipped binaries to the <LABKEY_HOME> subdirectory, i.e. to <LABKEY_ROOT>/labkey.

  • /bin (only applicable in Windows distributions)
  • /labkeywebapp
  • /modules
  • /pipeline-lib
  • Make sure that you move the /labkeywebapp directory to <LABKEY_HOME>. Do not move it to the <CATALINA_HOME>/webapps folder.
  • The user who is executing the Tomcat process must have write permissions for the /labkeywebapp and /modules directories within <LABKEY_HOME>

3. Move the LabKey Server Binary Files and Add a Path Reference

The Windows LabKey Server binary distribution includes a /bin directory that contains a number of pre-built Windows executable files required by LabKey Server. On Windows, you have already moved this directory to <LABKEY_HOME> in the previous step. On Unix you must download and either install or build these components for your system, and install them to <LABKEY_HOME>/bin. For more information see Third-Party Components and Licenses (Optional Install).

Once the components are in place, add a reference to this directory to the system path of the user account that will start Tomcat.

4. Move the LabKey Server Configuration File

The LabKey Server configuration file, named labkey.xml by default, contains a number of settings required by LabKey Server to run. Find it in the unzipped distribution and move this file into the following location. Create the intermediate "Catalina" and "localhost" subdirectories if necessary to make it possible to place it here:

  • <CATALINA_HOME>/conf/Catalina/localhost/labkey.xml.

5. Replace Values in the LabKey Server Configuration File

The LabKey Server configuration file contains basic settings for your LabKey Server application. When you install manually, you need to edit this file to provide these settings.

In order to make them easier to find, the parameter values you need to change are surrounded by "@@" in the template we provide. For example, @@appDocBase@@, @@jdbcUser@@, @@jdbcPassword@@, etc. stand in for the actual values you will enter here.

  • Replace @@appDocBase@@ with <LABKEY_HOME>/labkeywebapp (use the full path, i.e. C:\labkey\labkey\labkeywebapp
  • Replace @@jdbcUser@@ and @@jdbcPassword@@ with administrator credentials for your chosen database (PostgreSQL or SQL Server).
For more information on replacing values in labkey.xml, see labkey.xml Configuration File.

6. Configure Webapp Memory

Follow the instructions in the following topic: Configure Tomcat Webapp Memory.

7. Configure LabKey Server to Run Under SSL/TLS (Recommended)

You can configure LabKey Server to run under TLS (Transport Layer Security), or it's predecessor, SSL (Secure Socket Layer). We recommend that you take this step if you are setting up a production server to run over a network or over the Internet, so that your passwords and data are not passed over the network in clear text.

To configure Tomcat to run LabKey Server under SSL/TLS:

To require that users connect to LabKey Server using a secure (https) connection:
  • In the LabKey Server Admin Console ( (Admin) > Site > Admin Console), click Admin Console Links, then the Site Settings link.
  • Check Require SSL connections.
  • Enter the SSL/TLS port number that you configured in the previous step in the SSL Port field.
An example server.xml file is available at: https://github.com/LabKey/samples/blob/master/ops/config-examples/server-SSL.xml

8. Configure Tomcat Session Timeout (Optional)

Tomcat's session timeout specifies how long a user remains logged in after their last session activity, 30 minutes by default. To increase session timeout, edit <CATALINA_HOME>/conf/web.xml . Locate the <session-timeout> tag and set the value to the desired number of minutes.

9. Configure Tomcat to Support Extended Characters in URLs

We strongly recommend running on Tomcat 8.5.x or 9.0.x. These versions are configured to support extended characters by default.

If you are running on Tomcat 7.x then you will need to configure it to handle extended characters.

10. Configure Tomcat to Use Gzip (Optional)

You may be able to improve responsiveness of your server by configuring Tomcat to use gzip compression when it streams data back to the browser.

You can enable gzip in <CATALINA_HOME>/conf/server.xml by adding a few extra attributes to the active non-SSL <Connector> elements:

compression="on" compressionMinSize="2048"
noCompressionUserAgents="gozilla, traviata"
compressableMimeType="text/html, text/xml, text/javascript, text/plain, text/css, application/json"

Note that there is a comment in the default file that provides basic instructions for enabling gzip. The snippet above improves on the comment's recommendation by enabling compression on a few extra MIME types.

11. Start the Server

Start Tomcat as a Service

If you are using LabKey Server only on your local computer, you can start and stop Tomcat manually using the scripts in <CATALINA_HOME>/bin. In most cases, however, you'll probably want to run Tomcat automatically, so that the operating system manages the server's availability. Running Tomcat as a service is recommended on Windows. You can call the service.bat script in the <CATALINA_HOME>/bin directory to install or uninstall Tomcat as a service running on Windows.

Navigate to <CATALINA_HOME>/bin and run:

service.bat install

After Tomcat has been installed as a service, you can use the Windows Services management utility (Control Panel > Administrative Tools > Services) to start, stop, and restart the LabKey Server Apache Tomcat service.

If you are installing on a different operating system, you will probably also want to configure Tomcat to start on system startup.

Start Tomcat with Startup Scripts

You can also start the Tomcat server using the startup scripts in the <CATALINA_HOME>/bin directory.

View the LabKey Server Home Page

After you start the server, point your web browser at http://localhost:8080/labkey/ if you have installed LabKey Server on your local computer, or at http://<server-name>:8080/labkey/ if you have installed LabKey Server on a remote server.

Configure the Tomcat Default Port

Note that in the addresses list above, the port number 8080 is included in the URL. Tomcat uses port 8080 by default, and to load any page served by Tomcat, you must either specify the port number as shown above, or you must configure the Tomcat installation to use a different port number. To configure the Tomcat HTTP connector port, edit the server.xml file in the <tomcat-home>/conf directory. Find the entry that begins with <Connector port="8080" .../> and change the value of the port attribute to the desired port number. In most cases you'll want to change this value to "80", which is the default port number used by web browsers. If you change this value to "80", users will not need to include the port number in the URL to access LabKey Server.

Existing Installations of Tomcat

For detailed version information, see Supported Tomcat Versions.

You can run two web servers on the same machine only if they use different port numbers, so if you have a web server running you may need to reconfigure one to avoid conflicts.

If you have an existing installation of Tomcat, you can configure LabKey Server to run on that installation. Alternately, you can install a separate instance of Tomcat for LabKey Server; in that case you will need to configure each instance of Tomcat to use a different port. If you have another web server running on your computer that uses Tomcat's default port of 8080, you will also need to configure Tomcat to use a different port.

If you receive a JVM_BIND error when you attempt to start Tomcat, it means that the port Tomcat is trying to use is in use by another application. The other application could be another instance of Tomcat, another web server, or some other application. You'll need to configure one of the conflicting applications to use a different port. Note that you may need to reconfigure more than one port setting. For example, in addition to the default HTTP port defined on port 8080, Tomcat also defines a shutdown port at 8005. If you are running more than one instance of Tomcat, you'll need to change the value of the shutdown port for one of them as well.

Next Step in Manual Installation




labkey.xml Configuration File


The LabKey Server configuration file, labkey.xml by default, contains settings required for LabKey Server to run on Tomcat. This topic describes modifications you may need to make to that file. The previous step included moving it to <tomcat-home>/conf/Catalina/localhost.

Configuration File Name

The default URL for your LabKey Server installation is http://<servername>/labkey, so the configuration file name is labkey.xml.

If you wish to rename the default URL path to include something else in the place of "labkey", you can change the name by renaming the labkey.xml file. If you do this, you should do it when you first install LabKey Server, as changing the name later will cause any external links to your application to break. Note that Tomcat treats URLs as case-sensitive.

Note that if you rename the configuration file, you will also need to edit the context path setting within it, as described below.

If you wish for your LabKey Server application to run at the server root, you can rename labkey.xml to ROOT.xml. In this case, you should set the context path to be "/". You would then access your LabKey Server application with an address like http://<servername>/.

Securing the LabKey Configuration File

Important: The LabKey configuration file contains user name and password information for your database server, mail server, and network share. For this reason you should secure this file within the file system, so that only designated network administrators can view or change this file.

Modifying Configuration File Settings

You can edit the configuration file with your favorite text or XML editor. You will need to modify the LabKey Server configuration file if you are manually installing or upgrading LabKey Server, or if you want to change any of the following settings. Note that in our template, the placeholder values for these settings are marked with "@@", i.e "@@jdbcPassword@@". Replace that placeholder with the correct password, omitting the @@.

  • The appdocbase attribute, which indicates the location of the web application in the file system
  • Database settings, including server type, server location, username, and password for the database superuser.
  • SMTP settings, for specifying the mail server LabKey Server should use to send email to users.
  • Encryption Key - Configure an encryption key for the encrypted property set.
Note: Many other LabKey Server settings can be set in the Admin Console of the web application. For more information, see Site Settings.

The path Attribute

The path attribute of the Context tag specifies the context path for the application URL. The context path identifies this application as a unique application running on Tomcat. The context path is the portion of the URL that follows the server name and port number. By default, the context path is set to "labkey".

Note that the name of the configuration file must match the name of the context path, including case, so if you change the context path, you must also change the name of the file.

The appDocBase Attribute

The appDocBase attribute of the Context tag must be set to point to the directory where you have extracted or copied the labkeywebapp directory. For example, if the directory where you've copied labkeywebapp is C:\Program Files\LabKey Server on a Windows machine, you would change the initial value to "C:\Program Files\LabKey Server\labkeywebapp".

Database Settings

The username and password attributes must be set to a user name and password with admin rights on your database server. The user name and password that you provide here can be the ones that you specified during the PostgreSQL installation process for the database superuser. Both the name and password attribute are found in the Resource tag named "jdbc/labkeyDataSource". If you are running a local version of PostgreSQL as your database server, you don't need to make any other changes to the database settings in labkey.xml, since PostgreSQL is the default database choice.

The following template resource tag for PostgreSQL applies to Tomcat 8 and above:

<Resource name="jdbc/labkeyDataSource" auth="Container"
type="javax.sql.DataSource"
username="postgres"
password="sa"
driverClassName="org.postgresql.Driver"
url="jdbc:postgresql://localhost:5433/labkey162"
maxTotal="20"
maxIdle="10"
maxWaitMillis="120000"
accessToUnderlyingConnectionAllowed="true"
validationQuery="SELECT 1"
/>

If you are running LabKey Server against Microsoft SQL Server, you should comment out the Resource tag that specifies the PostgreSQL configuration, and add a Resource tag for the Microsoft SQL Server configuration. A template Resource tag for MS SQL Server is available at: Install Microsoft SQL Server.

Notes:

LabKey Server does not use Windows authentication to connect to Microsoft SQL Server; you must configure Microsoft SQL Server to accept SQL Server authentication.

If you are running LabKey Server against a remote installation of a database server, you will also need to change the url attribute to point to the remote server; by default it refers to localhost.

The maxWaitMillis parameter is provided to prevent server deadlocks. Waiting threads will time out when no connections are available rather than hang the server indefinitely.

GUID Settings

By default, LabKey Servers periodically communicate back to LabKey developers whenever the server has experienced an exception. LabKey rolls up this data and groups it by the GUID of each server. You can override the Server GUID stored in the database with the one specified in labkey.xml. This ensures that the exception reports sent to LabKey Server developers are accurately attributed to the server (staging vs. production) that produced the errors, allowing swift delivery of fixes. For details, see Tips for Configuring a Staging Server.

SMTP Settings (Optional, Recommended)

LabKey Server uses an SMTP mail server to send messages from the system, including email to new users when they are given accounts on LabKey. Configuring LabKey Server to connect to the SMTP server is optional; if you don't provide a valid SMTP server, LabKey Server will function normally, except it will not be able to send mail to users.

At installation, you will be prompted to specify an SMTP host, port number, user name, and password, and an address from which automated emails are sent. Note that if you are running Windows and you don't have an SMTP server available, you can set one up on your local computer.

The SMTP settings are found in the Resource tag named "mail/Session".

  • mail.smtp.host Set to the name of your organization's SMTP mail server.
  • mail.smtp.user Specifies the user account to use to log onto the SMTP server.
  • mail.smtp.port Set to the SMTP port reserved by your mail server; the standard mail port is 25. SMTP servers accepting a secure connection may use port 465 instead.
SMTP Authentication and Secure Connections: Many LabKey installations run an SMTP server on the same machine as the LabKey web server, which is configured for anonymous access from the local host only. Since only local applications can send mail, this ensures some amount of security without the hassle of using a central, authenticated mail server. If you choose instead to use an external authenticated server, you'll need to add the following:
  • mail.smtp.from This is the full email-address that you are would like to send the mail from. It can be the same as mail.smtp.user, but it doesn't need to be.
  • mail.smtp.password This is the password.
  • mail.smtp.starttls.enable When set to "true", configures the connection to use Transport Level Security (TLS).
  • mail.smtp.socketFactory.class When set to "javax.net.ssl.SSLSocketFactory", configures the connection to use an implementation that supports SSL.
  • mail.smtp.auth= When set to "true", forces the connection to attempt to authenticate using the user/password credentials.
When LabKey Server sends administrative emails, as when new users are added or a user's password is reset, the email is sent with the address of the logged-in user who made the administrative change in the From header. The system also sends emails from the Issue Tracker and Announcements modules, and these you can configure using the mail.from attribute so that the sender is an aliased address. The mail.from attribute should be set to the email address from which you want these emails to appear to the user; this value does not need to correspond to an existing user account. For example, you could set this value to "labkey@mylab.org".

Notes:

  • If you do not configure an SMTP server for LabKey Server to use to send system emails, you can still add users to the site, but they won't receive an email from the system. You'll see an error indicating that the email could not be sent that includes a link to an HTML version of the email that the system attempted to send. You can copy and send this text to the user directly if you would like them to be able to log into the system.
  • If you are running on Windows XP or a later version of Windows and you don't have a mail server available, you can configure the SMTP service. This service is included with Internet Information Server to act as your local SMTP server. Follow these steps:
    • From the Start menu, navigate to Control Panel | Add or Remove Programs, and click the Add/Remove Windows Components button on the left toolbar.
    • Install Internet Information Services (IIS).
    • From Start | Programs | Administrative Tools, open the Windows Services utility, select World Wide Web Publishing (the name for the IIS service), display the properties for the service, stop the service if it is running, and set it to start manually.
    • From Start | Programs | Administrative Tools, open the Internet Information Services utility.
    • Navigate to the Default SMTP Virtual Server on the local computer and display its properties.
    • Navigate to the Access tab, click Relay, and add the address for the local machine (127.0.0.1) to the list of computers which may relay through the virtual server.

Master Encryption Key

LabKey Server deployments can be configured to authenticate and connect to external systems to retrieve data or initiate analyses. In these cases, LabKey must store credentials (user names and passwords) in the primary LabKey database. While your database should be accessible only to authorized users, as an additional precaution, LabKey encrypts these credentials before storing them and decrypts them just before use. This encryption/decryption process uses a "master encryption key" that administrations set in labkey.xml; LabKey will refuse to save credentials if an encryption key is not configured.

Specify a master encryption key as follows:

<Parameter name="MasterEncryptionKey" value="@@masterEncryptionKey@@" />

Replace @@masterEncryptionKey@@ with a randomly generated, strong password, for example, a string of 32 random ASCII characters or 64 random hexadecimal digits. Once a key is specified and used, the key can't be changed, otherwise the existing values will be irretrievable. Different LabKey Server deployments should use different encryption keys, however, servers that use copies of the same database (for example, most test, staging, and production server combinations) need to use the same encryption key.

LDAP Synchronization

If you wish to synchronize with the user and groups on an LDAP server, add an appropriate <Resource> tag to the labkey.xml file.

An example configuration to a public LDAP test service.

<Resource name="ldap/ConfigFactory" auth="Container"
type="org.labkey.premium.ldap.LdapConnectionConfigFactory"
factory="org.labkey.premium.ldap.LdapConnectionConfigFactory"
host="ldap.forumsys.com"
port="389"
principal="cn=read-only-admin,dc=example,dc=com"
credentials="password"
/>

Related Topics

Next Step in Manual Installation




Third-Party Components and Licenses (Optional Install)


These components are often used in conjunction with LabKey Server to enhance its functionality. All of the components are optional and are not required for the server to work in general, but some of these components are required for specfic use cases.

R

Graphviz (All)

Trans Proteomic Pipeline (MS2)

  • Information on TPP: http://tools.proteomecenter.org/wiki/index.php?title=Software:TPP
  • Install Instructions:
    • Download the source distribution of v4.6.3 of the tools and unzip or obtain directly from Subversion at https://svn.code.sf.net/p/sashimi/code/tags/release_4-6-3.
    • Build the tools
      • Edit trans_proteomic_pipeline/src/Makefile.incl
      • Add a line with XML_ONLY=1
      • Modify TPP_ROOT to point to the location you intend to install the binaries. NOTE: This location must be on your server path (ie the path of the user running the Tomcat server process).
    • Run 'make configure all install' from trans_proteomic_pipeline/src
    • Copy the binaries to the directory you specified in TPP_ROOT above.
  • License: LGPL
  • Notes:
    • For OSX, this software is only supported for machines running on Intel CPUs.

X!Tandem (MS2)

  • Component Name: X!Tandem
  • Information: As of version 9.3, LabKey Server uses a version of X!Tandem that is included with the TPP.

Proteowizard (MS2)




Creating & Installing SSL/TLS Certificates on Tomcat


This topic describes how to create and install an SSL/TLS certificate on a Tomcat server. First we cover the process for creating a self-signed certificate and then an actual signed certificate from a Certificate Authority (CA).

Tomcat uses a Java KeyStore (JKS) repository to hold all of the security certificates and their corresponding private keys. This requires the use of the keytool utility that comes with the Java Development Kit (JDK) or the Java Runtime Environment (JRE). We highly recommend using OpenJDK 11 (or later) for better security of self-signed certificates. Review this topic for current version recommendations: Supported Technologies

Two things to understand:
  • The alias is simply a "label" used by Java to identify a specific certificate in the keystore (a keystore can hold multiple certificates). It has nothing to do with the server name, or the domain name of the Tomcat service. A lot of examples show "tomcat" as the alias when creating the keystore, but it really doesn’t matter what you call it. Just remember when you do use it, you stick with it.
  • The common name (CN) is an attribute of the SSL/TLS certificate. Your browser will usually complain if the CN of the certificate and the domain in the URI do not match (but since you’re using a self-signed certificate, your browser will probably complain anyway). HOWEVER, when generating the certificate, the keytool will ask for "your first and last name" when asking for the CN, so keep that in mind. The rest of the attributes are not really that important.

Create a Self-Signed Certificate

Why create a self-signed certificate?

  • It allows you to learn to create a keystore and certificate, which is good practice for getting an actual SSL/TLS certificate provided by a Certificate Authority.
  • It allows you to use a certificate right away and make sure it works successfully.
  • It's free.
How to get started creating your self-signed certificate:

Step 1. Locate the keytool application within your OpenJDK installation. Confirm that your <JAVA_HOME> environment variable points to the current supported version of OpenJDK and not to another JDK or JRE you may have installed previously.

The keytool will be in the bin directory. For example (where ## is the specific version number):

/usr/local/labkey/apps/Java/openjdk-##.jdk/Contents/Home/bin/keytool

Step 2. Create the keystore and the certificate. When you use the following commands, be sure to change <your password> to a password of your choosing and add a <keystore location> and <certificate location> that you will remember, likely the same location.

Use the following syntax to build your keystore and your self-signed certificate. Some examples follow.

  • The path to the keytool
  • The -genkeypair flag to indicate you are creating a key pair
  • The -exportcert flag to generate the certificate
  • The -alias flag and the alias you want to use for the keystore
  • The -keyalg flag and the algorithm type you want to use for the keystore
  • The -keysize flag and the value of the certificate encryption size
  • The -validity flag and the number of days for which you want the certificate to be valid
  • The -keystore flag and the path to where you want your keystore located
  • The -file flag and the path where you want the certificate located
  • The -storepass flag and your password
  • The -keypass flag and your password
  • The -ext flag to generate the SAN entry that is required by some modern browsers
For example:
$JAVA_HOME/bin/keytool -genkeypair -alias my_selfsigned -keyalg RSA -keysize 4096 -validity 720 -keystore <keystore location>/my_selfsigned.p12 -storepass <your password> -keypass <your password> -ext SAN=dns:localhost,ip:127.0.0.1

$JAVA_HOME/bin/keytool -exportcert -alias my_selfsigned -file <certificate location>/my_selfsigned.cer -keystore <keystore location>/my_selfsigned.p12 -storepass <your password>

Step 3. The string will then create a series of prompts, asking for you to supply a password for the keystore and create what is known as a Certificate Server Request (CSR):

  • The keystore password and to confirm the password.
  • The domain you want your SSL/TLS certificate made for.
NOTE: This prompt will literally ask the question “What is your first and last name? [Unknown]:” This is NOT your first and last name. This should be the domain you want to make the SSL/TLS certificate for. Since this section is about creating a self-signed certificate, please enter localhost as the domain and press Enter.
  • The name of your Organizational Unit. This is optional, but most will put down the name of the department the certificate is being used for or the department the certificate is being requested by.
  • The name of your Organization. This is also optional, but most will put in the name of their company.
  • The name of your city. This is also optional, but most will put in the city the company or department is located in.
  • The name of your state/province. This is also optional, but if you do choose to enter in this information DO NOT abbreviate the information. If you choose to enter data here, you must spell out the state/province. For example: California is acceptable, but not CA.
  • The two-letter country code. This is optional, but if you’ve already entered the rest above, you should enter the two-letter code. For example: US for United States, UK for the United Kingdom.
  • Confirmation that the information you entered is correct.
  • A final prompt to enter the certificate, but just press Enter to use the same one as the Keystore from earlier.
The steps above will create the new keystore and add the new self-signed certificate to the keystore.

Step 4. Configure the server.xml file to use the new keystore & self-signed certificate.

Now that the keystore and certificate are ready, you now will need to configure the server.xml file to use them.

  • Access the server.xml located in the Tomcat directory, under the conf directory.
  • Activate the HTTPS-Connector listed in the server.xml file. It should look something like the example below. Note: Be sure to change the certificateKeyAlias, certificateKeystoreFile, and certificateKeystorePassword to the ones you used earlier (especially the password).
<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="200" SSLEnabled="true" scheme="https" secure="true"
>
<SSLHostConfig
sslProtocol="TLSv1.2" protocols="TLSv1.2"
ciphers="TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384,
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256,
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384,
TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384,TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384,
TLS_ECDH_RSA_WITH_AES_256_CBC_SHA,TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256,
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA"

honorCipherOrder="true"
>
<Certificate
certificateKeyAlias="my_selfsigned"
certificateKeystoreFile="$/labkey/apps/lib/my_selfsigned.p12"
certificateKeystorePassword="password_here"
certificateKeystoreType="PKCS12"
type="RSA"/>
</SSLHostConfig>
</Connector>

Step 5. (Optional) You can create a working certificate without this step, but for some uses (like running LabKey automated tests in HTTPS) you'll need to add the certificate manually to your OS and to Java so that they know to trust it. To add the certificate to the Trusted Root Certificate Authorities and Trusted Publishers, follow the instructions for your operating system below.

  • If you're trying to run LabKey's automated tests, change, in the test.properties file, labkey.port to "8443" and labkey.server to "https://localhost".
For OSX:
  • Find your my_selfsigned.cer file from step #2, go to Applications > Utilities > Keychain Access, and drag the file there.
  • Now we need to make this certificate trusted. Go to the Certificates line in the Category section in the lower-left corner of this window, find the "localhost" entry, and double-click it. Then expand the Trust section, and change "When using this certificate" to "Always Trust". Click the close button in the upper left and type your password to finish this operation.
  • Import the same my_selfsigned.cer file into your Java cacerts (which assumes you have not changed your Java's default keystore password of "changeit") by executing this command on the command line:
    $JAVA_HOME/bin/keytool -importcert -cacerts -alias my_selfsigned -file <certificate location>/my_selfsigned.cer -storepass changeit
For Windows:
  • Go to Windows command line and run the command, certmgr. This will open certificate manager for current user. Perform the following in certificate manager.
    • Right-click on Personal/Certificates. Select All Tasks > Import. This should open Certificate Import Wizard.
    • Current User should be selected. Click Next.
    • Enter the path to the certificate you created, <certificate location>/my_selfsigned.cer. Click Next.
    • Select "Place all certificates in the following store". Certificate store should be "Personal".
    • Review and click Finish.
    • You should now see your certificate under Personal/Certificates.
    • Right-click and Copy the certificate.
    • Paste the certificate into Trusted Root Certification Authorities/Certificates.
    • Paste the certificate into Trusted Publishers/Certificates.
    • Close certmgr.
    • Before running LabKey automated tests, edit the test.properties file: change labkey.port to "8443" and labkey.server to "https://localhost".
  • Execute the following to add your certificate to Java cacerts:
    "%JAVA_HOME%/bin/keytool" -importcert -cacerts -alias my_selfsigned -file <certificate location>/my_selfsigned.cer -storepass changeit
Windows may not pick up the certificate right away. A reboot should pick up the new certificate.

Step 6: Restart Tomcat and try to connect to your local server as https://localhost:8443/labkey and see if you are able to connect to it via HTTPS. If you did everything correctly, you should see a grey padlock to the left of the URL, and not red "https" with a line through it.

Configure LabKey Server to use HTTPS

Be sure to change the LabKey Server settings in your admin console to support HTTPS better. Go to (Admin) > Site > Admin Console > Admin Console Links > Configuration > Site Settings, and change the following values (which assume localhost as your server and port 8443):

In particular, you'll want to do that second item ("Require SSL connections"), because you will no longer be able to sign in to LabKey properly via HTTP.

Create a Real Certificate

Once you’ve successfully created your own self-signed certificate, the steps to requesting and adding an actual certificate will be significantly easier.

To create your actual certificate, do the following:

1. Create the new keystore

  • Repeat step 2 in the self-signed section, but this time enter the domain you want the SSL/TLS certificate to be assigned to. For example, if your domain was “mylab.sciencelab.com”, you can run the following:
/usr/local/Java/bin/keytool 
-genkey
-keystore /labkey/apps/lib/mylab.sciencelab.com.jks
-alias tomcat
-keyalg RSA
-keysize 4096

This would create a new dedicated keystore for your new domain. You can opt to use the existing keystore you created, but I prefer to keep that self-signed one separate from the legitimate one.

  • Go through the same steps as step 3 in the self-signed section, enter in your actual domain name when prompted for “What is your first and last name? [Unknown]:”. Everything else stays the same as before.
2. Create the Certificate Signing Request (CSR): Now that the keystore is made, you can now make the CSR that you will then send to the certificate provider (such as GoDaddy.com, Comodo.com, or SSLShopper.com)

Run the following command:

/usr/local/Java/bin/keytool 
-certreq
-alias tomcat
-keyalg RSA
-keysize 4096
-keystore /labkey/apps/lib/mylab.sciencelab.com.jks
-file /labkey/apps/lib/mylab.sciencelab.com.csr

(Note: You can technically give the CSR any name, but I prefer to use the name of the domain and the extension .csr to keep things orderly)

When you run through the CSR, you’ll be prompted similarly like you were when you created the keystore. Enter in all the same respective values.

Once you are finished, open the .csr file using your favorite plain-text editor. You will see a long hash contained within two lines. This is the CSR that you will provide to the certificate provider to order your SSL/TLS certificate.

3. Applying your SSL/TLS certificate:

Once you receive the SSL/TLS certificate from your certificate provider, they may provide you with a few certificates, either 2 certificates (a Root certificate and the certificate for your domain) or 3 certificates (a Root certificate, an intermediate certificate, and the certificate for your domain). Sometimes, you may just get one certificate that has all of those certificates combined. Your certificate provider will provide you with an explanation on what they issued you and instructions on how to use them as well if you are in doubt.

Place the certificate(s) you’ve received in the same directory as the keystore.

If you are provided with a root and/or an intermediate certificate, run the following command:

/usr/local/Java/bin/keytool 
-import
-alias root
-keystore /labkey/apps/lib/mylab.sciencelab.com.jks
-trustcacerts
-file /labkey/apps/lib/certificate_file_name.crt

Take note that the alias is “root” and not the alias you’re using from before. This is intentional. Do not use the alias you used for the CSR or the keystore for this.

Otherwise, if you only received a single certificate, run the following:

/usr/local/Java/bin/keytool 
-import
-alias tomcat
-keystore /labkey/apps/lib/mylab.sciencelab.com.jks
-file /labkey/apps/lib/certificate_file_name.crt

Once the certificate is loaded into the keystore, the final step is to reconfigure the SSL/TLS connector in the server.xml file.

4. Update your server.xml with the new SSL/TLS certificate info:

Repeat step 4 from the self-signed section. Update the connector to point to your new keystore. For example:

<Connector port="443" protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS"
keyAlias="tomcat" keystoreFile="/labkey/apps/lib/mylab.sciencelab.com.jks"
keystorePass="THEPASSWORD" />

Once this is done, restart Tomcat for the changes to take effect.

Note: We recommend using the server.xml examples we provider here within our GitHub repository: https://github.com/LabKey/samples/tree/master/ops/config-examples

Create Special Wildcard and Subject Alternative Names (SAN) Certificates

  • Sometimes you may need to create a certificate that covers multiple domains. There are two types of additional certificates that can be created:
    • Wildcard certificates that would cover any subdomain under the main one.
    • Subject Alternative Name certificates (SAN) that would cover multiple domains.
  • To create a wildcard certificate, you would simply use an asterisk in lieu of a subdomain when creating your keystore and CSR. So the example of mylabs.sciencelab.com, you would use *.sciencelab.com instead and then when requesting your certificate from the provider, you would specifically indicate that you want a wildcard certificate.
  • To create a SAN certificate, you would insert the additional domains and IPs you wish the certificate to apply to when you run the keytool command.
For example:

/usr/local/Java/bin/keytool 
-genkey
-alias tomcat
-keyalg RSA
-keysize 4096
-keystore /labkey/apps/lib/mylab.sciencelab.com.jks
-file /labkey/apps/lib/mylab.sciencelab.com.csr
-ext SAN=dns:mylab.sciencelab.com,dns:mylab.ultimatescience.net,dns:mylab.researcherscience.org,ip:33.33.33.33

Related Topics




Example Hardware/Software Configurations


This topic shows example hardware/software configurations for different LabKey Server installations. These are intended as guidelines only -- your own configuration should be adjusted to suit your particular requirements.

Small Laboratory Installation

The following configuration is appropriate for 10-20 users with small file and table sizes. We assume that the server and database are located on the same machine.

CPUs2+ CPUs or Virtual CPUs
RAM4GB (minimum) - 16GB (recommended)
Disk Storage164GB (64GB for File Storage, 100GB for database storage)
Software• OS: Linux or Windows (LabKey Server is supported on both Operating Systems. Select the operating system which is best supported by your organization)
• Java
• Tomcat
• DB software (PostgreSQL or MS SQL Server)
• See Supported Technologies for the specific versions to use.

As usage increases, increase the amount of RAM memory to 8GB at a minimum (and also increase the memory used by Tomcat and the database accordingly).

Large Multi-project Installation

The following configuration is appropriate for a 100s of users working on multiple projects with large files and data tables.

We recommend placing the web server and the database server on different machines in order to optimize maintenance, update, and backup cadences.

Machine #1: Web Server

CPUs4+ CPUs or Virtual CPUs
RAM8GB (minimum) - 16GB (recommended)
Disk Storage64GB (for OS and LabKey binaries), 512GB (for file storage)
Software• OS: Linux or Windows (LabKey Server is supported on both Operating Systems. Select the operating system which is best supported by your organization)
• Java
• Tomcat
• See Supported Technologies for the specific versions to use.
Network1 GB/s

Machine #2: Database Server

CPUs4+ CPUs or Virtual CPUs
RAM8GB (minimum) - 16GB (recommended)
Disk Storage100GB (for database storage)
Software• OS: Linux or Windows (LabKey Server is supported on both Operating Systems. Select the operating system which is best supported by your organization)
• DB software (PostgreSQL or MS SQL Server)
• See Supported Technologies for the specific versions to use.
Network1 GB/s

Related Topics




Common Install Tasks





Install Microsoft SQL Server


As part of installing required components, you need to install a database server. You can use either PostgreSQL or Microsoft SQL Server as the primary LabKey database. This topic covers the process of setting up a Windows machine to use Microsoft SQL Server as the primary database. Information about other environments is provided below.

Install and Configure Microsoft SQL Server

For current supported versions of Microsoft SQL Server, see Supported Technologies.

Basic Installation Topics:

Additional Topics:

Install Microsoft SQL Server

If you already have a licensed version of Microsoft SQL Server in a supported version, follow the installation instructions noting the requirements outlined for installing an express edition below.

If you don't have a licensed version of Microsoft SQL Server, you can download a free Express Edition. Note that the Express Edition has database size limitations that generally make it inappropriate for production deployments. You can also use the Developer Edition; details may differ slightly from instructions in this topic.

Configure During Installation

During installation, configure Microsoft SQL Server to accept both Windows Authentication and SQL Server Authentication, ("Mixed Mode"), and specify a user name and password for the administrative account.

To enable this feature on SQL Server 2017 Express, select the Custom installation option and proceed through the wizard accepting defaults until the Database Engine Configuration step.

Keep track of the user name and password; LabKey Server uses it to authenticate to SQL Server. It must be provided in plaintext in labkey.xml or in your mssql.properties file later.

If you've already installed SQL Server without enabling SQL Server Authentication then see How to: Change Server Authentication Mode in the Microsoft SQL Server documentation.

Configure TCP/IP

After you've installed SQL Server, you'll need to configure it to use TCP/IP. Follow these steps:

  • Launch the SQL Server Configuration Manager.
  • Under the SQL Server Network Configuration node, select Protocols for <servername>.
  • In the right pane, right-click on TCP/IP and choose Enable.
  • Right-click on TCP/IP and choose Properties.
  • Switch to the IP Addresses tab.
  • Scroll down to the IPAll section, clear the value next to TCP Dynamic Ports and set the value for TCP Port to 1433 and click OK. By default, SQL Server will choose a random port number each time it starts, but the JDBC driver expects SQL Server to be listening on port 1433.
  • Click OK
  • Restart the service by selecting the SQL Server Services node in the left pane, selecting SQL Server <edition name> in the right pane, and choosing Restart from the Action menu (or use the Restart button on the toolbar).

SQL Server Management Studio

Download the SQL Server Management Studio graphical database management tool.

Set Up a Login

You may want to set up a new login (in addition to the "sa" system administrator) for LabKey Server to use to connect to SQL Server:

  • Run SQL Server Management Studio.
  • Connect to the database.
  • Under Security > Logins, add a new login, using SQL Server authentication.
  • Enter the user name and password.
  • Use this password to configure the data source below.

LabKey Configuration File

Next edit the LabKey Server configuration file, by default named labkey.xml, to configure the jTDS JDBC driver for Microsoft SQL Server, which is included in the LabKey Server installation. You should comment out the Resource tag that specifies the PostgreSQL configuration. This Resource tag can be identified by the driverClassName "org.postgresql.Driver". After commenting out this resource tag, it should look similar to the following:

Comment out PostgreSQL Resource Tag

<!--
<Resource name="jdbc/labkeyDataSource" auth="Container"
type="javax.sql.DataSource"
username="postgres"
password="sa"
driverClassName="org.postgresql.Driver"
url="jdbc:postgresql://localhost:5433/labkey162"
maxTotal="20"
maxIdle="10"
maxWaitMillis="120000"
accessToUnderlyingConnectionAllowed="true"
validationQuery="SELECT 1"
/>
-->

Add MS SQL Server Resource Tag

Use the following template for configuring a MS SQL Server data source. Replace USERNAME, PASSWORD, and DATABASE_NAME to fit the particulars of your target data source. The following template applies to Tomcat 8 and above. See an earlier version of this topic for a Tomcat 7 template.

<Resource name="jdbc/labkeyDataSource" auth="Container"
type="javax.sql.DataSource"
username="USERNAME"
password="PASSWORD"
driverClassName="net.sourceforge.jtds.jdbc.Driver"
url="jdbc:jtds:sqlserver://localhost:1433/DATABASE_NAME"
maxTotal="20"
maxIdle="10"
maxWaitMillis="120000"
accessToUnderlyingConnectionAllowed="true"
validationQuery="SELECT 1"/>

Documentation for this driver is available on SourceForge. Other JDBC drivers for Microsoft SQL Server have not been tested.

Note: The maxWaitMillis parameter is provided to prevent server deadlocks. Waiting threads will time out when no connections are available rather than hang the server indefinitely.

Install CLR Functions (GROUP_CONCAT)

When setting up SQL Server, you may need to separately install the GROUP_CONCAT Common Language Runtime (CLR) function. For details see GROUP_CONCAT Install.

For Premium Editions, you may also need to install the PremiumStats CLR functions separately. For details see PremiumStats Install.

Premium EHR users may also need to install LDKNaturalize, following similar methods.

SQL Server Synonyms

LabKey Server supports the use of SQL Server Synonyms. These alternative names function like shortcuts or symlinks, allowing you to "mount" tables and views which actually exist in another schema or database. For more information, see SQL Synonyms.

Installation on Other Platforms

VirtualBox Deployment

On all operating systems, you can set up a VirtualBox VM running Windows and install SQL Server there. These Mac-specific instructions, combined with the documentation above should get you running.

Linux Deployment

Starting with SQL Server 2017, Microsoft has started distributing a native Linux version:

Docker Deployment

The Linux edition of SQL Server is also available within a Docker container. This will work on Mac, Linux, or Windows.
You will need to install a separate visual database management tool:

Related Topics




Install PostgreSQL (Windows)


Install PostgreSQL on Windows

To install PostgreSQL on Windows:

  1. Download and run the Windows PostgreSQL one click installer. When the wizard prompts you to choose where to install PostgreSQL, point it to the apps subdirectory of your <LABKEY_ROOT>, i.e. C:\labkey\apps\postgresql-10.6\
  2. Keep track of the PostgreSQL Windows Service account name and password. LabKey Server needs to ask for it so that we can pass it along to the PostgreSQL installer.
  3. Keep track of the database superuser name and password. You'll need these to initially create the LabKey database, the LabKey database user, and grant that user the owner role.
  4. We recommend that you install the graphical tool pgAdmin 4.x for easy database administration. Leave the default settings as they are on the "Installation Options" page to include pgAdmin.
  5. Using superuser permissions:
    • Create an empty database.
    • Create a PostgreSQL user named 'labkey'.
    • Grant the owner role to the labkey user over the database.
    • Revoke public permissions from the database.
See the following example PostgreSQL commands:

Connect to the DB server as the Postgres Super User using the psql command:

psql -h <host_Name> -U <Super_username> -d postgres

Issue the following SQL commands to create the user, database and revoke public permissions from the database

create user labkey password 'PASSWORD_HERE' ;
create database "<labkey_db_name>" with owner labkey;
revoke all on database "<labkey_db_name>" from public;
\q

Please read the notes below to forestall any difficulties with the PostgreSQL installation.

Notes

Related Topics




Install PostgreSQL (Linux or OSX)


As part of installing required components, you need to install a database. To install PostgreSQL on Linux or a Mac:
  1. See Supported Technologies page for list of supported versions of PostgreSQL
  2. From http://www.postgresql.org/download/ download the PostgreSQL binary packages or source code. Follow the instructions in the downloaded package to install PostgreSQL.
  3. Please read the notes below to forestall any difficulties with the PostgreSQL installation.
Notes:
  • If you uninstall and reinstall PostgreSQL, you may need to manually delete the data directory in order to reinstall.
  • Don't forget to set the password for your database user, typically 'postgres'. (You can do this by opening Psql with something like ALTER USER "postgres" WITH PASSWORD 'new_password';) It should match whatever is in your config.properties file in the LabKey server directory.
  • See: Notes on Installing PostgreSQL on All Platforms
  • After installation, you may find it helpful to tune PostgreSQL for performance.



Notes on Installing PostgreSQL on All Platforms


Notes on Installing PostgreSQL on All Platforms

If you are using PostgreSQL, you will need to increase the join collapse limit. Edit postgresql.conf and change the following line:

# join_collapse_limit = 8

to

join_collapse_limit = 10

If you do not do this step, you may see the following error when running complex queries:

org.postgresql.util.PSQLException: ERROR: failed to build any 8-way joins



Install and Set Up R


Set Up Steps: Additional Steps for Linux/OSX: Once an administrator has set up the R environment, your users can create R Reports.

Install R

  • On the R site, choose a CRAN mirror near you.
  • Click Download R for the OS you are using (Linux, OSX, or Windows).
  • Click the subcategory base.
  • Download the installer using the link provided for your platform, for example Download R #.#.# for Windows.
  • Install using the downloaded file.

Tips:

  • You don’t need to download the “contrib” folder on the Install site. It’s easy to obtain additional R packages individually from within R.
  • Details of R installation/admin can be found here.
OS-Specific Instructions:
  • Linux. An example of installing R on Linux is included on the Configure R on Linux page.
  • Windows. On Windows, install R in a directory whose path does not include a space character. The R FAQ warns to avoid spaces if you are building packages from sources.

Configure LabKey Server to Work with R

Configure Authentication and Permissions

Authentication. If you wish to modify a password-protected LabKey Server database through the Rlabkey macros, you will need to set up authentication. See: Create a netrc file.

Permissions. Refer to Configure Permissions for information on how to adjust the permissions necessary to create and edit R Views. Note that only users who are part of the "Developers" site group or have Site Admin permissions can edit R Views.

Batch Mode. Scripts are executed in batch mode, so a new instance of R is started up each time a script is executed. The instance of R is run using the same privileges as the LabKey Server, so care must be taken to ensure that security settings (see above) are set accordingly. Packages must be re-loaded at the start of every script because each script is run in a new instance of R.

Install & Load Additional R Packages

You will likely need additional packages to flesh out functionality that basic install does not include. Additional details on CRAN packages are available here. Packages only need to be installed once on your LabKey Server. However, they will need to be loaded at the start of every script when running in batch mode.

How to Install R Packages

Use the R command line or a script (including a LabKey R script) to install packages. For example, use the following to install two useful packages, "GDD" and "Cairo":

install.packages(c("GDD", "Cairo"), repos="http://cran.r-project.org" )

To install knitr:

install.packages('knitr', dependencies=TRUE)

You can also use the R GUI (Packages > Install Packages) to select and install packages.

How to Load

Each package needs to be installed AND loaded. If the installed package is not set up as part of your native R environment (check ‘R_HOME/site-library’), it needs to be loaded every time you start an R session. Typically, when running R from the LabKey interface, you will need to load (but not install) packages at the start of every script because each script is run in a new instance of R.

To load an installed package (e.g., Cairo), call:

library(Cairo)

Recommended Packages

GDD &/or Cairo: If R runs on a headless Linux server, you will likely need at least one extra graphics package. When LabKey R runs on a headless Linux server, it may not have access to the X11 device drivers (and thus fonts) required by the basic graphics functions jpeg() and png(). Installing the Cairo and/or GDD packages will allow your users to output .jpeg and .png formats without using the jpeg() and png() functions. More details on these packages are provided on the Determine Available Graphing Functions page.

You can avoid the use of Cairo and/or GDD by installing a display buffer for your headless server (see below for more info).

Lattice: Optional. This package is the commonly used, sophisticated graphing package for R. It is particularly useful for creating Participant Charts.

Headless Linux Servers Only: Install the X Virtual Frame Buffer

On Linux servers, the png() and jpg() functions use the device drivers provided by the X-windows display system to do rendering. This is a problem on a headless server where there is generally no display running at all.

As a workaround, you can install the X Virtual Frame Buffer. This allows applications to connect to an X Windows server that renders to memory rather than a display.

For instructions on how to install and configure the X Virtual Frame Buffer on Linux, see Configure the Virtual Frame Buffer on Linux.

If you do not install the X Virtual Frame Buffer, your users may need to use graphics packages such as GDD or Cairo to replace the png() and jpeg() functions. See Determine Available Graphing Functions for further details.

OSX Only: Fix Hostname Resolution Problems

When a LabKey Server runs on OSX, R views can only resolve hostnames that are stored in the server's host file.

It appears that OSX security policy blocks spawned processes from accessing the name service daemon for the operating system.

You can use one of two work-arounds for this problem:

  1. Add the hostname (e.g., www.labkey.org) to the hosts file (/etc/hosts). Testing has shown that the spawned process is able to resolve hostnames that are in the host file.
  2. Use the name "localhost" instead of DNS name in the R script.

Related Topics




Configure an R Docker Engine


Premium Feature — This feature is part of the Professional Plus and Enterprise Editions. Learn more or contact LabKey for set up assistance.

As an alternative to installing a local instance of R, or using an Rserver server, LabKey Server can make use of an R engine inside a Docker image.

Multiple Docker-based R engines can be configured, each pointing to different Docker images, each of which may have different versions of R and packages installed.

Docker-based R engines are good candidates for "sandboxed" R engines.

To set up LabKey Server to use Docker-based R engines, complete the following steps:

  • Modify a Docker image to fit your needs.
  • Configure Docker and Startup the Image.
  • Configure LabKey Server to use the R engine inside the Docker image.
See details on each step below:

Install Docker

  • Install Docker appropriate your operating system.
    • On a Windows test/development machine, go to Docker Settings. Under General, place a checkmark next to Expose daemon on tcp://localhost:2375. Do not do this on a production server, as this is not a secure configuration. Only appropriate for local development and testing.

(On Development Machines) Set the Base server URL

  • Set the Base Server URL to something other than localhost:8080. For details, see Site Settings. Example value "http://my.server.com".

Make a Docker Image with R Included

LabKey provides two template images to start from. One template is for building a Dockerized R engine using the latest released version of R. The other template is for building an older version of R.

Startup the Docker Image

When the docker image build process finishes, look for the name of your image in the console, for example:

Successfully tagged labkey/rsandbox-base:3.5.1

Start the image:

docker run labkey/rsandbox-base:3.5.1

Configure Docker and Startup the Image

Configure LabKey Server to Use the Docker R Engine

  • In the Edit Engine Configuration dialog, enter the Docker Image Name (at a minimum).
  • The Mount Paths let you (1) map in-container paths to host machine paths and (2) to write script outputs the mapped path on the host machine. For example, the following settings will write the script.R and script.Rout generated files to your host file system at C:scriptsRreports:
Mount (rw): host directory: C:\scripts\Rreports
Mount (rw): container directory: /Users/someuser/cache

Mount (rw) paths refer to Read-Write directories, Mount (ro) refer to Read-Only paths.

  • The Site Default refers to the default sandboxed R engine. If you have only one sandboxed R engine, this cannot be unchecked. If you have more than one sandboxed R engine, you can choose one of them to be the Site Default.

Related Topics




Determine Available Graphing Functions


Determine Available Graphing Functions

Test Machine Capabilities. Before reading this section further, figure out whether you need to worry about its contents. Execute the following script in the R script builder:

if(!capabilities(what = "jpeg") || !capabilities(what="X11"))
warning("You cannot use the jpeg() function on your LabKey Server");
if(!capabilities(what = "png") || !capabilities(what="X11"))
warning("You cannot use the png() function on your LabKey Server");
If this script outputs both warnings, you’ll need to avoid both jpeg() and png() functions. If you do not receive warnings, you can ignore the rest of this section.

Why Don't png() and jpeg() Work? On Unix, jpeg() and png() rely on the x11() device drivers. These are unavailable when R is installed on a "headless" Unix server.

If png() and jpeg() Don't Work, What Are My Options?. You have two categories of options:

  1. Ask your admin to install a display buffer on the server such that it can access the appropriate device drivers.
  2. Avoid jpeg() and png(). There are currently three choices for doing so: Cairo(), GDD() and bitmap().
Which Graphics Function Should I Use? If you are working on a headless server without an installed display buffer, you will need to use Cairo(), GDD() or bitmap(). There are trade-offs for all options. If you use Cairo or GDD, your admin will need to install an additional graphics package. The Cairo package is based upon libraries undergoing continued development and maintenance, unlike the GDD package. Cairo does not require the use of Ghostscript to produce graphics, as does the bitmap() function. However, Cairo() fails to provide all graphics functions on all machines, so you will need to test its capabilities. GDD may provide functions unavailable in Cairo, depending on your machine setup.

Warning: LabKey R usually runs in batch mode, so any call to plot() must be preceded by a call to open the appropriate device (e.g., jpeg() or pdf()) for output. When R runs in its ordinary, interpreted/interactive mode, it opens an appropriate output device for graphics for you automatically. LabKey R does not do this, so you will need to open an output device for graphics yourself. Identifying appropriate devices and function calls is tricky and covered in this section.

Strategy #1: Use the Cairo and/or GDD Packages

You can use graphics functions from the GDD or Cairo packages instead of the typical jpeg() and png() functions.

There are trade-offs between GDD and Cairo. Cairo is being maintained, while GDD is not. GDD enables creation of .gif files, a feature unavailable in Cairo. You will want to check which image formats are supported under your installation of Cairo (this writer's Windows machine can not create .jpeg images in Cairo). Execute the following function call in the script-builder window to determine formats supported by Cairo on your machine:

Cairo.capabilities();
The syntax for using these packages is simple. Just identify the “type” of graphics output you desire when calling GDD or Cairo. The substitution parameters used for file variables are not unique to Cairo/GDD and are explained in subsequent sections.

#   Load the Cairo package, assuming your Admin has installed it:
library(Cairo);
# Identify which "types" of images Cairo can output on your machine:
Cairo.capabilities();
# Open a Cairo device to take your plotting output:
Cairo(file="${imgout:labkeyl_cairo.png}", type="png");
# Plot a LabKey L:
plot(c(rep(25,100), 26:75), c(1:100, rep(1, 50)), ylab= "L", xlab="LabKey",
xlim= c(0, 100), ylim=c(0, 100), main="LabKey in R");
dev.off();

# Load the GDD package, assuming your Admin has installed it:
library(GDD);
# Open a GDD device to take your plotting output:
GDD(file="${imgout:labkeyl_gdd.jpg}", type="jpeg");
# Plot a LabKey L:
plot(c(rep(25,100), 26:75), c(1:100, rep(1, 50)), ylab= "L", xlab="LabKey",
xlim= c(0, 100), ylim=c(0, 100), main="LabKey in R");
dev.off();

Strategy #2: Use bitmap()

It is possible to avoid using either GDD or Cairo for graphics by using bitmap(). Unfortunately, this strategy relies on Ghostscript, reportedly making it slower and lower fidelity than other options. Instructions for installing Ghostscript are available here.

NB: This method of creating jpegs has not been thoroughly tested.

Calls to bitmap will specify the type of graphics format to use:

bitmap(file="${imgout:p2.jpeg}", type = "jpeg");



Configure R on Linux


Steps

The following example shows how to install and configure R on a Linux machine.

> wget http://cran.r-project.org/src/base/R-2/R-2.6.2.tar.gz 
> tar xzf R-2.6.2.tar.gz
> cd R-2.6.2
> ./configure
...
> make
...
> make install
...

Additional Notes

  • These instructions install R under /usr/local (with the executable installed at /usr/local/bin/R
  • Support for the X11 device (including png() and jpeg()) is compiled in R by default.
  • In order to use the X11, png and jpeg devices, an Xdisplay must be available. Thus you may still need to Configure the Virtual Frame Buffer on Linux.

Related Topics




Install SAS/SHARE for Integration with LabKey Server


Overview

Publishing SAS datasets to your LabKey Server provides secure, dynamic access to datasets residing in a SAS repository. Published SAS data sets appear on the LabKey Server as directly accessible datasets. They are dynamic, meaning that LabKey treats the SAS repository as a live database; any modifications to the underlying data set in SAS are immediately viewable on LabKey. The data sets are visible only to those who are authorized to see them.

Authorized users view published data sets using the familiar, easy-to-use grid user interface used throughout LabKey. They can customize their views with filters, sorts, and column lists. They can use the data sets in custom queries and reports. They can export the data in Excel, web query, or TSV formats. They can access the data sets from JavaScript, SAS, R, and Java client libraries.

Several layers keep the data secure. SAS administrators expose selected SAS libraries to LabKey. LabKey administrators then selectively expose SAS libraries as schemas available within a specific folder. The folder is protected using standard LabKey security; only users who have been granted permission to that folder can view the published data sets.

SAS Setup

Before SAS datasets can be published to LabKey, an administrator needs to do three things:

  • Set up the SAS/SHARE service on the SAS installation
  • Set up the SAS/SHARE JDBC driver on the LabKey web server
  • Define SAS libraries as external schemas within LabKey
Set up the SAS/SHARE server. This server runs as part of the SAS installation (it does not run on the LabKey server itself). SAS/SHARE allows LabKey to retrieve SAS data sets over an internal corporate network. The SAS/SHARE server must be configured and maintained as part of the SAS installation. The LabKey installation must be able to connect to SAS/SHARE; it requires high-speed network connectivity and authentication credentials. SAS/SHARE must be configured to predefine all data set libraries that the LabKey installation needs to access.

Set up the SAS/SHARE JDBC driver. This driver allows LabKey to connect to SAS/SHARE and treat SAS data sets as if they were tables in a relational database. The SAS/SHARE JDBC driver must be installed on the LabKey installation. This requires copying two .jar files into the tomcat/lib directory on LabKey. It also requires adding a new DataSource entry in the labkey.xml file on LabKey containing several connection settings (e.g., SAS/SHARE URL and credentials). See External SAS Data Sources.

Define SAS libraries as external schemas within LabKey. A folder administrator chooses which SAS libraries to publish in a LabKey Server folder via the Schema Administration user interface. If a SAS data source is defined in the labkey.xml file, the "Data Source" drop-down list contains the name of this data source as an option. After selecting the data source, the administrator selects the schema (library) name to publish. After clicking the “Create” button, all data sets in that library are published; in other words, they can be viewed by anyone with read permissions in the folder.

For further instructions on how to choose an external schema, see Set Up an External Schema.

Usage

Once defined via the Schema Administration page, a SAS library can be treated like any other database schema (with a couple important exceptions listed below). The query schema browser lists all its data sets as “built-in tables.” A query web part can be added to the folder’s home page to display links to a library’s data sets. Links to key data sets can be added to wiki pages, posted on message boards, or published via email. Clicking any of these links displays the data set in the standard LabKey grid with filtering, sorting, exporting, paging, customizing views, etc. all enabled. Queries that operate on these datasets can be written. The data sets can be retrieved using client APIs (Java, JavaScript, R, and SAS).

Limitations

The two major limitations with SAS data sets are currently:
  • SAS data sets can be joined to each other but not joined to data in the LabKey database. Any attempt to join between different databases (e.g., a table in PostgreSQL with a data set in SAS) results in an error message.
  • The SAS/SHARE JDBC driver provides read-only access to SAS data sets. You cannot insert, update, or delete data in SAS data sets from LabKey.



Configure Tomcat Webapp Memory


This topic provides instructions for setting the Java Virtual Machine (JVM) memory configuration on Windows, Linux, and OSX operating systems. What do the JVM memory settings do? LabKey Server is Java web application that runs on Tomcat. The Tomcat server is run within a Java Virtual Machinex (JVM). This JVM that controls the amount of memory available to LabKey Server. LabKey recommends that the Tomcat web application be configured to have a maximum Java Heap size of 2GB for a test server, and 4GB for a production server.

Definitions

  • <CATALINA_HOME>: Installation location of the Apache Tomcat Web Server. If you are following our recommended folder configuration, the location will be (where #.#.## is the specific version installed):
    <LABKEY_ROOT>/apps/apache-tomcat-#.#.##
    • On Linux or OSX, you may also have created a symbolic link /usr/local/tomcat to this location.
  • Tomcat Version: Find the current recommended version of tomcat in this topic: Supported Technologies. This topic uses # to stand in for the version number.

Change the JVM Memory Configuration on Windows

Tomcat is usually started as a service on Windows, and it includes a dialog for configuring the JVM. The maximum total memory allocation is configured in its own text box, but other settings are configured in the general JVM options box using the Java command line parameter syntax.

If you changed the name of the LabKey Windows Service, you must use Method 2.

Method 1:

  • Open Windows Explorer.
  • Go to the <CATALINA_HOME>/bin directory.
  • Run tomcat#w.exe (replacing the # with the major version number). On some operating systems, you may have to run this as an administrator; if so, right-click the .exe file and select Run as administrator.
    • The command will open a program window.
    • If this produces an error that says "The specified service does not exist on the server", then please go to Method 2.
  • Go to the Java tab in the new window.
  • In the Java Options box, scroll to the bottom of the properties, and set the following property:
    -XX:-HeapDumpOnOutOfMemoryError
  • Change the Initial memory pool to 2GB for a test server or 4GB for a production server.
  • Change the Maximum memory pool to the same value.
  • Click OK
  • Restart LabKey Server.

Method 2:

You will need to use this method if you customized the name of the LabKey Windows Service.
  • Open a Command Prompt. (Click Start, type "cmd", and press the Enter key.)
  • Navigate to the <CATALINA_HOME>bin directory.
  • Execute the following command, making appropriate changes if you are running a different version of Tomcat. This example is for Tomcat 9:
    tomcat9w.exe //ES//LabKeyTomcat9
    • The command will open a program window.
    • If this produces an error that says "The specified service does not exist on the server", then see the note below.
  • Go to the Java tab in the new window.
  • In the Java Options box, scroll to the bottom of the properties, and set the following property:
    -XX:-HeapDumpOnOutOfMemoryError
  • Change the Initial memory pool to 2GB for a test server or 4GB for a production server.
  • Change the Maximum memory pool to the same value.
  • Click the OK button
  • Restart LabKey Server
NOTE: The text after the //ES// must exactly match the name of the Windows Service that is being used to start/stop your LabKey Server. You can determine the name of your Windows Service by taking the following actions:
  • Open the Windows Services panel. (Click Start, type "Services", and press the Enter key.)
  • In the Services panel, find the entry for LabKey Server. It might be called something like Apache Tomcat or LabKey
  • Double-click on the service to open the properties dialog.
In the command above, replace the text "LabKeyTomcat#" with the text shown next to ServiceName in the Properties dialog.

Change the JVM Heap Size on Linux and OSX

Method 1: If you use JSVC to start/stop LabKey Server

  • Find the JSVC service script.
    • On Linux servers, this is usually in the /etc/init.d directory and named either "tomcat" or "tomcat#"
    • On OSX servers this might be in /usr/local/jsvc/Tomcat#.sh
  • Open the JSVC service script using your favorite editor
  • Find the line for setting CATALINA_OPTS and add one of the following settings inside the double quotes.
    • For a production server, use:
      -Xms4g -Xmx4g -XX:-HeapDumpOnOutOfMemoryError
    • For a test server, use:
      -Xms2g -Xmx2g -XX:-HeapDumpOnOutOfMemoryError
  • Save the file
  • Restart LabKey Server

Method 2: If you use startup.sh and shutdown.sh to start/stop LabKey Server

The start script is located in <CATALINA_HOME>/bin/catalina.sh

  • Open the Start script using your favorite text editor
  • Above the line reading "# OS specific support. $var _must_ be set to either true or false.: add one of the following:
    • For a production server, use:
      JAVA_OPTS="$JAVA_OPTS -Xms4g -Xmx4g -XX:-HeapDumpOnOutOfMemoryError"
    • For a test server, use:
      JAVA_OPTS="$JAVA_OPTS -Xms2g -Xmx2g -XX:-HeapDumpOnOutOfMemoryError"
  • Save the file
  • Restart LabKey Server

Related Topics

See Wikipedia for more information about the JVM heap memory.




Server Bootstrap Properties


Some of the initial configuration and startup steps of bootstrapping LabKey Server, i.e. launching a new server on a new database, can be skipped by creating a properties file that is applied at startup.
Configuration steps which can be provided via the bootstrap properties file include, but are not limited to:
  • Some site settings, such as file root and base server URL
  • Script engine definitions, such as automatically enabling R scripting
  • User groups and permission roles
  • Custom content for the home page

Using Bootstrap Property Files

Bootstrap property files are named with a .properties extention and placed in the /trunk/build/deploy/startup directory. Create this directory in your enlistment if it does not exist.

One or more properties files can be defined and placed in this startup directory. The files are applied in reverse-alphabetical order. Any property defined in two such files will retain the "last" setting applied, i.e the one in the file that is "alphabetically" first.

To control specific ordering among files, you could name multiple .properties files with sequential numbers, ordered starting with the one you want to take precedence. For instance, you might have:

  • 01_application.properties: applied last
  • 02_other.properties
  • ...
  • 99_default.properties: applied first
In this example, anything specified in "99_default.properties" will be overridden if the same property is also defined in any lower-numbered file. All values set for properties in the file "01_application.properties" will override any settings included in other files.

Bootstrap Properties File Format

The properties file is a list of lines setting various properties. The format of a given line is:

<optional scope>.<name>;<optional modifier> = <value>

For example, to emulate going to the "Look and Feel Settings", then setting the "initial" "systemEmailAddress" to be "support@labkey.com", you would include this line:

LookAndFeelSettings.systemEmailAddress:bootstrap=support@labkey.com

Modifier Options

The "<optional modifier>" can be one of the following:

  • bootstrap: (default) Apply the properties only when the server first starts on a new database.
  • startup: Apply the properties every time the server starts.
  • immutable: Set and never change.

Create and Apply Property Files

As an example scenario, you might want to bypass the default "first login" UI by preconfiguring your login and password.

  • If it does not exist, create the directory: /trunk/build/deploy/startup
  • Create a .properties file, such as 01default.properties
  • Paste this content, substituting your own email address:
    • "LookAndFeelSettings.systemEmailAddress:initial=MYEMAIL@YOUR_DOMAIN.com"
  • Save the file, ensuring it maintains a .properties extension.
  • Restart your server on a new database.
  • Instead of the traditional new log in steps, the example property is prefilled.

Minimal Set of Properties to Bootstrap LabKey Server

SiteSettings.systemEmailAddress = {systemEmail} SiteSettings.defaultDomain = {defaultDomain} SiteSettings.baseServerURL = {baseServerUrl} SiteSettings.exceptionReportingLevel = HIGH SiteSettings.usageReportingLevel = MEDIUM SiteSettings.sslRequired = true

#Initial User, we may also want to define a site admin user UserRoles.{email} = org.labkey.api.security.roles.ApplicationAdminRole

#Script Engine Definition, not required for bootstrap but required for scenario: ScriptEngineDefinition.R.name;bootstrap = R Scripting Engine ScriptEngineDefinition.R.languageName = R ScriptEngineDefinition.R.languageVersion = {version} ScriptEngineDefinition.R.extensions = R,r ScriptEngineDefinition.R.exePath = {exePath} ScriptEngineDefinition.R.exeCommand = CMD BATCH --slave ScriptEngineDefinition.R.outputFileName = ${scriptName}.Rout

Related Topics

For more about options that can be controlled with a startup properties file:




Set Up Robots.txt and Sitemaps


If your server allows public access, you may wish to customize how external search engines crawl and index your site. Usually this is done through robots.txt and sitemap files.

You can place robots.txt and sitemap files (or other site-specific, static content) into the extraWebapp directory. This directory is located inside the <LabKey_Root>\build\deploy directory. It is a peer to the labkeyWebapp folder .

Files in the extraWebapp directory will not be deleted when your site is upgraded to a new version of LabKey Server.




GROUP_CONCAT Install


This topic explains how to install the Microsoft SQL Server GROUP_CONCAT CLR (Common Language Runtime) functions. You may need to install these functions as part of setting up a shared SQL Server installation.

GROUP_CONCAT is a SQL aggregate function (similar to SUM, MIN, or MAX) that combines values from multiple rows into a single string value. For example, executing GROUP_CONCAT on a column with row values "First", "Second", and "Third" produces a single value "First, Second, Third". Some databases, such as MySQL, include this as a built-in function. Microsoft SQL Server does not, so LabKey requires a CLR function that implements the capability.

This function is typically installed automatically as part of the regular LabKey installation process. However, the process can fail if, for example, the database user does not have permission to install CLR functions. In these cases, a database administrator needs to install the function manually.

Note on permissions: To install CLR functions, the database user must have sysadmin permissions. When installing on SQL Server in an Amazon AWS RDS instance, this permission is not available. Instead, you need to create a DB Parameter group in RDS. This advanced feature enables installation of CLRs.

On a workstation with a connection to the Microsoft SQL Server Database Server:

  • If the automatic installation has failed, site administrators will see a banner message on the running server reading "The GROUP_CONCAT aggregate function is not installed. This function is required for optimal operation of this server." with two links:
    • Click Download installation script in the banner message to download the required script, named "groupConcatInstall.sql"
    • Click View installation instructions in the banner message to open to this topic.
  • Connect to the Microsoft SQL Server using an account with membership in the sysadmin role.
  • Execute the downloaded SQL script in the database.
  • Confirm that group_concat is installed in the core schema.
  • Restart Tomcat. The changes to the database will be recognized by the server only after a restart.

The Microsoft SQL Server GROUP_CONCAT CLR functions were developed by opcthree. Code is published at http://groupconcat.codeplex.com/ and licensed under the Microsoft Public License (Ms-PL)

Related Topics




Sending Email from Non-LabKey Domains


This topic describes the steps required for a LabKey Cloud Server to be able to send emails from a non-LabKey email address, such as from a client's "home" email domain. In the legacy hosted environment, clients could configure such emails themselves without this process.

Background

To send email from the client email domain, clients must authorize LabKey to send email on their behalf by creating a new text record in their DNS system known as a DKIM (DomainKeys Identified Mail) record.

What is DKIM?

DKIM is an email authentication method designed to detect email spoofing and prevent forged sender email addresses. Learn more on Wikipedia.

Why has LabKey implemented this new requirement?

LabKey takes client security seriously. LabKey Cloud Servers typically do not use client email servers to send email. Further, many clients use LabKey to manage PHI data and thus need to meet strict compliance guidelines. With LabKey using DKIM authorization, clients can be assured that email originating from LabKey systems has been authorized by their organization thus increasing the level of trust that the content of the email is legitimate.

PostMark

How does mail get sent from a LabKey Cloud Server?

To prevent mail from our servers being filtered as spam and generating support calls when clients can't find messages, LabKey uses a mail service called PostMark.

PostMark confirms through various methods that mail being sent by its servers isn't spam, and can therefore be trusted by recipients.

One part of the configuration requires that every "FROM" email domain being sent by through the LabKey account has a DKIM record. A DKIM record is like a password that tells PostMark LabKey has permission to send mail from that domain. This prevents domain-spoofing in emails coming from LabKey and being sent through PostMark, thus ensuring the integrity of both LabKey and PostMark's reputation.

When LabKey sends a message from one of our cloud servers, it is sent to a specific PostMark email server via a password-protected account. PostMark then confirms the domain is one LabKey has a DKIM record for.

Because PostMark's model is to protect domains, LabKey cannot assign DKIM records to specific hosts, only to domains like labkey.com. As such, mail is sent from our cloud servers as username@domain, as opposed to username@host.domain.

If there's no DKIM for the domain in the email address, PostMark bounces the email from its server and never sends it. If the domain is DKIM approved, the mail is then sent on to the recipient.

Configure DNS Records

To configure DNS records so mail from client email address goes through, the following steps must be completed by both LabKey and the client:

  1. The client tells LabKey which domain they want to send email from.
  2. LabKey's DevOps team then configures PostMark to accept mail with that domain in the from address. At this point, PostMark gives LabKey a DKIM record.
  3. LabKey passes the DKIM records to the client for the client to add to their DNS provider.
  4. The client tells LabKey when they've done this and the LabKey DevOps team confirms that the DKIM record is properly configured.
  5. LabKey sends a test message from that domain to ensure the mail is being accepted and sent.
  6. LabKey informs the client that they can then send from their domain.
This entire process can be done in less than a day, provided the client is able to add the DKIM record with quick turnaround.

DKIM Records

What are the ramifications of adding a DKIM record for the client?

Because DKIM records are TXT records specific to PostMark, these records have no impact on the client apart from authorizing their LabKey Cloud Server to send email with their domain name. DKIM records do not impact existing mail configurations that the client is already using. They do not supplant MX records or add to them. For all intents and purposes, this record is invisible to the client -- it is only used by PostMark when mail is sent from a LabKey server with the client's domain in the from field.

Workarounds

Is there any way around needing the client to add a DKIM record?

  • If the client wants to send mail from their domain from a LabKey Cloud Server, they must add the DKIM record.
  • If they do not add this record, clients can configure their LabKey Cloud Server to send email from a LabKey domain (e.g. do_not_reply@labkey.com). LabKey has already created DKIM records for its email domains.



PremiumStats Install


Premium Feature — An enhanced set of summary statistics is available in the Professional, Professional Plus, and Enterprise Editions. Learn more or contact LabKey.

This topic explains how to install the Microsoft SQL Server PremiumStats CLR functions. You may need to install these functions as part of setting up a SQL Server installation with a premium edition of LabKey Server.

PremiumStats is a CLR (Common Language Runtime) assembly with aggregate functions supporting LabKey Premium summary statistics, including median, median absolute deviation, quartiles, and interquartile ranges. Microsoft SQL Server does not support these natively, so LabKey requires a CLR assembly to implement these capabilities.

This assembly and functions are typically installed automatically as part of the regular LabKey installation process. However, the process can fail if, for example, the database user does not have permission to install CLR assemblies. In these cases, a database administrator needs to install the assembly manually.

On a workstation with a connection to the Microsoft SQL Server Database Server:

  • If the automatic installation has failed, site administrators will see a banner message on the running server reading "The premium aggregate functions are not installed. These functions are required for premium feature summary statistics." with two links.
    • Click Download installation script in the banner message to download the required script: "premiumAggregatesInstall.sql"
    • Click View installation instructions in the banner message to open to this topic.
  • Connect to the Microsoft SQL Server using an account with membership in the sysadmin role.
  • Execute the downloaded SQL script in the database.
  • Confirm that PremiumStats is installed in the core schema.
  • Restart Tomcat. The changes to the database will be recognized by the server only after a restart.

Related Topics




Deploying an AWS Windows Application Firewall


This topic outlines the process for deploying an AWS Web Application Firewall (WAF) to protect LabKey instances from DDoS and Malicious Attacks.

Purpose

Public facing LabKey instances are subject to “internet background radiation” by nefarious miscreants who seek to comprise systems to gain access to protected data. Typically motivated by financial extortion, these individuals use DDoS and bot networks to attack victims. Fortunately there are some easy and low cost tools to protect against many attacks. This document describes how to deploy an AWS Web Application Firewall (WAF) to protect against the OWASP top 10 vulnerabilities and many malicious bot networks.

Related Documentation

Tutorial: Implementing a DDoS-resistant Website Using AWS Services:

Requirements

  • LabKey Instance Deployed in AWS on EC2
  • Configured Elastic Load Balancer with target group routing to LabKey EC2 Instance
  • Required AWS Permissions to use CloudFormation, WAF, IAM Policies, S3, Lambda etc.

Considerations

Many LabKey core features require uploading and downloading of files. These types of activities are difficult to distinguish from malicious activities as the methods used to upload malicious code is indistinguishable from normal workflows. To address possible false positives, clients have the following options:

  • Use the IP Address whitelist function of the WAF to whitelist specific IP addresses or IP Address ranges of users originating from within the clients network. (e.g. whitelist the Public NAT gateway of the client’s network).
  • If whitelisting is not feasible due to expectation of random user IP addresses from various internet locations; consider setting the XSS rule to count vs block. (See information below). While this may reduce the effectiveness of the WAF to protect against XSS attacks, clients still gain the benefit of other WAF features which block known malicious attacker source IP’s.

Deployment

Architecture

Deployment Steps

Follow the AWS Tutorial for detailed steps to deploy the WAF using the CloudFormation Template: High level steps:
  1. Deploy the WAF in the same region as your LabKey Instance.
  2. Configure the WAF to protect your Elastic Load Balancer
  3. Optional: Configuring IP Address Whitelists
  4. Login to AWS and go to the WAF Console.
  5. From the WAF Console, choose WebACL’s.
  6. In the WebACL Conditions, choose IP Addresses.
  7. Edit the "Whitelist Set" rule to include the IP addresses or address ranges you wish to whitelist.
  8. Test.

Optional: Configuring XSS Rule from Block to Count

Edit the WebACL following the steps below:

  1. Login to AWS and go to the WAF Console.
  2. From the WAF Console, choose WebACL’s.
  3. Click the WAF Name in the WebACL’s list. In the resulting dialog that opens, click the Rules tab to see the list of the ACL rules.
  4. Click the Edit web ACL button.
  5. For the XSS Rule, change the rule from "Block" to "Count".
  6. Click the Update button to save changes.
  7. Test.



Configure the Virtual Frame Buffer on Linux


You may need to configure the X virtual frame buffer in order for graphics functions such as png() to work properly in R. This page walks you through an example installation and configuration of the X virtual frame buffer on Linux. For further information on when and why you would need to configure the virtual frame buffer, see Install and Set Up R.

Example Configuration

  • Linux Distro: Fedora 7
  • Kernel: 2.6.20-2936.fc7xen
  • Processor Type: x86_64

Install R

Make sure you have completed the steps to install and configure R. See Install and Set Up R for general setup steps. For Linux-specific instructions, see Configure R on Linux.

Install Xvfb

If the name of your machine is <YourServerName>, use the following:

[root@<YourServerName> R-2.6.1]# yum update xorg-x11-server-Xorg 
[root@<YourServerName> R-2.6.1]# yum install xorg-x11-server-Xvfb.x86_64

Start and Test Xvfb

To start Xvfb, use the following command:

[root@<YourServerName> R-2.6.1]# /usr/bin/Xvfb :2 -nolisten tcp -shmem

This starts a Display on servernumber = 2 and screen number = 0.

To test whether the X11, PNG and JPEG devices are available in R:

[root@<YourServerName> R-2.6.1]# export DISPLAY=:2.0 
[root@<YourServerName> R-2.6.1]# bin/R

You will see many lines of output. At the ">" prompt, run the capabilities() command. It will tell you whether the X11, JPEG and PNG devices are functioning. The following example output shows success:

> capabilities() 
jpeg png tcltk X11 http/ftp sockets libxml fifo
TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
cledit iconv NLS profmem
TRUE TRUE TRUE FALSE

Make configuration changes to ensure that Xvfb is started at boot-time

You need to make sure that Xvfb runs at all times on the machine or R will not function as needed. There are many ways to do this. This example uses a simple start/stop script and treats it as a service.

The script:

[root@<YourServerName> R-2.6.1]# cd /etc/init.d 
[root@<YourServerName> init.d]# vi xvfb
#!/bin/bash
#
# /etc/rc.d/init.d/xvfb
#
# Author: Brian Connolly (LabKey.org)
#
# chkconfig: 345 98 90
# description: Starts Virtual Framebuffer process to enable the
# LabKey server to use R.
#
#

XVFB_OUTPUT=/usr/local/labkey/Xvfb.out
XVFB=/usr/bin/Xvfb
XVFB_OPTIONS=":2 -nolisten tcp -shmem"

# Source function library.
. /etc/init.d/functions


start() {
echo -n "Starting : X Virtual Frame Buffer "
$XVFB $XVFB_OPTIONS >>$XVFB_OUTPUT 2>&1&
RETVAL=$?
echo
return $RETVAL
}

stop() {
echo -n "Shutting down : X Virtual Frame Buffer"
echo
killproc Xvfb
echo
return 0
}

case "$1" in
start)
start
;;
stop)
stop
;;
*)
echo "Usage: xvfb {start|stop}"
exit 1
;;
esac
exit $?

Now test the script with the standard:

[root@<YourServerName> etc]# /etc/init.d/xvfb start 
[root@<YourServerName> etc]# /etc/init.d/xvfb stop
[root@<YourServerName> etc]# /etc/init.d/xvfb
This should work without a hitch.

Note: Any error messages produced by Xvfb will be sent to the file set in

$XVFB_OUTPUT.
If you experience problems, these messages can provide further guidance.

The last thing to do is to run chkconfig to finish off the configuration. This creates the appropriate start and kills links in the rc#.d directories. The script above contains a line in the header comments that says "# chkconfig: 345 98 90". This tells the chkconfig tool that xvfb script should be executed at runlevels 3,4,5. It also specifies the start and stop priority (98 for start and 90 for stop). You should change these appropriately.

[root@<YourServerName> init.d]# chkconfig --add xvfb
Check the results:
[root@<YourServerName> init.d]# chkconfig --list xvfb 
xvfb 0:off 1:off 2:off 3:on 4:on 5:on 6:off

Verify that the appropriate soft links have been created:

[root@<YourServerName> init.d]# ls -la /etc/rc5.d/ | grep xvfb 
lrwxrwxrwx 1 root root 14 2008-01-22 18:05 S98xvfb -> ../init.d/xvfb

Start the Xvfb Process and Setup the DISPLAY Env Variable

Start the process using:
[root@<YourServerName> init.d]# /etc/init.d/xvfb start

Now you will need to the set the DISPLAY env variable for the user. This is the DISPLAY variable that is used to run the TOMCAT server. Add the following the .bash_profile for this user. On this serer, the TOMCAT process is run by the user tomcat

[root@<YourServerName> ~]# vi ~tomcat/.bash_profile 
[added]
# Set DISPLAY variable for using LabKey and R.
DISPLAY=:2.0
export DISPLAY

Restart the LabKey Server or it will not have the DISPLAY variable set

On this server, we have created a start/stop script for TOMCAT within /etc/init.d. So I will use that to start and stop the server

[root@<YourServerName> ~]# /etc/init.d/tomcat restart

Test the configuration

The last step is to test that when R is run inside of the LabKey server, the X11,JPEG and PNG devices are available

Example:

The following steps enable R in a folder configured to track Issue/Bug Tracking:

  1. Log into the Labkey Server with an account with Administrator privs
  2. In any Project, create a new SubFolder
  3. Choose a "Custom"-type folder
  4. Uncheck all boxes on the right side of the screen except "Issues."
  5. Hit Next
  6. Click on the button "Views" and a drop-down will appear
  7. Select "Create R View"
  8. In the text box, enter "capabilities()" and hit the "Execute Script" button.
You should see the following output:
jpeg png tcltk X11 http/ftp sockets libxml fifo 
TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE
cledit iconv NLS profmem
FALSE TRUE TRUE FALSE
> proc.time()
user system elapsed
0.600 0.040 0.631

The important thing to see here is that X11, png and jpeg all say "TRUE." If they do not, something is wrong.




Supported Technologies


Supported Technologies Roadmap

This chart summarizes server-side dependency recommendations for past & current releases, and predictions for upcoming releases.

Do not use: not yet available or tested with this version of LabKey
        Recommended: fully supported and thoroughly tested with this version of LabKey
Upgrade ASAP: deprecated and no longer supported with this version of LabKey
Do not use: incompatible with this version of LabKey and/or past end of life (no longer supported by the organization that develops the component)

 

ComponentVersionLabKey 18.3.x
(Nov 2018)
LabKey 19.1.x
(Mar 2019)
LabKey 19.2.x
(Jul 2019)
LabKey 19.3.x
(Nov 2019)
LabKey 20.1.x
(Mar 2020)
JavaJava 13  
Java 12   
Tomcat9.0.x     
8.5.x     
7.0.x     
PostgreSQL11.x     
10.x     
9.6.x     
9.5.x     
9.4.x     
Microsoft SQL Server2017     
2016     
2014     
2012     

Browsers

LabKey Server requires a modern browser for many advanced features, such as locking data table column headers in place when scrolling, creating charts, or visualizing security group relationships. To take advantage of all features, we recommend upgrading your browser(s) to the latest release. Since this is not always possible, the LabKey team attempts to maintain basic compatibility and fix major issues with the following browsers:

As a general rule, LabKey Server supports the latest version of the browsers listed above. To review and download the latest versions, click the links provided. Note that Internet Explorer is not supported.

If you experience a problem with a supported browser feel free to post the details to the support forum so we're made aware of the issue.

Java

LabKey Server requires Oracle OpenJDK 12, Oracle's production-ready open-source build of the Java Development Kit. We strongly recommend installing the latest point release, currently 12.0.2. OpenJDK 12 is the only version of Java receiving public updates (i.e., regular patches to address security and reliability issues); as a result, LabKey does not support Java 8, 9, 10, or 11. You may want to familiarize yourself with recent changes to the Java licensing and release schedule that affect every LabKey deployment; see this post on the LabKey blog.

LabKey Server has not been tested with other Java distributions such as the Oracle commercial Java SE, AdoptOpenJDK, Amazon Corretto, Red Hat, Zulu, OpenJ9, etc.

Apache Tomcat

We recommend using the latest point release of Apache Tomcat 9.0.x (currently 9.0.22).

LabKey continues to support older versions 8.5.x and 7.0.x, though we strongly recommend installing the latest point release of each (currently 8.5.43 or 7.0.96); older point releases have major compatibility issues and security vulnerabilities. Note that Tomcat 7.0.x is not compatible with some advanced features of LabKey Server (e.g., Docker and RStudio integration). Tomcat 8.0.x is no longer supported as it has reached end of life; it has been superseded by Tomcat 8.5.x.

We recommend installing Tomcat using the binary distributions; if a package manager is used, the lib directory may be installed in a different location. Also, the packages sometimes include alternative versions of some components (like JDBC connection pools) that can cause incompatibilities.

We recommend not using the Apache Tomcat Native library; this library can interfere with SSL and prevent server access via LabKey's client libraries.

PostgreSQL

For installations using PostgreSQL as the primary database, we recommend using the latest point release of PostgreSQL 11.x (currently 11.5).

For those who can't transition to 11.x yet, LabKey continues to support PostgreSQL 10.x, 9.6.x, 9.5.x, and 9.4.x, though here also we strongly recommend installing the latest point release (currently 10.10, 9.6.15, 9.5.19, and 9.4.24) to ensure you have all the latest security, reliability, and performance fixes.

PostgreSQL provides instructions for how to upgrade your installation, including moving your existing data database.

Microsoft SQL Server

For installations using Microsoft SQL Server as the primary database, we recommend using Microsoft SQL Server 2017, which we've tested on both Windows and Linux. We continue to support and test on SQL Server 2016, 2014, and 2012. LabKey does not support SQL Server 2008 R2 or earlier releases.




Troubleshoot Server Installation and Configuration


In case of errors or other problems when installing and running LabKey Server, first review installation basics and options linked on the Install LabKey page.

Developer Mode

Running a server in "devmode" provides enhanced logging and enables the MiniProfiler. To run the server in devmode, follow these steps. The supported version of tomcat will change over time; replace the # in our examples with the actual version number.

  • Open a command prompt.
  • Go to the <CATALINA_HOME>/bin directory, for example:
C:\labkey\apps\apache\apache-tomcat-#.#.##\bin
  • Execute the tomcat#w.exe program:
tomcat#w.exe //ES//LabKeyTomcat#
  • The command will open a program window. Click the Java tab.
  • In the Java Option box, scroll to the bottom of the properties. Add the following property at the bottom of the list:
-Ddevmode=true
  • Close the program window and restart the server.
  • To confirm that the server is running in devmode:
    • Go to (Admin) > Site > Admin Console.
    • Click Admin Console Links.
    • Under Diagnostics, click System Properties.
    • Locate the devmode property, and confirm that its value is 'true'.

Diagnostic Information

Which Version of LabKey Server Is Running?

  • Find your version number at (Admin) > Site > Admin Console.
  • Click Module Information.
  • Click the (Expand) icon for the Core module. The Version is the version number for your installation of LabKey Server.

Common Issues

Conflicting Applications

Before installing LabKey Server, you should shut down all other running applications. If you have problems during the installation, try additionally shutting down any virus scanning application, internet security applications, or other applications that run in the background.

Compatible Component Versions

Confirm that you are using the supported versions of the required components, as detailed in the Supported Technologies Roadmap. It is possible to have multiple versions of some software, like Java, installed at the same time. Check that LabKey and other applications, such as Tomcat, are configured to use the correct versions.

PostgreSQL Installation

You may need to remove references to Cygwin from your Windows system path before installing LabKey, due to conflicts with the PostgreSQL installer. The PostgreSQL installer also conflicts with some antivirus or firewall software programs. (see http://wiki.postgresql.org/wiki/Running_%26_Installing_PostgreSQL_On_Native_Windows for more information).

Tomcat Failure to Start

If installation fails to start tomcat (such as with an error like "The specified service already exists."), you may need to manually stop or delete a failing Tomcat service.

To stop the service on Windows, open Control Panel > Administrative Tools > Services. Select the relevant service, such as LabKey Server Apache Tomcat #.0 and click Stop.

To delete, run the following from the command line as an administrator, substituting the major version number for the #:

sc delete LabKeyTomcat#

Connection Pool Size

If your server becomes unresponsive, it could be due to the depletion of available connections to the database. Watch for a Connection Pool Size of 8, which is the Tomcat connection pool default size and insufficient for a production server. To see the connection pool size for the LabKey data source, select (Admin) > Site > Admin Console and check the setting of Connection Pool Size on the Server Information panel. The connection pool size for every data source is also logged at server startup.

To set the connection pool size, edit your labkey.xml configuration file and change the appropriate setting for your LabKey data source to at least 20. Depending on the number of simultaneous users and the complexity of their requests, your deployment may require a larger connection pool size. The setting name depends on your Tomcat version:

  • Tomcat 7: the setting to change is called "maxActive".
  • Tomcat 8 and later: the setting to change is called "maxTotal".
You should also consider changing this setting for external data sources to match the usage you expect.

The Tomcat data source connection pool settings changed significantly in Tomcat 8, so you must change your data source settings when you upgrade from Tomcat 7 to a later version. The most important change is to replace your "maxActive" setting (used with Tomcat 7) with "maxTotal". Review the Database Connection Pooling section of the Tomcat Migration Guide for more information. See External Schemas and Data Sources for details and data source configuration examples.

Restart Installation from Scratch

If you have encountered prior failed installations, don't have any stored data you need to keep, and want to clean up and start completely from scratch, the following process may be useful:

  • Delete the Tomcat service (if it seems to be causing the failure):
    • From the command line (may need to run as administrator):
sc delete LabKeyTomcat7
  • Uninstall PostgresSQL using their uninstaller.
    • Control Panel > Programs and Features
    • Select PostgreSQL program.
    • Click Uninstall.
  • Delete the entire LabKey installation directory.
  • Install LabKey again.

Installation Forum

You can search for problems resolved through community support in the LabKey Installation Forum.

If you don't see your issue listed there, you can post a new question. If the install seems successful, it is often helpful to submit debugging logs for diagnosis.

If the install failed to complete, please include the install.log and install-debug.log from your selected LabKey install directory.

PostgreSQL logs its installation process separately. If PostgreSQL installation/upgrage fails, please locate and include the PostgreSQL install logs as well.

Related Topics




Installation Error Messages


If you have encountered errors or other problems when installing and starting LabKey Server, first review the topic Troubleshooting: Common Issues. If you're still encountering problems, please review the list below for common errors, messages, and problems.

You can also search the LabKey Community Support Forums for guidance. If you don't already see your issue listed there, please post a new question.


1.

Error

Error on startup, "Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections."

ProblemTomcat cannot connect to the database.
Likely causes
  • The database is not running
  • The database connection URL or user credentials in the Tomcat configuration files are wrong
  • Tomcat was started before the database finished starting up
SolutionMake sure that database is started and fully operational before starting Tomcat. Check the database connection URL, user name, and password in the <tomcat>/conf/Catalina/localhost/labkey.xml file.

2.

ProblemError when connecting to LabKey server on Linux: Can't connect to X11 window server or Could not initialize class ButtonServlet.
SolutionRun tomcat headless. Edit tomcat's catalina.sh file, and add the following line near the top of the file:
CATALINA_OPTS="-Djava.awt.headless=true"
Then restart tomcat.

3.

Problem

Viewing certain pages results in a specific NoSuchMethodError.

Error

java.lang.NoSuchMethodError: org.apache.jasper.runtime.JspRuntimeLibrary.releaseTag(Ljavax/servlet/jsp/tagext/Tag;Lorg/apache/tomcat/InstanceManager;Z)V
       at org.labkey.jsp.compiled.org.labkey.core.admin.maintenance_jsp._jspx_meth_labkey_005ferrors_005f0(maintenance_jsp.java:159)
       at org.labkey.jsp.compiled.org.labkey.core.admin.maintenance_jsp._jspService(maintenance_jsp.java:110)
       at org.labkey.api.view.JspView.renderView(JspView.java:170)
       at org.labkey.api.view.WebPartView.renderInternal(WebPartView.java:372)


Solution

Upgrade your Apache Tomcat deployment to a supported version.


4.

Error

You receive a message "The requested resource () is not available." OR "500: Unexpected server error" and see something like one of the following in the log file:

Problem
SEVERE: Error deploying configuration descriptor labkey.xml 
java.lang.IllegalStateException: ContainerBase.addChild: start: LifecycleException: start: : java.lang.UnsupportedClassVersionError: org/labkey/bootstrap/LabkeyServerBootstrapClassLoader : Unsupported ...
A failure occurred during LabKey Server startup.
java.lang.NoClassDefFoundError: javax/script/ScriptEngineFactory....
A failure occurred during LabKey Server startup.
java.lang.UnsupportedClassVersionError: Bad version number in .class file
       at java.lang.ClassLoader.defineClass1(Native Method)
       at java.lang.ClassLoader.defineClass(ClassLoader.java:675) ...
Solution

You may need to use a newer version of the JDK. See Install Required Components.

Confirm that Tomcat is configured to use the correct version of Java, as it is possible to have multiple versions installed simultaneously.


5.

Problem

Fatal Error in Java Runtime Environment

Error
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x0000000000000000, pid=23893, tid=39779
#
# JRE version: Java(TM) SE Runtime Environment (8.0_45-b14) (build 1.8.0_45-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.45-b02 mixed mode bsd-amd64 compressed oops)
# Problematic frame:
# C 0x0000000000000000
#
# Failed to write core dump. Core dumps have been disabled.
# To enable core dumping, try "ulimit -c unlimited" before starting Java again
Cause

These are typically bugs in the Java Virtual Machine itself.

Solution

Ensuring that you are on the latest patched release for your preferred Java version is best practice for avoiding these errors. If you have multiple versions of Java installed, be sure that JAVA_HOME and other configuration is pointing at the correct location. If you are running through the debugger in IntelliJ, check the JDK configuration: under Project Structure > SDKs check the JDK home path and confirm it points to the newer version.




Dump Memory and Thread Usage Debugging Information


To assist in debugging errors, an administrator can force the LabKey Server process to dump the list of threads and memory to disk. This information can then be sent to LabKey for review.

A thread dump is useful for diagnosing issues where LabKey Server is hung or some requests spin forever in the web browser.

A memory/heap dump is useful for diagnosing issues where LabKey Server is running out of memory, or in some cases where the process is sluggish and consuming a lot of CPU performing garbage collection.

Definitions:

  • <LABKEY_HOME>: Installation location of the LabKey Server, for example:
    • Linux: /usr/local/labkey/labkey
    • Windows: C:\labkey\labkey\
  • <CATALINA_HOME>: Installation location of the Apache Tomcat Web Server (where x.x.xx is the specific version installed, for example:
    • Linux: /usr/local/labkey/apps/apache/apache-tomcat-x.x.xx
    • Windows: C:\labkey\apps\apache\apache-tomcat-x.x.xx

Thread Dump

To dump the list of running threads, you can use either the UI or the command line.

Using the UI

  • Go to (Admin) > Site > Admin Console.
  • Select the Admin Console Links tab.
  • Under Diagnostics, click Running Threads
  • This will show the state of all active threads in the browser, as well as writing the thread dump to the server log file.

Manually via the Command Line

The server monitors the timestamp of a threadDumpRequest file and when it notices a change, it initiates the thread dump process. The content of the file does not matter.

Unix. On Linux and OSX, use the command line to execute the following commands:

  • Change to the <LABKEY_HOME> directory. For example, if your <LABKEY_HOME> directory is located at /usr/local/labkey/labkey, then the command will be:
    cd /usr/local/labkey/labkey
  • Force the server to dump its memory:
    touch threadDumpRequest
Windows. On a Windows Server, do the following:
  • Open a Command Prompt.
  • Change to the <LABKEY_HOME> directory. For example, if your <LABKEY_HOME> directory is located at C:\labkey\labkey\, then the command will be:
cd "C:\labkey\labkey"
  • Force the server to dump its memory. The command will open the heapDumpRequest file in the notepad program.
notepad threadDumpRequest
  • Place the cursor at the top of the file and hit the "Enter" key twice.
    • Save the file.
    • Close notepad.
Location of the file containing the thread dump

The list of threads is dumped the "labkey.log" file, which is located in the <CATALINA_HOME>/logs directory.

Memory Dump

To request LabKey Server dump its memory, you can use either the UI or the command line. By default, it will write the file to the LABKEY_HOME directory. Java can also be configured to dump the heap if the virtual machine runs out of memory.

Heap dumps can be large: several gigabytes in size is typical. You can tell the server to write to a different location by using a JVM startup argument,

-XX:HeapDumpPath=/path/to/desired/target

Using the UI

  • Go to (Admin) > Site > Admin Console.
  • Select the Admin Console Links tab.
  • Under Diagnostics, click Dump Heap.
  • You will see a full path to where the heap was dumped.

Manually via the Command Line

The server monitors the timestamp of a heapDumpRequest file and when it notices a change, it initiates the heap dump process. The content of the file does not matter.

Unix. On Linux and OSX, execute the following commands on the command line:

  • Navigate to the <LABKEY_HOME> directory.
  • Force the server to dump its memory
touch heapDumpRequest

Windows. On a Windows Server, do the following:

  • Open a Command Prompt
  • Navigate to the <LABKEY_HOME> directory.
  • Force the server to dump its memory. This command will open the heapDumpRequest file in the notepad program.
notepad heapDumpRequest
  • Place the cursor at the top of the file and hit the Enter key twice.
    • Save the file
    • Close notepad
Location of the file containing the memory dump

The file will be located in the <LABKEY_HOME> directory. The file will have the ".hprof" extension.

Automatically Capturing Heap Dump when Server Runs Out of Memory

The Java virtual machine supports an option that will automatically trigger a heap dump when the server runs out of memory. This is accomplished via this JVM startup parameter:

-XX:+HeapDumpOnOutOfMemoryError

For more information, see Oracle's documentation.

Debugging PostgreSQL Errors

LabKey server is unable to log errors thrown by PostgreSQL, so in the case of diagnosing some installation and startup errors, it may be helpful to to view the event log.

On Windows:
  • Launch eventvwr.msc
  • Navigate to Windows Logs > Application.
  • Search for errors there corresponding to the installation failures, which may assist LabKey support in diagnosing the problem.
  • If you can't find relevant messages, you may be able to trigger the error to occur again by running net start LabKey_pgsql-9.2 from the command line.

In addition, you may find assistance debugging PostgreSQL on their troubleshooting installation page. Included there are instructions to "Collect the installer log file."

Collect SQL Server Information

To get a full picture of some problems, it's also useful to have information about running queries and locks. For this, you want both a thread dump (described above) and information about the state of database connections. The latter needs to be obtained directly from SQL Server.

  • Launch SQL Server Management Console or a similar tool
  • Open a connection to the LabKey Server database, often named "labkey."
  • Run and capture the output from the following queries/stored procedures:
    • sp_lock
    • sp_who2
    • SELECT t1.resource_type, t1.resource_database_id, t1.resource_associated_entity_id, t1.request_mode, t1.request_session_id, t2.blocking_session_id FROM sys.dm_tran_locks as t1 INNER JOIN sys.dm_os_waiting_tasks as t2 ON t1.lock_owner_address = t2.resource_address;



Community Edition Modules - Descriptions


Modules are the functional building blocks of LabKey Server. A folder's functionality is determined by the set of modules that are enabled in that folder. The Folder Type of a project or folder determines an initial set of enabled modules, and additional modules included in the deployment can be enabled as necessary. For details see Enable a Module in a Folder.

To learn which modules are included in the various editions of LabKey Server, see Modules in LabKey Server Editions.

This topic provides a few details about some modules included in the standard distribution of LabKey Server Community Edition. Other open source modules not included in the Community Edition can be downloaded and built locally with sufficient expertise and are listed in Additional Source Modules.

Announcements

Provides a ready-to-use message board where users can post announcements and files, and participate in threaded discussions. See Messages.

Audit

Records user activity on the server. See Audit Site Activity.

Core

The Core module provides central services such administration, folder management, user management, module upgrade, file attachments, analytics, and portal page management.

DataIntegration

Implements the Extract-Transform-Load (ETL) functionality.

Demo

The Demo module helps you get started building your own LabKey Server module. It demonstrates all the basic concepts you need to understand to extend LabKey Server with your own module.

Elisa

Implements the ELISA assay.

ELISpotAssay

Implements the ELISpot assay.

Experiment

The Experiment module provides annotation of experiments based on FuGE-OM standards. This module defines the XAR (eXperimental ARchive) file format for importing and exporting experiment data and annotations, and allows user-defined custom annotations for specialized protocols and data.

Web Parts included:

  • Experiment Runs
  • Experiments
  • Lists
  • Sample Sets
  • Single List
  • Experiments -> Narrow
  • Protocols -> Narrow
  • Sample Sets -> Narrow

FCSExpress

Supports importation and analysis of flow cytometry data from FCS Express.

FileContent

The FileContent module lets you share files on your LabKey Server via the web.

Web parts included:

  • Files

Flow

The Flow module supplies flow-specific services to the flow application.

Web Parts included:

  • Flow Analysis (Flow Analysis Folders)
  • Flow Analysis Scripts
  • Flow Overview (Experiment Management)

Issues

The Issues module provides a ready-to-use workflow system for tracking tasks and problems across a group.

List

Lists are light-weight data tables, often used to hold utility data that supports an application or project, such as a list of instrument configurations.

Luminex

Supports Luminex data import and analysis.

MS2

The MS2 module supplies MS2-specific services to the MS2/CPAS application.

Web Parts included:

  • MS2 Runs
  • MS2 Runs, Enhanced
  • MS2 Sample Preparation Runs
  • Protein Search
  • MS2 Statistics -> Narrow
  • Protein Search -> Narrow

NAb

The NAb module provides tools for planning, analyzing and organizing experiments that address Neutralizing Antibodies. No web parts are provided. Access NAb services via a custom tab in a custom folder.

Pipeline

The Data Pipeline module uploads experiment data files to LabKey Server. You can track the progress of uploads and view log and output files. These provide further details on the progress of data files through the pipeline, from file conversion to the final location of the analyzed runs.

Query

The Query module allows you to create customized views by filtering and sorting data.

Search

The Search module offers full-text search of server contents, implemented by Lucene.

Study

The Study module provides a variety of tools for integration of heterogeneous data types, such as demographic, clinical, and experimental data. Cohorts and participant groups are also supported by this module.

Survey

The Survey module supports custom user surveys for collecting user information, feedback, or participant data.

TargetedMS

Supports targeted mass spectrometry proteomics experiments.

Visualization

Implements the core data visualization features, including box plots, scatter plots, time charts, etc.

Wiki

The Wiki module provides a simple publishing tool for creating and editing web pages on the LabKey site. It includes the Wiki, Narrow Wiki, and Wiki TOC web parts.

Related Topics




Additional Source Modules


LabKey Server modules requiring significant customization by a developer are not included in LabKey distributions. Developers can build these modules from source code in the LabKey repository. Please contact LabKey to inquire about support options.

Additional Open Source Modules

ModuleDescriptionDocumentationSource
Reagent InventoryOrganize and track lab reagents.Reagent InventorySVN Source
File TransferTransfer files from a Globus endpoint.File Transfer Module / Globus File SharingGitHub Source
SNDStructured Narrative DatasetsStructured Narrative DatasetsContact LabKey
GitHub ProjectsMany more modules are available as GitHub projects, including Signal Data and Workflow.For documentation, see the README.md file in each project.GitHub Module List

If you wish to engage professional services to help you adopt these modules, please contact LabKey.

Related Topics