This topic provides step-by-step instructions to assist developers, including acquiring the LabKey Server source code, installing required components, and building LabKey Server from source. The instructions here are written for a Windows
machine. To set up development on a Mac/OSX/Linux
machine, use these instructions in conjunction with the topic: Notes on Setting up OSX for LabKey Development
Obtain the LabKey Source Files
The LabKey source files are stored in multiple GitHub repositories. To build LabKey Server, follow these steps.
Install a Git Client
You can use GitHub Desktop or another Git client.
- Install GitHub Desktop:
- Install a command-line git option for your platform:
Clone LabKey Source Files from GitHub
- Begin by cloning the LabKey server repository from git into a directory such as labkeyHome.
git clone https://github.com/LabKey/server.git labkeyHome
For example, on Windows this may be: C:\dev\labkey\labkeyHome. This directory is referred to as $LABKEY_HOME below.
- Clone the following core repositories. Note that the first two go into $LABKEY_HOME\server\modules, while the third goes into $LABKEY_HOME\server.
$LABKEY_HOME\server\modules> git clone https://github.com/LabKey/platform.git
$LABKEY_HOME\server\modules> git clone https://github.com/LabKey/commonAssays.git
$LABKEY_HOME\server> git clone https://github.com/LabKey/testAutomation.git
An abbreviated map of your finished enlistment, showing the main folder structure (with many other elements omitted):
LABKEY_HOME (i.e. C:\dev\labkey\labkeyHome)
- If you will be using a branch other than develop (the default branch when you clone a repository), you can check out all repositories to a given branch, by using the following commands from the root of your enlistment. First recursively check out all the subdirectories, then check out the root on the same branch. For example, to switch from develop to the release21.11-SNAPSHOT branch:
$LABKEY_HOME> gradlew gitCheckout -Pbranch=release21.11-SNAPSHOT
$LABKEY_HOME> git checkout release21.11-SNAPSHOT
- You could also choose to check out each repository, including the server repository separately. For example, to change the platform repo to the release21.11-SNAPSHOT branch:
$LABKEY_HOME\server\modules> cd platform
$LABKEY_HOME\server\modules\platform> git checkout release21.11-SNAPSHOT
Note: If you are subscribed to a premium edition of LabKey Server, you will need to set up an Artifactory account in order to access premium modules. Follow the steps in this topic:
Install Required Prerequisites
To update the JDK in the future, you will:
- 1. Repeat the steps above (installing the new version in a location parallel to the old).
- 2. Update your system variable JAVA_HOME to point to the new location.
- 3. Update the IntelliJ > File > Project Structure > SDK > labkey home path to point to the new location. Rename the existing 'labkey' to it's version number. Add the new JDK, then rename the new version "labkey".
- 4. If necessary, add any additional flags to the VM options section. For example, when updating to Java 16, you'll add these flags to your JVM switches:
Platform-specific installation instructions:
- PostgreSQL: Install using your Linux machine's native tools or download a PostgreSQL binary package. Follow the instructions in the downloaded package to install PostgreSQL.
Consider installing the database using docker
, facilitating launch with a command like:
$ docker run -p 5432:5432 --name postgres -e POSTGRES_PASSWORD=[password] -d postgres
Node.js and npm
The LabKey build depends on Node.js and the node package manager (npm). The build process automatically downloads and installs the versions of npm and node that it requires. You should not install npm and node yourself. For details on the Node.js dependency and build-prod script requirement, see Node.js Build Dependency
Note that you do not
need to install Gradle. LabKey uses the gradle wrapper to make updating version numbers easier. The gradle wrapper script (either gradlew or gradlew.bat) is included in the enlistment and is already in the $LABKEY_HOME directory.
Create a gradle properties file to contain information about global settings for your gradle build following these steps:
- Create a ".gradle" directory in your home directory.
- On OSX and Linux, typically: /Users/<you>/.gradle.
- On Windows, typically: C:\Users\<you>\.gradle). Note: the Windows file explorer may not allow you to create a folder beginning with a period. To solve this navigate to C:\Users\<you>\ in the command prompt and type mkdir .gradle.
- Create a gradle.properties file in the .gradle directory using the following process:
- Copy the file $LABKEY_HOME/gradle/global_gradle.properties_template to C:\Users\<you>\.gradle. Rename the file to create a file named gradle.properties.
- In the gradle.properties file, substitute your <CATALINA_HOME> directory (the location of your Tomcat installation), including the specific version number, for the value after systemProp.tomcat.home. Use forward slashes, not backslashes, for the Tomcat path, even on Windows. For example:
Environment Variables and System PATH
- Create or modify the system environment variable JAVA_HOME so it points to your JDK installation location (for example, C:\labkey\apps\jdk-##). Note: If you've already set the JAVA_HOME variable to point to your installation of the JRE, you should modify it to point to the JDK.
- Create or modify the system environment variable CATALINA_HOME so that it points to your Tomcat installation (for example, C:\labkey\apps\tomcat-#.#.##).
- Add the JDK to your system PATH. Using the JAVA_HOME variable will simplify upgrades later
- Add the following location to your system PATH. This directory won't exist yet, but add it to the path anyway. The Gradle build downloads binaries used by the server, such as graphvis, dot, and proteomics tools, to this directory.
- For example, C:\dev\labkey\labkeyHome\build\deploy\bin.
On OSX, for example, you would place the environment variables in your .bash_profile:
export JAVA_HOME=`/usr/libexec/java_home -v 1.11`
GWT Browser Settings (Optional)
The default developer build is optimized for Chrome but the target browser can be controlled through either a command line parameter or by setting the 'gwtBrowser' property in the gradle.properties file. Available settings are: gwt-user-chrome (the default value), gwt-user-firefox, or gwt-user-ie.
Open the LabKey Project in IntelliJ
The LabKey development team develops LabKey using IntelliJ IDEA. You can use the license-free Community Edition if you are planning on modifying or extending the LabKey source code.
Below we describe how to configure the IntelliJ development environment; we recommend employing the same general principles if you are using a different development environment. Some developers have experimented with Eclipse as the IDE and you can find some set up details on the Developer Message Board
- Download and install the latest version of IntelliJ IDEA. Either the Community or Ultimate Edition will work.
Over time, you will want to update IntelliJ to stay current with features and enhancements. You can use the Help > Check for updates...
option in the application. Watch for installation alerts about any .idea files you may have edited. Note that updates may be incremental and after updating, a second more recent update may be available.
After updating, recheck your settings described in the following section, particularly your JDK and CATALINA_HOME settings.
Configure the LabKey Project in IntelliJ
- Run the following gradle commands to configure the workspace and other default settings:
- Build the application from the command line
- Open the LabKey project in IntelliJ:
- Launch IntelliJ.
- If your IntelliJ install is brand new, you will see the "Welcome to IntelliJ" pop up screen. Click Open.
- If you have previously installed IntelliJ, select File > Open.
- Select the LabKey IntelliJ project directory: $LABKEY_HOME
- Set Path Variables
- Select File > Settings > Appearance & Behavior > Path Variables. (On a Mac, the menu path is IntelliJ IDEA > Preferences > Appearance & Behavior > Path Variables).
- Click the plus icon in the upper right. Set the CATALINA_HOME path variable to <CATALINA_HOME>, the root directory of your Tomcat installation, for example, C:\labkey\apps\apache\apache-tomcat-#.#.##.
- If developing with TypeScript, add a NODE_PATH variable as well and set it to the path to the node binary that is installed by the build, for example, $LABKEY_HOME\build\.node\node-v#.#.##\bin\node
- Click OK to close the Settings window.
- Configure the Target JDK
- In IntelliJ, select File > Project Structure.
- Under Project Settings, click Project.
- Under Project SDK click New and then click JDK.
- Browse to the path of your JDK, for example, (C:\java\jdk-##), and click OK.
- Click Edit. Change the name of the JDK to "labkey".
- Click OK to close the Project Structure window.
- Open the Gradle tool window at View > Tool Windows > Gradle.
- Click the refresh icon. This will take as much as 5-20 minutes. You should start seeing messages about its progress. If not, something is probably hung up. Wait for this sync to complete before progressing with further IntelliJ configuration steps.
Edit configuration options as follows:
- Select Run > Edit Configurations. (If the menu is greyed-out, wait until IntelliJ finishes indexing the project files.)
- Open the Application node in the left panel and select LabKey Dev.
- The following instructions apply to IntelliJ version 2020.3. You can update now, or for earlier releases of IntelliJ, follow this section in the documentation archives
- From the Modify options... menu, check that both Add VM Options and Use classpath of module are checked.
- Hover over the entry panels in the upper part of the Run panel to see a description of what to select or enter in each one.
- VM options:
- Confirm that the path separators are appropriate for your operating system. On Windows, ensure that the paths to the jar files are separated by semicolons. For example: "./bin/bootstrap.jar;./bin/tomcat-juli.jar;C:/Program Files (x86)/JetBrains/IntelliJ IDEA 20##.#.#/lib/idea_rt.jar". For Macs, the paths should be separated by a colon (:) instead.
- Check that any JVM memory settings (like "-Xms###" and "-Xmx###") in this line are sufficient (if present at all). Determine the correct values in the topic Configure Tomcat Webapp Memory. If these settings are not present, the system allocations for memory will be used.
- To run your instance in development mode include the argument -Ddevmode=true
- For JDK 16 and up, include the arguments --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED
- Select classpath:
- Hover over the upper right panel and set the classpath to "...platform.api.main" (or "api_main" or "org.labkey-api_main" in previous releases).
- Working directory:
- Confirm that this points to your current Tomcat installation (i.e. to the same location as you set CATALINA_HOME).
- Click OK, to close the Run/Debug Configurations window.
Be sure that IntelliJ has enough heap memory. The default max is OK if you’re just dealing with the core modules, but you will likely need to raise the limit
if you’re adding in custom modules, 3GB seems sufficient.
Build and Run LabKey
Configure the pg.properties File
The LabKey source includes a configuration file for use with PostgreSQL (pg.properties
), specifying JDBC settings, including URL, port, username, password, etc.
- Open the file $LABKEY_HOME/server/configs/pg.properties
- Edit the file, adding your values for the jdbcUser and jdbcPassword. (This password is the one you specified when installing PostgreSQL. If your password contains an ampersand or other special XML characters, you will need to escape them, as the value will be substituted into an XML template without encoding. For example, if your JDBC password is "this&that", then use the escaped version "this&that".)
- In a command window, go to $LABKEY_HOME
- Run "gradlew pickPg" to configure labkey.xml with the corresponding database settings.
- You may need to manually create the directory <CATALINA_HOME>/conf/Catalina/localhost.
When you build LabKey, the values that you've specified in the properties file are copied into the LabKey configuration file, labkey.xml, overwriting previous values. This file is then copied into <CATALINA_HOME>/conf/Catalina/localhost
To learn more about the build process, the various build targets available, and how the source code is transformed into deployed modules, see Build LabKey from Source
- On the command line, go to the $LABKEY_HOME directory, and invoke the gradle build target:
To control which modules are included in the build, see Customize the Build
Run LabKey Server
To run and debug LabKey:
- Select Run > Debug 'LabKey Dev' in IntelliJ.
- If Tomcat starts up successfully, navigate your browser to http://localhost:8080/labkey to begin debugging (assuming that your local installation of Tomcat is configured to use the Tomcat default port 8080).
While you are debugging, you can usually make changes, rebuild, and redeploy LabKey to the server without stopping and restarting Tomcat. Occasionally you may encounter errors that do require stopping and restarting Tomcat.
Run the Basic Test Suite
- Install Geckodriver to run tests on Firefox.
- Run the command within your $LABKEY_HOME directory, to initiate automated tests of LabKey's basic functionality:
gradlew :server:testAutomation:uiTest -Psuite=DRT
Other automated tests are available. For details, see Run Automated Tests
. Note that 'R' must first be configured
in order to run some tests.
Optional Modules on GitHub
Many optional modules are available from the LabKey repository on GitHub
. To include these modules in your build, install a Git client and clone individual modules into the LabKey Server source.
Clone Modules from LabKey's GitHub Repository
- To add a GitHub module to your build, clone the desired module into labkeyHome/labkey/server/modules. For example, to add the 'reagent' module:
...labkeyHome\server\modules>git clone https://github.com/LabKey/reagent.git
Note that you can get the URL by going to the module page on GitHub (for example, https://github.com/LabKey/reagent), clicking Clone or Download
, and copying the displayed URL.
Manage GitHub Modules via IntelliJ
Once you have cloned a GitHub module, you can have IntelliJ handle any updates:
To add the GitHub-based module to IntelliJ (and have IntelliJ generate an .iml file for the module):
- Edit your settings.gradle file to include the new module
- In IntelliJ, open the Gradle tool window at View > Tool Windows > Gradle.
- Refresh the Gradle window by clicking the arrow circle in the upper left of the Gradle window
To update the GitHub-based module using IntelliJ:
- To have IntelliJ handle source updates from GitHub, go to File > Settings (or Intellij > Preferences).
- Select Version Control.
- In the Directory panel, scroll down to the Unregistered roots section, select the module, and click the Plus icon in the lower left.
- In the Directory panel, select the target module and set its VCS source as Git, if necessary.
- Note that IntelliJ sometimes thinks that subdirectories of the module, like module test suites, have their sources in SVN instead of Git. You can safely delete these SVN sources using the Directory panel.
- To sync to a particular GitHub branch: in IntelliJ, go to VCS > Git > Branches. A popup menu will appear listing the available Git modules. Use the popup menu to select the branch to sync to.
If you have added a new module to your enlistment, be sure to customize the build
to include it in your Gradle project and then refresh the Gradle window to incorporate it into IntelliJ, as described above.
To generate documentation, such as the XML reference and Javadoc, clone the tools repository into the enlistment root (to create a sibling of the /server directory):
$LABKEY_HOME>git clone https://github.com/LabKey/tools.git
Then you can run the documentation generation tasks, such as:
Premium Resources Available
Subscribers to premium editions of LabKey Server can learn more about developing with LabKey with the developer topics and code examples listed on this page:
Premium edition clients also have the option to use Microsoft SQL Server instead of PostgreSQL as the database. Learn more in this topic:
Learn more about premium editions