This topic provides step-by-step instructions to assist developers in acquiring the LabKey Server source code, installing required components, and building LabKey Server from source. The instructions here are written for a Windows machine. To set up development on a Mac/OSX machine, use these instructions in conjunction with the topic: Set up OSX for LabKey Development.

• Obtain the LabKey Source Files
• Git Client
• Clone Repositories
• Development Machine Directory Structure
• Check Out Desired Branch
• Other Required Prerequisites
• Java
• PostgreSQL Database
• Gradle Configuration
• Environment Variables and System PATH
• Set Up the LabKey Project in IntelliJ
• Build and Run LabKey
• Optional Additions
• Test Automation
• Additional Modules
• Manage GitHub Modules via IntelliJ
• Generate Reference Docs
• Install R
• Run the Basic Test Suite
• Update a Development Machine
• Update to Use Embedded Tomcat
• Update Java
• Update IntelliJ
• Update LabKey Source Code

• Troubleshooting

Obtain the LabKey Source Files

The LabKey Server source files are stored in multiple GitHub repositories. In these steps, you'll establish and name a new local enlistment location on your machine, which we refer to here as <LK_ENLISTMENT>. One option is to use "C:\labkey\labkeyEnlistment".

Install a Git Client

You can use GitHub Desktop or another Git client.

In addition, in order to use gradle commands like gitCheckout, you must obtain a GitHub personal access token (classic), then set the environment variable GIT_ACCESS_TOKEN to this value. When this variable is not set, some gradle commands will generate an error starting with something like this:

Problem executing request for repository data with query 'query {
    search(query: "org:LabKey  fork:true  archived:false ", type:REPOSITORY, first:100

Clone Repositories

Start by cloning the server repository from GitHub to create the <LK_ENLISTMENT> location. Then navigate to it and add the core repositories (and any additional modules required).

git clone https://github.com/LabKey/server.git labkeyEnlistment

LK_ENLISTMENT> cd server/modules

LK_ENLISTMENT\server\modules> git clone https://github.com/LabKey/platform.git

LK_ENLISTMENT\server\modules> git clone https://github.com/LabKey/commonAssays.git

If you need other modules maintained by LabKey, clone them into <LK_ENLISTMENT>\server\modules. See more information on addition of optional modules from other repositories below.

Development Machine Directory Structure

The basic hierarchy of a development machine should look like this. Many files and directories are not shown here.

LK_ENLISTMENT (C:\labkey\labkeyEnlistment) 
│
└───server
    │
    └───modules
        │
        ├───commonAssays
        │
        ├───platform
        │
        └───[additional LabKey premium modules if applicable]

Check Out Desired Branch

By default, the develop branch is checked out when a new repository is cloned. If you would like a different branch you will need to check out the desired branch in each repository as well as the root of your enlistment. You must keep all parts of your enlistment in sync. For example, to change a the default/minimal set of repositories to the release24.7 branch, start from LK_ENLISTMENT:

LK_ENLISTMENT> cd server/modules/platform
LK_ENLISTMENT\server\modules\platform> git checkout release24.7
LK_ENLISTMENT\server\modules\platform> cd ../commonAssays
LK_ENLISTMENT\server\modules\commonAssays> git checkout release24.7
LK_ENLISTMENT\server\modules\commonAssays> cd ../../..
LK_ENLISTMENT> git checkout release24.7

You can simultaneously check out all of your repositories to a given branch by using both of the following commands from the root of your enlistment in this order. However, you cannot use these commands until after you have set the environment variable GIT_ACCESS_TOKEN:

LK_ENLISTMENT> gradlew gitCheckout -Pbranch=release24.7
LK_ENLISTMENT> git checkout release24.7

Install Required Prerequisites

Java

PostgreSQL Database

Node.js and npm

The LabKey build depends on Node.js and the node package manager (npm). The build process automatically downloads and installs the versions of npm and node that it requires. You should not install npm and node yourself. For details on the Node.js dependency and build-prod script requirement, see Node.js Build Dependency.

Gradle Configuration

Please note that you do not need to install Gradle. LabKey uses the gradle wrapper to make updating version numbers easier. The gradle wrapper script (either gradlew or gradlew.bat) is included and is already in the <LK_ENLISTMENT> directory.

However, you must:

1. Create a global gradle.properties file that contains global settings for your Gradle build. Your enlistment includes a template file to use.
2. Also edit the separate <LK_ENLISTMENT>/gradle.properties file to enable options required for developing locally.

If you are a Premium Edition subscriber, you may also want to generate an Artifactory Identity Token and add it to your global gradle.properties file at this time.

Environment Variables and System PATH

Set Up the LabKey Project in IntelliJ

The LabKey development team develops LabKey using IntelliJ IDEA.

Install IntelliJ

Configure the LabKey Project in IntelliJ

• Run the following gradle commands (in the <LK_ENLISTMENT> directory) to configure the workspace and other default settings:

gradlew ijWorkspaceSetup
gradlew ijConfigure

• Open the LabKey project in IntelliJ:
• Launch IntelliJ.
• If your IntelliJ install is brand new, you will see the "Welcome to IntelliJ" pop up screen. Click Open.
• If you have previously installed IntelliJ, select File > Open.
• Select your <LK_ENLISTMENT> directory. You may need to confirm you trust the directory.
• If you are developing with TypeScript, add a Path Variable. Most users will not need to do this.
• Click the four-bars menu in the upper left, the select File > Settings > Appearance & Behavior > Path Variables.
• Click the plus icon in the upper right.
• Add a NODE_PATH variable and set it to the path to the node binary that is installed by the build, for example, <LK_ENLISTMENT>\build\.node\node-v#.#.##\bin\node
• Click OK to close the Settings window.
• Configure the Target JDK
• In IntelliJ, select File > Project Structure.
• Under Project Settings, click Project.
• Under SDK select Add JDK...
• Browse to the path of your JDK, for example, (C:\labkey\apps\jdk-##), and click OK.
• Click Edit. Change the name of the JDK to "labkey".
• Click OK to close the Project Structure window.

• Open the Gradle tool window at View > Tool Windows > Gradle.
• Click the refresh icon. This will take as much as 5-20 minutes. You should start seeing messages about its progress. If not, something is probably hung up. Wait for this sync to complete before progressing with further IntelliJ configuration steps.

Set the Run/Debug Configuration by opening the "Run" menu. It may initially read "GWT" or something else.

• Users of IntelliJ Community should select LabKey Embedded Tomcat Dev.
• Users of IntelliJ Ultimate should select Spring Boot LabKey Embedded Tomcat Dev.

Be sure that IntelliJ has enough heap memory. The default max is OK if you’re just dealing with the core modules, but you will likely need to raise the limit if you’re adding in custom modules, 3GB seems sufficient.

You may also want to increase the number of open tabs IntelliJ will support. The default is 10, and depending on your working process, you may see tabs disappear unexpectedly. Select Window > Editor Tabs > Configure Editor Tabs... and scroll down to the Closing Policy section to increase the number of tabs you can have open in IntelliJ at one time.

Build and Run LabKey

Configure the pg.properties File

The LabKey source includes a configuration file for use with PostgreSQL (pg.properties), specifying JDBC settings, including URL, port, username, password, etc.

Run pickPg

When you run 'pickPg', the values that you've provided in both <LK_ENLISTMENT>/configs/application.properties and the pg.properties file in the same location will be copied into the <LK_ENLISTMENT>/build/deploy/embedded/config/application.properties file, overwriting previous values.

Build LabKey

To learn more about the build process, the various build tasks available, and how the source code is transformed into deployed modules, see Build LabKey from Source. In some cases, you may find it efficient to build all or part of your server using pre-built artifacts rather than source.

To control which modules are included in the build, see Customize the Build.

Run LabKey Server

While you are debugging, you can usually make changes, rebuild, and redeploy LabKey to the server without stopping and restarting Tomcat. Occasionally you may encounter errors that do require stopping and restarting Tomcat.

Optional Additions

Test Automation

To be able to run automated tests, you also need to clone the testAutomation repository into the server directory under <LK_ENLISTMENT>. Do not clone this repo into the 'modules' directory.:

LK_ENLISTMENT\server> git clone https://github.com/LabKey/testAutomation.git

An abbreviated map of your finished enlistment, showing the main folder structure (with many other elements omitted):

LK_ENLISTMENT 
│
└───server
    │
    ├───modules
    │   ├───commonAssays
    │   ├───platform
    │   └───[optional additional modules]
    │
    └───testAutomation

Additional Modules

Many optional modules are available from the LabKey repository on GitHub. To include these modules in your build, install a Git client and clone individual modules into the LabKey Server source.

• To add a GitHub module to your build, clone the desired module into <LK_ENLISTMENT>/labkey/server/modules. For example, to add the 'reagent' module:

LK_ENLISTMENT\server\modules>git clone https://github.com/LabKey/reagent.git

Note that you can get the URL by going to the module page on GitHub (for example, https://github.com/LabKey/reagent), clicking Clone or Download, and copying the displayed URL.

Manage GitHub Modules via IntelliJ

Once you have cloned a GitHub module, you can have IntelliJ handle any updates:

To add the GitHub-based module to IntelliJ (and have IntelliJ generate an .iml file for the module):

• Edit your settings.gradle file to include the new module
• In IntelliJ, open the Gradle tool window at View > Tool Windows > Gradle.
• Refresh the Gradle window by clicking the arrow circle in the upper left of the Gradle window

• To have IntelliJ handle source updates from GitHub, go to File > Settings (or IntelliJ > Preferences).
• Select Version Control.
• In the Directory panel, scroll down to the Unregistered roots section, select the module, and click the Plus icon in the lower left.
• In the Directory panel, select the target module and set its VCS source as Git, if necessary.
• Note that IntelliJ sometimes thinks that subdirectories of the module, like module test suites, have their sources in SVN instead of Git. You can safely delete these SVN sources using the Directory panel.
• To sync to a particular GitHub branch: in IntelliJ, go to VCS > Git > Branches. A popup menu will appear listing the available Git modules. Use the popup menu to select the branch to sync to.

Generate Reference Docs

To generate documentation, such as the XML reference and Javadoc, clone the tools repository into the enlistment root (to create a sibling of the /server directory):

LK_ENLISTMENT>git clone https://github.com/LabKey/tools.git

Then you can run the documentation generation tasks, such as:

./gradlew :server:xsddoc
./gradlew :server:javadoc

Install R

Run the Basic Test Suite

• Run this command from your <LK_ENLISTMENT> directory, to initiate automated tests of LabKey's basic functionality:

  gradlew :server:testAutomation:uiTest -Psuite=DRT

Update a Development Machine

Update to Use Embedded Tomcat

Beginning with LabKey version 24.3, an embedded version of Tomcat 10 is now used. To update a development machine to use embedded Tomcat for the first time, follow these steps:

Update Java

To update the JDK in the future, follow these steps.

1. Repeat the steps for originally installing the JDK. Install the new version in a location parallel to the old.
2. Update your system variable JAVA_HOME to point to the new location, unless you changed folder names so that the location path is the same.
3. Update the IntelliJ > File > Project Structure > SDK > labkey home path to point to the new location. Rename the existing 'labkey' to its version number. Add the new JDK, then rename the new version "labkey".
4. If necessary, add any additional flags to the VM options section. For example, when using Java 16 or 17, you'll need these flags in your JVM switches:

   --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.desktop/java.awt.font=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED

Update IntelliJ

Over time, you will want to update IntelliJ to stay current with features and enhancements. You can use the Help > Check for updates... option from within the application. Watch for installation alerts about any .idea files you may have edited. Note that updates may be incremental and after updating, a second more recent update may be available.

After updating, recheck the IntelliJ settings described in the above section.

Note that if you are developing against an older version of LabKey, you may need to use a version of IntelliJ that was available at the time of that release. We do not test newer releases of IntelliJ with older versions of LabKey.

Update LabKey Source Code

If you're working against the develop branch, you will continuously use "Update Project" to keep your local enlistment in sync. Where this option is in IntelliJ varies by version. It may be represented with a down-left-arrow, on the "four bars > Git" menu or accessed using a shortcut like ctrl-T.

If you have been synchronized to a specific branch (such as "release23.11"), you can change to a newer branch by using the recursive gitCheckout option. Use "Update Project" first, then:

LK_ENLISTMENT> gradlew gitCheckout -Pbranch=release24.7
LK_ENLISTMENT> git checkout release24.7

Related Topics

• Tutorial: Hello World Module
• Build LabKey from Source
• Customize the Build
• Git Ignore Configurations
• Build Offline
• Gradle Cleaning
• Gradle Properties
• Gradle: How to Add Modules
• Gradle: Declare Dependencies
• Gradle Tips and Tricks
• Run Automated Tests
• Create Production Builds
• Gradle Build Overview
• Node.js Build Dependency
• Machine Security
• Set up OSX for LabKey Development
• Troubleshoot Development Machines