• Obtain the LabKey Source Files
• Developer Tools and Git Client
• Clone Repositories
• Development Machine Directory Structure
• Check Out Desired Branch
• Other Required Prerequisites
• Java
• PostgreSQL Database
• Gradle Configuration
• Environment Variables and System PATH
• Set Up the LabKey Project in IntelliJ
• Build and Run LabKey
• Optional Additions
• Test Automation
• Additional Modules
• Manage GitHub Modules via IntelliJ
• Generate Reference Docs
• Install R
• Run the Basic Test Suite
• Update a Development Machine
• Update to Use Embedded Tomcat
• Update Java
• Update IntelliJ
• Update LabKey Source Code

• Troubleshooting

Obtain the LabKey Source Files

The LabKey Server source files are stored in multiple GitHub repositories. In these steps, you'll establish and name a new local enlistment location on your machine, which we refer to here as <LK_ENLISTMENT>. One option is to use "~/labkey/labkeyEnlistment".

Install Developer Tools and a Git Client

On a Mac, install the Apple command line developer tools, which include git. You can also use GitHub Desktop or another Git client.

In addition, in order to use gradle commands like gitCheckout, you must obtain a GitHub personal access token (classic), then set the environment variable GIT_ACCESS_TOKEN to this value. When this variable is not set, some gradle commands will generate an error starting with something like this:

Problem executing request for repository data with query 'query {
    search(query: "org:LabKey  fork:true  archived:false ", type:REPOSITORY, first:100

Clone Repositories

Start by cloning the server repository from GitHub to create the <LK_ENLISTMENT> location. Then navigate to it and add the core repositories (and any additional modules required).

git clone https://github.com/LabKey/server.git labkeyEnlistment

LK_ENLISTMENT> cd server/modules

LK_ENLISTMENT/server/modules> git clone https://github.com/LabKey/platform.git

LK_ENLISTMENT/server/modules> git clone https://github.com/LabKey/commonAssays.git

If you need other modules maintained by LabKey, clone them into <LK_ENLISTMENT>/server/modules. See more information on addition of optional modules from other repositories below.

Development Machine Directory Structure

The basic hierarchy of a development machine should look like this. Many files and directories are not shown here.

LK_ENLISTMENT (~/labkey/labkeyEnlistment)
│
└───server
    │
    └───modules
        │
        ├───commonAssays
        │
        ├───platform
        │
        └───[additional LabKey premium modules if applicable]

Check Out Desired Branch

By default, the develop branch is checked out when a new repository is cloned. If you would like a different branch you will need to check out the desired branch in each repository as well as the root of your enlistment. You must keep all parts of your enlistment in sync. For example, to change a the default/minimal set of repositories to the release26.3 branch, start from LK_ENLISTMENT:

LK_ENLISTMENT> cd server/modules/platform
LK_ENLISTMENT/server/modules/platform> git checkout release26.3
LK_ENLISTMENT/server/modules/platform> cd ../commonAssays
LK_ENLISTMENT/server/modules/commonAssays> git checkout release26.3
LK_ENLISTMENT/server/modules/commonAssays> cd ../../..
LK_ENLISTMENT> git checkout release26.3

You can simultaneously check out all of your repositories to a given branch by using both of the following commands from the root of your enlistment in this order. However, you cannot use these commands until after you have set the environment variable GIT_ACCESS_TOKEN:

LK_ENLISTMENT> ./gradlew gitCheckout -Pbranch=release26.3
LK_ENLISTMENT> git checkout release26.3

Install Required Prerequisites

Java

PostgreSQL Database

You can install PostgreSQL directly on your machine, or run it in a Docker container. A container keeps the database isolated from the rest of your system, and makes it easy to run a different PostgreSQL version later.

Node.js and npm

The LabKey build depends on Node.js and the node package manager (npm). The build process automatically downloads and installs the versions of npm and node that it requires. You should not install npm and node yourself. For details on the Node.js dependency and build-prod script requirement, see Node.js Build Dependency.

Gradle Configuration

Please note that you do not need to install Gradle. LabKey uses the gradle wrapper to make updating version numbers easier. The gradle wrapper script (either gradlew or gradlew.bat) is included and is already in the <LK_ENLISTMENT> directory.

However, you must:

1. Create a global gradle.properties file that contains global settings for your Gradle build. Your enlistment includes a template file to use.
2. Also edit the separate <LK_ENLISTMENT>/gradle.properties file to enable options required for developing locally.

If you are a Premium Edition subscriber, you may also want to generate an Artifactory Identity Token and add it to your global gradle.properties file at this time. Then use the same token to create a .npmrc file, which some modules require to obtain resources from Artifactory.

Environment Variables and System PATH

Set environment variables in your shell's startup file: ~/.zshrc for zsh (the default shell on a Mac), or ~/.bash_profile for bash.

Set Up the LabKey Project in IntelliJ

The LabKey development team develops LabKey using IntelliJ IDEA.

Install IntelliJ

Configure the LabKey Project in IntelliJ

./gradlew ijWorkspaceSetup
./gradlew ijConfigure

You may also want to increase the number of open tabs IntelliJ will support. The default is 10, and depending on your working process, you may see tabs disappear unexpectedly. Select Window > Editor Tabs > Configure Editor Tabs... and scroll down to the Closing Policy section to increase the number of tabs you can have open in IntelliJ at one time.

Build and Run LabKey

Configure the pg.properties File

The LabKey source includes a configuration file for use with PostgreSQL (pg.properties), specifying JDBC settings, including URL, port, username, password, etc.

Run pickPg

When you run 'pickPg', the values that you've provided in both <LK_ENLISTMENT>/configs/application.properties and the pg.properties file in the same location will be copied into the <LK_ENLISTMENT>/build/deploy/embedded/config/application.properties file, overwriting previous values. It is important to remember that you will need to repeat this task any time you change values in either of these properties files.

Build LabKey

To learn more about the build process, the various build tasks available, and how the source code is transformed into deployed modules, see Build LabKey from Source. In some cases, you may find it efficient to build all or part of your server using pre-built artifacts rather than source.

To control which modules are included in the build, see Customize the Build.

Run LabKey Server

While you are debugging, you can usually make changes, rebuild, and redeploy LabKey to the server without stopping and restarting Tomcat. Occasionally you may encounter errors that do require stopping and restarting Tomcat.

Optional Additions

Test Automation

To be able to run automated tests, you also need to clone the testAutomation repository into the server directory under <LK_ENLISTMENT>. Do not clone this repo into the 'modules' directory.:

LK_ENLISTMENT/server> git clone https://github.com/LabKey/testAutomation.git

An abbreviated map of your finished enlistment, showing the main folder structure (with many other elements omitted):

LK_ENLISTMENT
│
└───server
    │
    ├───modules
    │   ├───commonAssays
    │   ├───platform
    │   └───[optional additional modules]
    │
    └───testAutomation

Additional Modules

Many optional modules are available from the LabKey repository on GitHub. To include these modules in your build, install a Git client and clone individual modules into the LabKey Server source.

• To add a GitHub module to your build, clone the desired module into <LK_ENLISTMENT>/server/modules. For example, to add the 'reagent' module:

LK_ENLISTMENT/server/modules> git clone https://github.com/LabKey/reagent.git

Note that you can get the URL by going to the module page on GitHub (for example, https://github.com/LabKey/reagent), clicking Clone or Download, and copying the displayed URL.

Manage GitHub Modules via IntelliJ

Once you have cloned a GitHub module, you can have IntelliJ handle any updates:

To add the GitHub-based module to IntelliJ (and have IntelliJ generate an .iml file for the module):

• Edit your settings.gradle file to include the new module
• In IntelliJ, open the Gradle tool window at View > Tool Windows > Gradle.
• Refresh the Gradle window by clicking the arrow circle in the upper left of the Gradle window

• To have IntelliJ handle source updates from GitHub, go to IntelliJ IDEA > Settings.
• Select Version Control.
• In the Directory panel, scroll down to the Unregistered roots section, select the module, and click the Plus icon in the lower left.
• In the Directory panel, select the target module and set its VCS source as Git, if necessary.
• Note that IntelliJ sometimes thinks that subdirectories of the module, like module test suites, have their sources in SVN instead of Git. You can safely delete these SVN sources using the Directory panel.
• To sync to a particular GitHub branch: in IntelliJ, go to VCS > Git > Branches. A popup menu will appear listing the available Git modules. Use the popup menu to select the branch to sync to.

Generate Reference Docs

To generate documentation, such as the XML reference and Javadoc, clone the tools repository into the enlistment root (to create a sibling of the /server directory):

LK_ENLISTMENT> git clone https://github.com/LabKey/tools.git

Then you can run the documentation generation tasks, such as:

./gradlew :server:xsddoc
./gradlew :server:javadoc

Install R

Run the Basic Test Suite

• Run this command from your <LK_ENLISTMENT> directory, to initiate automated tests of LabKey's basic functionality:

  ./gradlew :server:testAutomation:uiTest -Psuite=DRT

Update a Development Machine

Update to Use Embedded Tomcat

LabKey includes an embedded version of Tomcat. To update a development machine to use embedded Tomcat for the first time, follow these steps:

Update Java

To update the JDK in the future, follow these steps.

1. Repeat the steps for originally installing the JDK. Install the new version in a location parallel to the old.
2. Update JAVA_HOME in your shell's startup file (for example, change the version number passed to /usr/libexec/java_home), then open a new Terminal window.
3. Update the IntelliJ > File > Project Structure > SDK > labkey home path to point to the new location. Rename the existing "labkey" to its version number. Add the new JDK, then rename the new version "labkey".
4. Add additional flags to the VM options section. For example, Java 17 requires these flags in your JVM switches:

   --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.desktop/java.awt.font=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED

Update IntelliJ

Over time, you will want to update IntelliJ to stay current with features and enhancements. You can use the IntelliJ IDEA > Check for Updates... option from within the application. Watch for installation alerts about any .idea files you may have edited. Note that updates may be incremental and after updating, a second more recent update may be available.

After updating, recheck the IntelliJ settings described in the above section.

Note that if you are developing against an older version of LabKey, you may need to use a version of IntelliJ that was available at the time of that release. We do not test newer releases of IntelliJ with older versions of LabKey.

Update LabKey Source Code

If you're working against the develop branch, you will continuously use "Update Project" to keep your local enlistment in sync. Where this option is in IntelliJ varies by version. It may be represented with a down-left-arrow, on the Git menu, or accessed using the shortcut Command-T.

If you have been synchronized to a specific branch (such as "release25.11"), you can change to a newer branch by using the recursive gitCheckout option. Use "Update Project" first, then:

LK_ENLISTMENT> ./gradlew gitCheckout -Pbranch=release26.3
LK_ENLISTMENT> git checkout release26.3

Related Topics

• Tutorial: Hello World Module
• Build LabKey from Source
• Customize the Build
• Git Ignore Configurations
• Build Offline
• Gradle Cleaning
• Gradle Properties
• Gradle: How to Add Modules
• Gradle: Declare Dependencies
• Gradle Tips and Tricks
• Run Automated Tests
• Create Production Builds
• Gradle Build Overview
• Node.js Build Dependency
• Machine Security
• Set Up a Development Machine: Windows
• Troubleshoot Development Machines
• Advanced Topics