This topic provides step-by-step instructions to assist developers in acquiring the LabKey Server source code, installing required components, and building LabKey Server from source. The instructions here are written for a
Windows machine. To set up development on a
Mac/OSX machine, use these instructions in conjunction with the topic:
Set up OSX for LabKey Development.
Additional Topic:
Obtain the LabKey Source Files
The LabKey Server source files are stored in multiple GitHub repositories. In these steps, you'll establish and name a new local enlistment location on your machine, which we refer to here as <LK_ENLISTMENT>. One option is to use "C:\labkey\labkeyEnlistment".
Install a Git Client
You can use GitHub Desktop or another Git client.
- Install GitHub Desktop:
- Install a command-line git option for your platform:
In addition, in order to use
gradle commands like gitCheckout, you must
obtain a GitHub personal access token (classic), then set the environment variable GIT_ACCESS_TOKEN to this value. When this variable is not set, some gradle commands will generate an error starting with something like this:
Problem executing request for repository data with query 'query {
search(query: "org:LabKey fork:true archived:false ", type:REPOSITORY, first:100
Clone Repositories
Start by cloning the
server repository from GitHub to create the <LK_ENLISTMENT> location. Then navigate to it and add the core repositories (and any additional modules required).
- Begin a level "above" where you want the <LK_ENLISTMENT>. For example, you might start in a "C:\labkey\" directory.
- Clone the server repository from git into a directory with your <LK_ENLISTMENT> name, such as labkeyEnlistment.
git clone https://github.com/LabKey/server.git labkeyEnlistment
- Navigate to that new <LK_ENLISTMENT> location ("cd labkeyEnlistment").
- Clone the platform and commonAssay repositories into <LK_ENLISTMENT>\server\modules.
LK_ENLISTMENT> cd server/modules
LK_ENLISTMENT\server\modules> git clone https://github.com/LabKey/platform.git
LK_ENLISTMENT\server\modules> git clone https://github.com/LabKey/commonAssays.git
If you need other modules maintained by LabKey, clone them into <LK_ENLISTMENT>\server\modules. See more information on
addition of optional modules from other repositories below.
Note: LabKey uses Artifactory to maintain the premium modules of LabKey Server. If you are using a Premium Edition, please follow
these setup instructions for your Artifactory account.
Development Machine Directory Structure
The basic hierarchy of a development machine should look like this. Many files and directories are not shown here.
LK_ENLISTMENT (C:\labkey\labkeyEnlistment)
│
└───server
│
└───modules
│
├───commonAssays
│
├───platform
│
└───[additional LabKey premium modules if applicable]
Check Out Desired Branch
By default, the
develop branch is checked out when a new repository is cloned. If you would like a different branch you will need to check out the desired branch in each repository as well as the root of your enlistment. You must keep all parts of your enlistment in sync. For example, to change a the default/minimal set of repositories to the release24.7 branch, start from LK_ENLISTMENT:
LK_ENLISTMENT> cd server/modules/platform
LK_ENLISTMENT\server\modules\platform> git checkout release24.7
LK_ENLISTMENT\server\modules\platform> cd ../commonAssays
LK_ENLISTMENT\server\modules\commonAssays> git checkout release24.7
LK_ENLISTMENT\server\modules\commonAssays> cd ../../..
LK_ENLISTMENT> git checkout release24.7
You can simultaneously check out
all of your repositories to a given branch by using both of the following commands from the root of your enlistment in this order. However, you cannot use these commands until after you have
set the environment variable GIT_ACCESS_TOKEN:
LK_ENLISTMENT> gradlew gitCheckout -Pbranch=release24.7
LK_ENLISTMENT> git checkout release24.7
Install Required Prerequisites
Java
PostgreSQL Database
Node.js and npm
The LabKey build depends on Node.js and the node package manager (npm). The build process automatically downloads and installs the versions of npm and node that it requires. You should not install npm and node yourself. For details on the Node.js dependency and build-prod script requirement, see
Node.js Build Dependency.
Gradle Configuration
Please note that you do
not need to install Gradle. LabKey uses the gradle wrapper to make updating version numbers easier. The gradle wrapper script (either gradlew or gradlew.bat) is included and is already in the <LK_ENLISTMENT> directory.
However, you must:
- Create a global gradle.properties file that contains global settings for your Gradle build. Your enlistment includes a template file to use.
- Also edit the separate <LK_ENLISTMENT>/gradle.properties file to enable options required for developing locally.
Global gradle.properties file for your user account:
- Create a .gradle directory in your home directory.
- On Windows the home directory is typically: C:\Users\<you>.
- Create a .gradle folder there.
- If the file explorer does not allow you to create a folder beginning with a period, you may need to navigate to that location and use a command prompt to type:
- Create a gradle.properties file in that new location using the following process:
- Copy the file <LK_ENLISTMENT>/gradle/global_gradle.properties_template to C:\Users\<you>\.gradle.
- Rename the file so it is named just gradle.properties.
Edit <LK_ENLISTMENT>/gradle.properties:
- In the <LK_ENLISTMENT> directory, edit "gradle.properties" to uncomment this line:
If you are a Premium Edition subscriber, you may also want to
generate an Artifactory Identity Token and add it to your global gradle.properties file at this time.
Environment Variables and System PATH
- JAVA_HOME
- Create or modify the system environment variable JAVA_HOME so it points to your JDK installation location (for example, C:\labkey\apps\jdk-##).
- PATH
- Add the JDK to your system PATH. Using the JAVA_HOME variable will simplify upgrades later:
- Add the <LK_ENLISTMENT>\build\deploy\bin location to your system PATH. This directory does not exist yet, but add it to the path anyway. The Gradle build uses this directory to store binaries used by the server, such as graphvis, dot, and proteomics tools.
LK_ENLISTMENT\build\deploy\bin
- For example, C:\labkey\labkeyEnlistment\build\deploy\bin.
Set Up the LabKey Project in IntelliJ
The LabKey development team develops LabKey using IntelliJ IDEA.
Install IntelliJ
- Download and install the latest version of IntelliJ IDEA. Either the Community or Ultimate Edition will work.
Note that if you are developing against an older version of LabKey, you may need to use a version of IntelliJ that was available at the time of that release. We do not test newer releases of IntelliJ with older versions of LabKey.
Configure the LabKey Project in IntelliJ
- Run the following gradle commands (in the <LK_ENLISTMENT> directory) to configure the workspace and other default settings:
gradlew ijWorkspaceSetup
gradlew ijConfigure
- Open the LabKey project in IntelliJ:
- Launch IntelliJ.
- If your IntelliJ install is brand new, you will see the "Welcome to IntelliJ" pop up screen. Click Open.
- If you have previously installed IntelliJ, select File > Open.
- Select your <LK_ENLISTMENT> directory. You may need to confirm you trust the directory.
- If you are developing with TypeScript, add a Path Variable. Most users will not need to do this.
- Click the four-bars menu in the upper left, the select File > Settings > Appearance & Behavior > Path Variables.
- Click the plus icon in the upper right.
- Add a NODE_PATH variable and set it to the path to the node binary that is installed by the build, for example, <LK_ENLISTMENT>\build\.node\node-v#.#.##\bin\node
- Click OK to close the Settings window.
- Configure the Target JDK
- In IntelliJ, select File > Project Structure.
- Under Project Settings, click Project.
- Under SDK select Add JDK...
- Browse to the path of your JDK, for example, (C:\labkey\apps\jdk-##), and click OK.
- Click Edit. Change the name of the JDK to "labkey".
- Click OK to close the Project Structure window.
- Open the Gradle tool window at View > Tool Windows > Gradle.
- Click the refresh icon. This will take as much as 5-20 minutes. You should start seeing messages about its progress. If not, something is probably hung up. Wait for this sync to complete before progressing with further IntelliJ configuration steps.
Set the
Run/Debug Configuration by opening the "Run" menu. It may initially read "GWT" or something else.
- Users of IntelliJ Community should select LabKey Embedded Tomcat Dev.
- Users of IntelliJ Ultimate should select Spring Boot LabKey Embedded Tomcat Dev.
Be sure that IntelliJ has enough heap memory. The default max is OK if you’re just dealing with the core modules, but you will
likely need to raise the limit if you’re adding in custom modules, 3GB seems sufficient.
You may also want to increase the number of open tabs IntelliJ will support. The default is 10, and depending on your working process, you may see tabs disappear unexpectedly. Select
Window > Editor Tabs > Configure Editor Tabs... and scroll down to the
Closing Policy section to increase the number of tabs you can have open in IntelliJ at one time.
Build and Run LabKey
Configure the pg.properties File
The LabKey source includes a configuration file for use with PostgreSQL (
pg.properties), specifying JDBC settings, including URL, port, username, password, etc.
Note that for users of
Microsoft SQL Server, the file will be
mssql.properties and the task for configuration will be "pickMSSQL" instead of "pickPg".
- Open the file <LK_ENLISTMENT>/server/configs/pg.properties
- Edit the file, adding your values for the jdbcUser and jdbcPassword. (This password is the one you specified when installing PostgreSQL.)
Run pickPg
- In a command window, go to <LK_ENLISTMENT>
- Run this task to configure the application.properties file:
When you run 'pickPg', the values that you've provided in both <LK_ENLISTMENT>/configs/application.properties and the pg.properties file in the same location will be copied into the <LK_ENLISTMENT>/build/deploy/embedded/config/application.properties file, overwriting previous values.
Build LabKey
To learn more about the build process, the various build tasks available, and how the source code is transformed into deployed modules, see
Build LabKey from Source. In some cases, you may find it efficient to
build all or part of your server using pre-built artifacts rather than source.
- On the command line, go to the <LK_ENLISTMENT> directory, and invoke the gradle build task:
To control which modules are included in the build, see
Customize the Build.
Run LabKey Server
To run and debug LabKey:
- Click either the or icon to run or debug either 'Spring Boot LabKey Embedded Tomcat Dev' or 'LabKey Embedded Tomcat Dev' (depending on your edition of intelliJ). Hover text will guide you.
- Once LabKey Server starts up successfully, navigate your browser to http://localhost:8080/ to begin debugging.
- You should see the LabKey application and be invited to create a new user account the first time you start up using a new database.
While you are debugging, you can usually make changes, rebuild, and redeploy LabKey to the server without stopping and restarting Tomcat. Occasionally you may encounter errors that do require stopping and restarting Tomcat.
Optional Additions
Test Automation
To be able to run automated tests, you also need to clone the
testAutomation repository into the server directory under <LK_ENLISTMENT>.
Do not clone this repo into the 'modules' directory.:
LK_ENLISTMENT\server> git clone https://github.com/LabKey/testAutomation.git
An abbreviated map of your finished enlistment, showing the main folder structure (with many other elements omitted):
LK_ENLISTMENT
│
└───server
│
├───modules
│ ├───commonAssays
│ ├───platform
│ └───[optional additional modules]
│
└───testAutomation
Additional Modules
Many optional modules are available from the
LabKey repository on GitHub. To include these modules in your build, install a Git client and clone individual modules into the LabKey Server source.
- To add a GitHub module to your build, clone the desired module into <LK_ENLISTMENT>/labkey/server/modules. For example, to add the 'reagent' module:
LK_ENLISTMENT\server\modules>git clone https://github.com/LabKey/reagent.git
Note that you can get the URL by going to the module page on GitHub (for example, https://github.com/LabKey/reagent), clicking
Clone or Download, and copying the displayed URL.
Manage GitHub Modules via IntelliJ
Once you have cloned a GitHub module, you can have IntelliJ handle any updates:
To add the GitHub-based module to IntelliJ (and have IntelliJ generate an .iml file for the module):
- Edit your settings.gradle file to include the new module
- In IntelliJ, open the Gradle tool window at View > Tool Windows > Gradle.
- Refresh the Gradle window by clicking the arrow circle in the upper left of the Gradle window
To update the GitHub-based module using IntelliJ:
- To have IntelliJ handle source updates from GitHub, go to File > Settings (or IntelliJ > Preferences).
- Select Version Control.
- In the Directory panel, scroll down to the Unregistered roots section, select the module, and click the Plus icon in the lower left.
- In the Directory panel, select the target module and set its VCS source as Git, if necessary.
- Note that IntelliJ sometimes thinks that subdirectories of the module, like module test suites, have their sources in SVN instead of Git. You can safely delete these SVN sources using the Directory panel.
- To sync to a particular GitHub branch: in IntelliJ, go to VCS > Git > Branches. A popup menu will appear listing the available Git modules. Use the popup menu to select the branch to sync to.
If you have added a new module to your enlistment, be sure to
customize the build to include it in your Gradle project and then refresh the Gradle window to incorporate it into IntelliJ, as described above.
Generate Reference Docs
To generate documentation, such as the XML reference and Javadoc, clone the tools repository into the enlistment root (to create a sibling of the /server directory):
LK_ENLISTMENT>git clone https://github.com/LabKey/tools.git
Then you can run the documentation generation tasks, such as:
./gradlew :server:xsddoc
./gradlew :server:javadoc
Install R
Run the Basic Test Suite
- Run this command from your <LK_ENLISTMENT> directory, to initiate automated tests of LabKey's basic functionality:
gradlew :server:testAutomation:uiTest -Psuite=DRT
Other automated tests are available. For details, see
Run Automated Tests. Note that
'R' must first be configured in order to run some tests.
Update a Development Machine
Update to Use Embedded Tomcat
Beginning with LabKey version 24.3, an embedded version of Tomcat 10 is now used. To update a development machine to use embedded Tomcat for the first time, follow these steps:
- Get the latest changes from all key repos (server, platform, etc.) with git pull. Don't forget to also update the root of your enlistment. Using the two source upgrade steps below is recommended.
- Update IntelliJ with the latest run configurations with gradlew ijConfigure
- Within your <LK_ENLISTMENT> level gradle.properties file, uncomment the useEmbeddedTomcat and useLocalBuild properties
- gradlew cleanBuild
- gradlew pickPg (or gradlew pickMSSQL)
- gradlew deployApp
- Within IntelliJ, do a Gradle Refresh in the Gradle Window.
Update Java
To update the JDK in the future, follow these steps.
- Repeat the steps for originally installing the JDK. Install the new version in a location parallel to the old.
- Update your system variable JAVA_HOME to point to the new location, unless you changed folder names so that the location path is the same.
- Update the IntelliJ > File > Project Structure > SDK > labkey home path to point to the new location. Rename the existing 'labkey' to its version number. Add the new JDK, then rename the new version "labkey".
- If necessary, add any additional flags to the VM options section. For example, when using Java 16 or 17, you'll need these flags in your JVM switches:
--add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.desktop/java.awt.font=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED
Update IntelliJ
Over time, you will want to update IntelliJ to stay current with features and enhancements. You can use the
Help > Check for updates... option from within the application. Watch for installation alerts about any .idea files you may have edited. Note that updates may be incremental and after updating, a second more recent update may be available.
After updating, recheck the IntelliJ settings described in
the above section.
Note that if you are developing against an older version of LabKey, you may need to use a version of IntelliJ that was available at the time of that release. We do not test newer releases of IntelliJ with older versions of LabKey.
Update LabKey Source Code
If you're working against the develop branch, you will continuously use "Update Project" to keep your local enlistment in sync. Where this option is in IntelliJ varies by version. It may be represented with a down-left-arrow, on the "four bars > Git" menu or accessed using a shortcut like ctrl-T.
If you have been synchronized to a specific branch (such as "release23.11"), you can change to a newer branch by using the
recursive gitCheckout option. Use "Update Project" first, then:
LK_ENLISTMENT> gradlew gitCheckout -Pbranch=release24.7
LK_ENLISTMENT> git checkout release24.7
Related Topics
Premium Resources AvailableSubscribers to premium editions of LabKey Server can learn more about developing with LabKey with the developer topics and code examples listed on this page:
Learn more about premium editions