Set Up a Development Machine

Documentation

This topic provides step-by-step instructions to assist developers in acquiring the LabKey Server source code, installing required components, and building LabKey Server from source. The instructions here are written for a Windows machine. To set up development on a Mac/OSX machine, use these instructions in conjunction with the topic: Set up OSX for LabKey Development.

Additional Topic:

Obtain the LabKey Source Files

The LabKey Server source files are stored in multiple GitHub repositories. In these steps, you'll establish and name a new local enlistment location on your machine, which we refer to here as <LK_ENLISTMENT>. One option is to use "C:\labkey\labkeyEnlistment".

Install a Git Client

You can use GitHub Desktop or another Git client.

In addition, in order to use gradle commands like gitCheckout, you must obtain a GitHub personal access token (classic), then set the environment variable GIT_ACCESS_TOKEN to this value. When this variable is not set, some gradle commands will generate an error starting with something like this:

Problem executing request for repository data with query 'query {
search(query: "org:LabKey fork:true archived:false ", type:REPOSITORY, first:100

Clone Repositories

Start by cloning the server repository from GitHub to create the <LK_ENLISTMENT> location. Then navigate to it and add the core repositories (and any additional modules required).

  • Begin a level "above" where you want the <LK_ENLISTMENT>. For example, you might start in a "C:\labkey\" directory.
  • Clone the server repository from git into a directory with your <LK_ENLISTMENT> name, such as labkeyEnlistment.
git clone https://github.com/LabKey/server.git labkeyEnlistment
  • Navigate to that new <LK_ENLISTMENT> location ("cd labkeyEnlistment").
  • Clone the platform and commonAssay repositories into <LK_ENLISTMENT>\server\modules.
LK_ENLISTMENT> cd server/modules

LK_ENLISTMENT\server\modules> git clone https://github.com/LabKey/platform.git

LK_ENLISTMENT\server\modules> git clone https://github.com/LabKey/commonAssays.git

If you need other modules maintained by LabKey, clone them into <LK_ENLISTMENT>\server\modules. See more information on addition of optional modules from other repositories below.

Note: LabKey uses Artifactory to maintain the premium modules of LabKey Server. If you are using a Premium Edition, please follow these setup instructions for your Artifactory account.

Development Machine Directory Structure

The basic hierarchy of a development machine should look like this. Many files and directories are not shown here.

LK_ENLISTMENT (C:\labkey\labkeyEnlistment) 

└───server

└───modules

├───commonAssays

├───platform

└───[additional LabKey premium modules if applicable]

Check Out Desired Branch

By default, the develop branch is checked out when a new repository is cloned. If you would like a different branch you will need to check out the desired branch in each repository as well as the root of your enlistment. You must keep all parts of your enlistment in sync. For example, to change a the default/minimal set of repositories to the release24.7 branch, start from LK_ENLISTMENT:

LK_ENLISTMENT> cd server/modules/platform
LK_ENLISTMENT\server\modules\platform> git checkout release24.7
LK_ENLISTMENT\server\modules\platform> cd ../commonAssays
LK_ENLISTMENT\server\modules\commonAssays> git checkout release24.7
LK_ENLISTMENT\server\modules\commonAssays> cd ../../..
LK_ENLISTMENT> git checkout release24.7

You can simultaneously check out all of your repositories to a given branch by using both of the following commands from the root of your enlistment in this order. However, you cannot use these commands until after you have set the environment variable GIT_ACCESS_TOKEN:

LK_ENLISTMENT> gradlew gitCheckout -Pbranch=release24.7
LK_ENLISTMENT> git checkout release24.7

Install Required Prerequisites

Java

PostgreSQL Database

Node.js and npm

The LabKey build depends on Node.js and the node package manager (npm). The build process automatically downloads and installs the versions of npm and node that it requires. You should not install npm and node yourself. For details on the Node.js dependency and build-prod script requirement, see Node.js Build Dependency.

Gradle Configuration

Please note that you do not need to install Gradle. LabKey uses the gradle wrapper to make updating version numbers easier. The gradle wrapper script (either gradlew or gradlew.bat) is included and is already in the <LK_ENLISTMENT> directory.

However, you must:

  1. Create a global gradle.properties file that contains global settings for your Gradle build. Your enlistment includes a template file to use.
  2. Also edit the separate <LK_ENLISTMENT>/gradle.properties file to enable options required for developing locally.
Global gradle.properties file for your user account:
  • Create a .gradle directory in your home directory.
    • On Windows the home directory is typically: C:\Users\<you>.
    • Create a .gradle folder there.
    • If the file explorer does not allow you to create a folder beginning with a period, you may need to navigate to that location and use a command prompt to type:
      mkdir .gradle
  • Create a gradle.properties file in that new location using the following process:
    • Copy the file <LK_ENLISTMENT>/gradle/global_gradle.properties_template to C:\Users\<you>\.gradle.
    • Rename the file so it is named just gradle.properties.
Edit <LK_ENLISTMENT>/gradle.properties:
  • In the <LK_ENLISTMENT> directory, edit "gradle.properties" to uncomment this line:
    useLocalBuild

If you are a Premium Edition subscriber, you may also want to generate an Artifactory Identity Token and add it to your global gradle.properties file at this time.

Environment Variables and System PATH

  • JAVA_HOME
    • Create or modify the system environment variable JAVA_HOME so it points to your JDK installation location (for example, C:\labkey\apps\jdk-##).
  • PATH
    • Add the JDK to your system PATH. Using the JAVA_HOME variable will simplify upgrades later:
      %JAVA_HOME%\bin
    • Add the <LK_ENLISTMENT>\build\deploy\bin location to your system PATH. This directory does not exist yet, but add it to the path anyway. The Gradle build uses this directory to store binaries used by the server, such as graphvis, dot, and proteomics tools.
      LK_ENLISTMENT\build\deploy\bin
    • For example, C:\labkey\labkeyEnlistment\build\deploy\bin.

Set Up the LabKey Project in IntelliJ

The LabKey development team develops LabKey using IntelliJ IDEA.

Install IntelliJ

  • Download and install the latest version of IntelliJ IDEA. Either the Community or Ultimate Edition will work.
Note that if you are developing against an older version of LabKey, you may need to use a version of IntelliJ that was available at the time of that release. We do not test newer releases of IntelliJ with older versions of LabKey.

Configure the LabKey Project in IntelliJ

  • Run the following gradle commands (in the <LK_ENLISTMENT> directory) to configure the workspace and other default settings:
gradlew ijWorkspaceSetup
gradlew ijConfigure
  • Open the LabKey project in IntelliJ:
    • Launch IntelliJ.
    • If your IntelliJ install is brand new, you will see the "Welcome to IntelliJ" pop up screen. Click Open.
    • If you have previously installed IntelliJ, select File > Open.
    • Select your <LK_ENLISTMENT> directory. You may need to confirm you trust the directory.
  • If you are developing with TypeScript, add a Path Variable. Most users will not need to do this.
    • Click the four-bars menu in the upper left, the select File > Settings > Appearance & Behavior > Path Variables.
    • Click the plus icon in the upper right.
    • Add a NODE_PATH variable and set it to the path to the node binary that is installed by the build, for example, <LK_ENLISTMENT>\build\.node\node-v#.#.##\bin\node
    • Click OK to close the Settings window.
  • Configure the Target JDK
    • In IntelliJ, select File > Project Structure.
    • Under Project Settings, click Project.
    • Under SDK select Add JDK...
    • Browse to the path of your JDK, for example, (C:\labkey\apps\jdk-##), and click OK.
    • Click Edit. Change the name of the JDK to "labkey".
    • Click OK to close the Project Structure window.
  • Open the Gradle tool window at View > Tool Windows > Gradle.
    • Click the refresh icon. This will take as much as 5-20 minutes. You should start seeing messages about its progress. If not, something is probably hung up. Wait for this sync to complete before progressing with further IntelliJ configuration steps.

Set the Run/Debug Configuration by opening the "Run" menu. It may initially read "GWT" or something else.

  • Users of IntelliJ Community should select LabKey Embedded Tomcat Dev.
  • Users of IntelliJ Ultimate should select Spring Boot LabKey Embedded Tomcat Dev.
Be sure that IntelliJ has enough heap memory. The default max is OK if you’re just dealing with the core modules, but you will likely need to raise the limit if you’re adding in custom modules, 3GB seems sufficient.

You may also want to increase the number of open tabs IntelliJ will support. The default is 10, and depending on your working process, you may see tabs disappear unexpectedly. Select Window > Editor Tabs > Configure Editor Tabs... and scroll down to the Closing Policy section to increase the number of tabs you can have open in IntelliJ at one time.

Build and Run LabKey

Configure the pg.properties File

The LabKey source includes a configuration file for use with PostgreSQL (pg.properties), specifying JDBC settings, including URL, port, username, password, etc.

Note that for users of Microsoft SQL Server, the file will be mssql.properties and the task for configuration will be "pickMSSQL" instead of "pickPg".

  • Open the file <LK_ENLISTMENT>/server/configs/pg.properties
  • Edit the file, adding your values for the jdbcUser and jdbcPassword. (This password is the one you specified when installing PostgreSQL.)

Run pickPg

  • In a command window, go to <LK_ENLISTMENT>
  • Run this task to configure the application.properties file:
    gradlew pickPg

When you run 'pickPg', the values that you've provided in both <LK_ENLISTMENT>/configs/application.properties and the pg.properties file in the same location will be copied into the <LK_ENLISTMENT>/build/deploy/embedded/config/application.properties file, overwriting previous values.

Build LabKey

To learn more about the build process, the various build tasks available, and how the source code is transformed into deployed modules, see Build LabKey from Source. In some cases, you may find it efficient to build all or part of your server using pre-built artifacts rather than source.

  • On the command line, go to the <LK_ENLISTMENT> directory, and invoke the gradle build task:
    gradlew deployApp

To control which modules are included in the build, see Customize the Build.

Run LabKey Server

To run and debug LabKey:
  • Click either the or icon to run or debug either 'Spring Boot LabKey Embedded Tomcat Dev' or 'LabKey Embedded Tomcat Dev' (depending on your edition of intelliJ). Hover text will guide you.
  • Once LabKey Server starts up successfully, navigate your browser to http://localhost:8080/ to begin debugging.
  • You should see the LabKey application and be invited to create a new user account the first time you start up using a new database.

While you are debugging, you can usually make changes, rebuild, and redeploy LabKey to the server without stopping and restarting Tomcat. Occasionally you may encounter errors that do require stopping and restarting Tomcat.

Optional Additions

Test Automation

To be able to run automated tests, you also need to clone the testAutomation repository into the server directory under <LK_ENLISTMENT>. Do not clone this repo into the 'modules' directory.:

LK_ENLISTMENT\server> git clone https://github.com/LabKey/testAutomation.git

An abbreviated map of your finished enlistment, showing the main folder structure (with many other elements omitted):

LK_ENLISTMENT 

└───server

├───modules
│ ├───commonAssays
│ ├───platform
│ └───[optional additional modules]

└───testAutomation

Additional Modules

Many optional modules are available from the LabKey repository on GitHub. To include these modules in your build, install a Git client and clone individual modules into the LabKey Server source.

  • To add a GitHub module to your build, clone the desired module into <LK_ENLISTMENT>/labkey/server/modules. For example, to add the 'reagent' module:
LK_ENLISTMENT\server\modules>git clone https://github.com/LabKey/reagent.git

Note that you can get the URL by going to the module page on GitHub (for example, https://github.com/LabKey/reagent), clicking Clone or Download, and copying the displayed URL.

Manage GitHub Modules via IntelliJ

Once you have cloned a GitHub module, you can have IntelliJ handle any updates:

To add the GitHub-based module to IntelliJ (and have IntelliJ generate an .iml file for the module):

  • Edit your settings.gradle file to include the new module
  • In IntelliJ, open the Gradle tool window at View > Tool Windows > Gradle.
  • Refresh the Gradle window by clicking the arrow circle in the upper left of the Gradle window
To update the GitHub-based module using IntelliJ:
  • To have IntelliJ handle source updates from GitHub, go to File > Settings (or IntelliJ > Preferences).
  • Select Version Control.
  • In the Directory panel, scroll down to the Unregistered roots section, select the module, and click the Plus icon in the lower left.
  • In the Directory panel, select the target module and set its VCS source as Git, if necessary.
  • Note that IntelliJ sometimes thinks that subdirectories of the module, like module test suites, have their sources in SVN instead of Git. You can safely delete these SVN sources using the Directory panel.
  • To sync to a particular GitHub branch: in IntelliJ, go to VCS > Git > Branches. A popup menu will appear listing the available Git modules. Use the popup menu to select the branch to sync to.
If you have added a new module to your enlistment, be sure to customize the build to include it in your Gradle project and then refresh the Gradle window to incorporate it into IntelliJ, as described above.

Generate Reference Docs

To generate documentation, such as the XML reference and Javadoc, clone the tools repository into the enlistment root (to create a sibling of the /server directory):

LK_ENLISTMENT>git clone https://github.com/LabKey/tools.git

Then you can run the documentation generation tasks, such as:

./gradlew :server:xsddoc
./gradlew :server:javadoc

Install R

Run the Basic Test Suite

  • Run this command from your <LK_ENLISTMENT> directory, to initiate automated tests of LabKey's basic functionality:
    gradlew :server:testAutomation:uiTest -Psuite=DRT
Other automated tests are available. For details, see Run Automated Tests. Note that 'R' must first be configured in order to run some tests.

Update a Development Machine

Update to Use Embedded Tomcat

Beginning with LabKey version 24.3, an embedded version of Tomcat 10 is now used. To update a development machine to use embedded Tomcat for the first time, follow these steps:

  1. Get the latest changes from all key repos (server, platform, etc.) with git pull. Don't forget to also update the root of your enlistment. Using the two source upgrade steps below is recommended.
  2. Update IntelliJ with the latest run configurations with gradlew ijConfigure
  3. Within your <LK_ENLISTMENT> level gradle.properties file, uncomment the useEmbeddedTomcat and useLocalBuild properties
  4. gradlew cleanBuild
  5. gradlew pickPg (or gradlew pickMSSQL)
  6. gradlew deployApp
  7. Within IntelliJ, do a Gradle Refresh in the Gradle Window.

Update Java

To update the JDK in the future, follow these steps.

  1. Repeat the steps for originally installing the JDK. Install the new version in a location parallel to the old.
  2. Update your system variable JAVA_HOME to point to the new location, unless you changed folder names so that the location path is the same.
  3. Update the IntelliJ > File > Project Structure > SDK > labkey home path to point to the new location. Rename the existing 'labkey' to its version number. Add the new JDK, then rename the new version "labkey".
  4. If necessary, add any additional flags to the VM options section. For example, when using Java 16 or 17, you'll need these flags in your JVM switches:
    --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.desktop/java.awt.font=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED

Update IntelliJ

Over time, you will want to update IntelliJ to stay current with features and enhancements. You can use the Help > Check for updates... option from within the application. Watch for installation alerts about any .idea files you may have edited. Note that updates may be incremental and after updating, a second more recent update may be available.

After updating, recheck the IntelliJ settings described in the above section.

Note that if you are developing against an older version of LabKey, you may need to use a version of IntelliJ that was available at the time of that release. We do not test newer releases of IntelliJ with older versions of LabKey.

Update LabKey Source Code

If you're working against the develop branch, you will continuously use "Update Project" to keep your local enlistment in sync. Where this option is in IntelliJ varies by version. It may be represented with a down-left-arrow, on the "four bars > Git" menu or accessed using a shortcut like ctrl-T.

If you have been synchronized to a specific branch (such as "release23.11"), you can change to a newer branch by using the recursive gitCheckout option. Use "Update Project" first, then:

LK_ENLISTMENT> gradlew gitCheckout -Pbranch=release24.7
LK_ENLISTMENT> git checkout release24.7

Related Topics


Premium Resources Available

Subscribers to premium editions of LabKey Server can learn more about developing with LabKey with the developer topics and code examples listed on this page:


Learn more about premium editions

Was this content helpful?

Log in or register an account to provide feedback


previousnext
 
expand allcollapse all