The LabKey Server source code is available via enlistment in LabKey's Subversion repository. Creating an enlistment will allow you to monitor, build, and develop against the most current, unreleased LabKey source code as well as released versions of the product. The following topic provides step-by-step instructions for acquiring the LabKey Server source code, installing required components, and building LabKey Server from source.

Checklist

A checklist, guiding you through the setup process, is available for download: LabKey_Development_Server_Checklist.xlsx

Obtain the LabKey Source Files

The LabKey source files are stored in two repositories: the core modules are stored in an SVN repository, selected modules are stored on GitHub.

The following instructions apply to Windows machines. To install SVN on non-Windows machines see Enlisting in the Version Control Project.

Install TortoiseSVN

  • Download the latest version of TortoiseSVN.
  • Install TortoiseSVN on your local computer.
  • On the list of features to install, include the command line client tools.
  • Add the TortoiseSVN/bin directory to your PATH.

Checkout LabKey Source Files

  • Create a new directory in the Windows file system for the source files, for example, C:\dev\labkey\trunk
  • In Windows Explorer, right-click the new directory and select SVN Checkout.
  • Enter the URL for the LabKey repository: https://hedgehog.fhcrc.org/tor/stedi/trunk
    • The user/password is cpas/cpas
  • Click OK to checkout the source files. At this point all the LabKey source files, tests, and sample data will be copied to your computer.

Install Java, Tomcat, and a Database

Java

Download the Oracle JDK version 8 and install it.

Tomcat

Download the most recent release of Tomcat 8.5.x. Download a ZIP or TAR.GZ distribution, not the Windows Service Installer. To install Tomcat, unzip it to a chosen directory (for example, C:\apache\tomcat).

LabKey supports older versions of Tomcat as well; find more information about supported versions here. If using Tomcat 7.0.x, follow instructions on this page: Encoding in Tomcat 7

Install a Database

Install one of the following database servers: PostgreSQL or Microsoft SQL Server

Platform-specific installation instructions:

Environment Variables and System PATH

JAVA_HOME

Create or modify the system environment variable JAVA_HOME so it points to your JDK installation location (for example, %ProgramFiles%\Java\jdk1.8.0_xx).
If you've already set the JAVA_HOME variable to point to your installation of the JRE, you should modify it to point to the JDK.

CATALINA_HOME

Create or modify the system environment variable CATALINA_HOME so that it points to your Tomcat installation (for example, C:\apache\tomcat).

PATH

Add the following locations to your system PATH, where LABKEY_HOME is the root of your SVN enlistment.

  • <LABKEY_HOME>/external/ant/bin
  • <LABKEY_HOME>/build/deploy/bin (This directory won't exist yet, but add it to the PATH anyways.)
For example, C:\dev\labkey\external\ant\bin;C:\dev\labkey\build\deploy\bin.
These directories contain Apache Ant for building the LabKey source, as well as a number of executable files used by LabKey.
The build process creates build/deploy/bin, so it may not exist initially.

Apache Ant is included in the project as a convenience. If you have a recent version of Ant already installed you can use that instead. Ant 1.9.3 or newer is required to build.

For example, on OSX, place the environment variables in your .bash_profile:

export JAVA_HOME=`/usr/libexec/java_home -v 1.8`
export CATALINA_HOME=$HOME/apps/tomcat
export LABKEY_HOME=$HOME/labkey/trunk
export LABKEY_GWT_USER_OVERRIDE="gwt-user-firefox"
export PATH=$LABKEY_HOME/external/ant/bin:$LABKEY_HOME/build/deploy/bin:$PATH

GWT_HOME

Installing and configuring GWT is required only if you plan to modify existing or develop new GWT components. If you do not plan to develop with GWT you can disable IntelliJ's notifications by going to File > Project Structure. Click Facets and disable framework detection (remove the checkmark at the top of the dialog).

Open the LabKey Project in IntelliJ

The LabKey development team develops LabKey using IntelliJ IDEA. You can use this tool or a different Java environment if you are planning on modifying or extending the LabKey source code. Developers at non-profit organizations that are contributing open source code may qualify for a free license. Please contact LabKey for more information.

Below we describe how to configure the IntelliJ development environment; we recommend employing the same general principles if you are using a different development environment. Some users have experimented with Eclipse as the IDE and you can find some set up details on the Developer Message Board.

Download and install IntelliJ IDEA.

Configure the LabKey Project in IntelliJ

  • Create the workspace.xml file.
    • Copy the file <LABKEY_HOME>/server/.idea/workspace.template.xml. Rename the copy to create a file called <LABKEY_HOME>/server/.idea/workspace.xml
    • This file configures the debug information for LabKey project. To review the debug settings go to Run > Edit Configurations in IntelliJ.
  • Open the LabKey project.
    • Launch IntelliJ.
    • If your IntelliJ install is brand new, you will see the "Welcome to IntelliJ" pop up screen. Click Open. If you have previously installed IntelliJ, select File > Open.
    • Select the LabKey IntelliJ project directory, <LABKEY_HOME>/server
  • Set CATALINA_HOME
    • Select File > Settings > Appearance & Behavior > Path Variables.
    • Click the green plus icon in the upper right. Set the CATALINA_HOME path variable to the root directory of your Tomcat installation, for example, C:\apache\apache-tomcat-8.0.28.
    • Click OK to close the Settings window.
  • Set the Classpath
    • Select Run > Edit Configurations. (If the menu is greyed-out, wait until IntelliJ finishes indexing the project files.)
    • Confirm that LabKey Development is the selected Application in the left panel.
    • Confirm that the dropdown labeled Use classpath of module is set to LabKey.
    • Click OK, to close the Run/Debug Configurations window.
  • Configure the Target JDK
    • In IntelliJ, select File > Project Structure.
    • Under Project Settings, click Project.
    • Under Project SDK click New and then click JKD.
    • Browse to the path of your JDK, for example, (C:\Program Files (x86)\Java\jdk1.8.0_66), and click OK.
    • Click Edit. Change the name of the JDK to "labkey".
    • Click Ok to close the Project Structure window.
  • Verify the Target JDK for Ant
    • In IntelliJ, select View > Tool Windows > Ant Build.
    • In the Ant Build panel (on the far right), click the Properties button (which is directly left of the Help question mark '?' button).
    • Click the Execution tab.
    • Verify that Use project default Ant is selected.
    • Verify that Run under JDK drop-down is set to "Project JDK (labkey)".
    • Click OK.

Build and Run LabKey

Configure the Appropriate .properties File

The LabKey source includes two configuration files, one for use with PostgreSQL (pg.properties) and one for use with Microsoft SQL Server (mssql.properties), each specifying JDBC settings, including URL, port, user name and password, etc.

  • If using PostgreSQL, open the file LABKEY_HOME/server/configs/pg.properties
  • If using MS SQL Server, open the file LABKEY_HOME/server/configs/mssql.properties
  • Edit the appropriate file, adding your values for the jdbcUser and jdbcPassword. (This password is the one you specified when installing PostgreSQL or MS SQL Server. If your password contains an ampersand or other special XML characters, you will need to escape it in the .properties file, as the value will be substituted into an XML template without encoding. For example, if your JDBC password is "this&that", then use the escaped version "this&amp;that".)

Run pick_pg or pick_mssql

  • In a command window, go to the directory LABKEY_HOME/server
  • Run "ant pick_pg" or "ant pick_mssql" to configure labkey.xml with the corresponding database settings.
When you build LabKey, the values that you've specified in the mssql.properties or pg.properties file are copied into the LabKey configuration file, labkey.xml, overwriting previous values. This file is then copied into TOMCAT_HOME/conf/Catalina/localhost.

Build LabKey

To build LabKey, invoke the Ant build targets from the command line in the <LABKEY_HOME>/server directory.

To control which modules are included in the build, see Customizing the Build.

The most important targets:

Ant TargetoooooooooooooooooDescription
ant pick_pg
ant pick_mssql
Specify the database server to use. The first time you build LabKey, you need to invoke one of these targets to configure your database settings. If you are running against PostgreSQL, invoke the pick_pg target. If you are running against SQL Server, invoke the pick_mssql target. These Ant targets copy the settings specified in the pg.properties or mssql.properties file, which you previously modified, to the LabKey configuration file, labkey.xml.
ant buildBuild the LabKey Server source for development purposes. This is a fast, development-only build that skips many important steps needed for production environments, including GWT compilation for popular browsers, gzipping of scripts, production of Java & JavaScript API documentation, and copying of important resources to the deployment location. Builds produced by this target will not run in production mode.
ant <module_name>For convenience, we've added targets to build each of the standard modules. If your changes are restricted to a single module then building just that module is a faster option than a full build. Examples: 'ant study', 'ant query', or 'ant api'.
ant productionBuild the LabKey Server source for deployment to a production server. This build takes longer than ant build but results in artifacts that are suitable and optimized for production environments.
ant cleanDelete all artifacts from previous builds.
ant rebuildDelete all artifacts from previous builds and build the LabKey Server from source. This build target is sometimes required after certain updates.

Ant targets can also be invoked from within IntelliJ via the "Ant Build" tab.

If you get an error message along the lines of "Please use Ant 1.8.3 or greater", you can configure IntelliJ to use a compatible version of Ant that is included in the <labkey-home>/external/ant directory. Click on the Properties button in the Ant Window and use the Execution tab to define a custom Ant installation.

You can speed up development-time builds by restricting GWT complication to a subset of browsers. (GWT compilation is one of the most time-consuming parts of the LabKey build.) For example, setting "LABKEY_GWT_USER_OVERRIDE=gwt-user-firefox" as an environment variable will cause the build to target FireFox only. Other browsers (e.g., Internet Explorer) will not work properly with the resulting server. (Note: gwt-user-override environment variable is a synonym that works on windows but not Mac/Linux.) If set, this environment variable is respected for development builds (ant build) but ignored for production builds (ant production).

To run and debug LabKey, select Run > Debug 'LabKey Development' in IntelliJ. If Tomcat starts up successfully, navigate your browser to http://localhost:8080/labkey to begin debugging (assuming that your local installation of Tomcat is configured to use the Tomcat default port 8080).

While you are debugging, you can usually make changes, rebuild, and redeploy LabKey to the server without stopping and restarting Tomcat. Occasionally you may encounter errors that do require stopping and restarting Tomcat.

Post-installation Steps

Install R

Install and configure the R programming language

Run the Basic Test Suite

Run the command 'ant drt' from within your <labkey-home>/server directory, to initiate automated tests of LabKey's basic functionality. Note that 'R' must first be configured for these tests to run. Other automated tests are available as Ant targets. For details, see Running Automated Tests.

Modules on GitHub

Many optional modules are available from the LabKey repository on GitHub. To included these modules in your build, install a Git client and clone individual modules into the LabKey Server source.

Install a Git Client

Clone Modules from LabKey's GitHub Repository

  • To add a GitHub module to your build, clone the desired module into trunk/labkey/server/optionalModules. For example, to add the 'workflow' module:
C:\svn\trunk\server\optionalModules>git clone https://github.com/LabKey/workflow.git

Note that you can get the URL by going to the module page on GitHub (for example, https://github.com/LabKey/workflow), clicking Clone or Download, and copying the displayed URL.

Manage GitHub Modules via IntelliJ

Once you have cloned a GitHub module, you can have IntelliJ handle any updates:

To add the GitHub-based module to IntelliJ:

  • In IntellJ, go to File > Project Structure.
  • Under Project Settings, select Modules.
  • Click the green plus sign (top of the second column) and select Import Module.
  • Navigate to the module you've cloned from the GitHub repository and select its .iml file, for example, NLP.iml., and click OK.
To update the GitHub-based module using IntelliJ:
  • To have IntelliJ handle source updates from GitHub, go to File > Settings.
  • Select Version Control.
  • In the Directory panel, select the target module and set its VCS source as Git, if necessary.
  • Note that IntelliJ will sometimes think that embedded 'test' modules have their sources in SVN instead of Git. You can safely delete these embedded 'test' modules using the Directory panel.
  • To sync to a particular GitHub branch: in IntelliJ, go to VCS > Git > Branches. A popup menu will appear listing the available Git modules. Use the popup menu to select the branch to sync to.

Install Optional Components

GWT

Installing and configuring GWT is required only if you plan to modify existing or develop new Google Web Toolkit (GWT) components.

Please see GWT Integration for instructions on installation and configuration of GWT.

Mass Spec and Proteomics Tools

LabKey Server's mass spectrometry and proteomics binaries are provided as a separate (and optional) enlistment. To add these binaries, follow the instructions in the topic: Enlisting Proteomics Binaries

Troubleshooting

1. Tomcat

If Tomcat fails to start successfully, check the steps above to ensure that you have configured your JDK and development environment correctly. Some common errors you may encounter include:

org.postgresql.util.PSQLException: FATAL: password authentication failed for user "<username>" or java.sql.SQLException: Login failed for user '<username>'

These error occurs when the database user name or password is incorrect. If you provided the wrong user name or password in the .properties file that you configured above, LabKey will not be able to connect to the database. Check that you can log into the database server with the credentials that you are providing in this file.

java.net.BindException: Address already in use: JVM_Bind:<port x>:

This error occurs when another instance of Tomcat or another application is running on the same port. Specifically, possible causes include:

  • Tomcat is already running under IntelliJ.
  • Tomcat is running as a service.
  • Microsoft Internet Information Services (IIS) is running on the same port.
  • Another application is running on the same port.
In any case, the solution is to ensure that your development instance of Tomcat is running on a free port. You can do this in one of the following ways:
  • Shut down the instance of Tomcat or the application that is running on the same port.
  • Change the port for the other instance or application.
  • Edit the Tomcat server.xml file to specify a different port for your development installation of Tomcat.
java.lang.NoClassDefFoundError: com/intellij/rt/execution/application/AppMain:
or
Error: Could not find or load main class com.intellij.rt.execution.application.AppMain:

In certain developer configurations, you will need to add an IntelliJ utility JAR file to your classpath.

  • Edit the Debug Configuration in IntelliJ.
  • Under the "VM Options" section, find the "-classpath" argument.
  • Find your IntelliJ installation. On Windows machines, this is typically "C:\Program Files\JetBrains\IntelliJ IDEA <Version Number>" or similar. On Mac OSX, this is typically "/Applications/IntelliJ IDEA <Version Number>.app" or similar.
  • The required JAR file is in the IntelliJ installation directory, and is ./lib/idea_rt.jar. Add it to the -classpath argument value, separating it from the other values with a ":" on OSX and a ";" on Windows.
  • Save your edits and start Tomcat.

2. Database State

If you build the LabKey source yourself from the source tree, you may need to periodically delete and recreate your LabKey database. The daily drops often include SQL scripts that modify the data and schema of your database.

3. IntelliJ Warnings and Errors

  • Warning: Class "org.apache.catalina.startup.Bootstrap" not found in module "LabKey": You may ignore this warning in the Run/Debug Configurations dialog in IntelliJ.
  • Error: Could not find or load main class org.apache.catalina.startup.Bootstrap on OSX (or Linux): you might see this error in the console when attempting to start LabKey server. Update the '-classpath' VM option for your Run/Debug configuration to have Unix (:) path separators, rather than Windows path separators (;).
  • Certain lines in build.xml files and other Ant build files may be incorrectly flagged as errors.
  • Can't find workspace.template.xml? On older enlistments of LabKey, for example version 15.3, copy <LABKEY_HOME>/server/LabKey.iws.template to LabKey.iws instead.

4. IntelliJ Slow

You can help IntelliJ run faster by increasing the amount of memory allocated to it. To increase memory:

  • Go to C:\Program Files\JetBrains\IntelliJ IDEA <Version Number>\bin, assuming that your copy of IntelliJ is stored in the default location on a Windows machine.
  • Right click on the idea.exe.vmoptions file and open it in notepad.
  • Edit the first two lines of the file to increase the amount of memory allocated to IntelliJ. For example, on a 2 Gig machine, it is reasonable to increase memory from 32m to 512m. The first two lines of this file then read:
-Xms512m
-Xmx512m
  • Save the file
  • Restart IntelliJ
1.1.5 JAVA_TOOL_OPTIONS

Most users will not have this problem. However, if you see a build error something like the following:

error: unmappable character for encoding ASCII

then setting this environment variable may fix the problem

export JAVA_TOOL_OPTIONS=-Dfile.encoding=UTF8

discussion

Discussion

Search 

Pages 

previousnext
 
expand all collapse all