In case of errors or other problems when installing and running LabKey Server, first review installation basics and options linked in the topic: Install LabKey. This topic provides additional troubleshooting suggestions if the instructions in that topic do not resolve the issue.

LabKey Logs

From time to time Administrators may need to review the LabKey Tomcat logs to troubleshoot system startup or other issues. The LabKey Tomcat logs are located at:

/labkey/apps/tomcat/logs

The logs of interest are:

  • catalina.out: Contains the log file entries from Tomcat as it is starting the LabKey application.
  • labkey.log: Once the application starts, logging of INFO, ERROR, WARN and other messages is recorded here.
  • labkey-errors.log: Contains ERROR messages.
Useful commands:

# monitor the catalina.out log file while LabKey/Tomcat starts
sudo tail -f /usr/local/labkey/apps/tomcat/logs/catalina.out

# monitor the labkey.log file while LabKey/Tomcat starts
sudo tail -f /usr/local/labkey/apps/tomcat/logs/labkey.log

Log Rotation and Storage

To avoid massive log files and retain recent information with some context, log files are rotated (or "rolled-over") into new files to keep them manageable.

labkey.log: Rotated when the file reaches 10MB. Creates labkey.log.1, labkey.log.2, etc. Up to 7 archived files (80MB total).

labkey-errors.log: Rotated on server startup or at 100MB. Up to 3 labkey-errors log files are rotated. If the server is about to delete the first labkey-errors file from the current session (meaning it has generated hundreds of megabytes of errors since it started up), it will retain that first log file as labkey-errors-YYYY-MM-DD.log. This can be useful in determining a root cause of the many errors.

To save additional disk space, older log archives can be compressed for storage. A script to regularly store and compress them can be helpful. You should also check the logs to see that

Developer Mode

Running a server in development mode ("devmode") provides enhanced logging and enables the MiniProfiler, and has the server prevent the browser from caching resources like JavaScript and CSS files as aggressively. You do not want to run your production server in devmode, but might set up a shared development server to support future work, or have each developer work on their own local dev machine.

To check whether the server is running in devmode:

  • Go to (Admin) > Site > Admin Console.
  • Under Diagnostics, click System Properties.
  • Check the value of the devmode property.

Set -Ddevmode=true

If you are using a local development machine, include the following in your VM options as part of your intelliJ configuration:

-Ddevmode=true

If you are not using a development machine, you can set this option follow these steps. The supported version of tomcat will change over time; replace the # in our examples with the actual version number.

  • Open a command prompt.
  • Go to the <CATALINA_HOME>/bin directory, for example:
C:\labkey\apps\apache\apache-tomcat-#.#.##\bin
  • Execute the tomcat#w.exe program:
tomcat#w.exe //ES//LabKeyTomcat#
  • The command will open a program window. Click the Java tab.
  • In the Java Option box, scroll to the bottom of the properties. Add the following property at the bottom of the list:
-Ddevmode=true
  • Close the program window and restart the server.

JVM Caching

Note that the "caching" JVM system property can also be used to control just the caching behavior, without all of the other devmode behaviors. To disallow the normal caching, perhaps because files are being updated directly on the file system, add -Dcaching=false to the JVM arguments.

Diagnostic Information

Which Version of LabKey Server Is Running?

  • Find your version number at (Admin) > Site > Admin Console.
  • At the top of the Server Information panel, you will see the release version.

Learn more about other diagnostic information on the admin console in this topic:

Common Issues

Compatible Component Versions

Confirm that you are using the supported versions of the required components, as detailed in the Supported Technologies Roadmap. It is possible to have multiple versions of some software, like Java, installed at the same time. Check that LabKey and other applications, such as Tomcat, are configured to use the correct versions.

For example, if you see an error similar to "...this version of the Java Runtime only recognizes class file versions up to ##.0..." it likely means that Tomcat is running under an older version of the JDK than is supported.

Tomcat Failure to Start

If installation fails to start tomcat (such as with an error like "The specified service already exists."), you may need to manually stop or delete a failing Tomcat service.

To stop the service on Windows, open Control Panel > Administrative Tools > Services. Select the relevant service, such as LabKey Server Apache Tomcat #.0 and click Stop.

To delete, run the following from the command line as an administrator, substituting the major version number for the #:

sc delete LabKeyTomcat#

Deleted or Deactivated Users

Over time, you may have personnel leave the organization. Such user accounts should be deactivated and not deleted. If you encounter problems with accessing linked schemas, external schemas, running ETLs, or similar, check the logs to see if the former user may have "owned" these long term resources.

Connection Pool Size

If your server becomes unresponsive, it could be due to the depletion of available connections to the database. Watch for a Connection Pool Size of 8, which is the Tomcat connection pool default size and insufficient for a production server. To see the connection pool size for the LabKey data source, select (Admin) > Site > Admin Console and check the setting of Connection Pool Size on the Server Information panel. The connection pool size for every data source is also logged at server startup.

To set the connection pool size, edit your labkey.xml configuration file and change the "maxTotal" setting for your LabKey data source to at least 20. Depending on the number of simultaneous users and the complexity of their requests, your deployment may require a larger connection pool size.

You should also consider changing this setting for external data sources to match the usage you expect. Learn more in this topic: External Schemas and Data Sources.

Disable sslRequired

If you configure "SSL Required" and then cannot access your server, you will not be able to disable this requirement in the user interface. To resolve this, you can directly reset this property in your database.

Confirm that this query will select the "sslRequired" property row in your database:

SELECT * FROM prop.Properties WHERE Name = 'sslRequired' AND Set = (SELECT Set FROM prop.PropertySets WHERE Category = 'SiteConfig')

You should see a single row with a name of "sslRequired" and a value of "true". To change that value to false, run this update query:

UPDATE prop.Properties SET Value = FALSE WHERE Name = 'sslRequired' AND Set = (SELECT Set FROM prop.PropertySets WHERE Category = 'SiteConfig')

You can then verify with the SELECT query above to confirm the update.

Once the value is changed you'll need to restart the server to force the new value into the cache.

If you are using SQL Server, you'll need to double quote every reference to "Set" because that database considers Set a keyword.

Conflicting Applications

If you have problems during installation, retry after shutting down all other running applications. Specifically, you may need to try temporarily shutting down any virus scanning application, internet security applications, or other applications that run in the background to see if this resolves the issue.

PostgreSQL Installation

You may need to remove references to Cygwin from your Windows system path before installing LabKey, due to conflicts with the PostgreSQL installer. The PostgreSQL installer also conflicts with some antivirus or firewalls. (see http://wiki.postgresql.org/wiki/Running_%26_Installing_PostgreSQL_On_Native_Windows for more information).

Restart Installation from Scratch

If you have encountered prior failed installations, don't have any stored data you need to keep, and want to clean up and start completely from scratch, the following process may be useful:

  • Delete the Tomcat service (if it seems to be causing the failure).
  • Uninstall PostgresSQL using their uninstaller.
    • Control Panel > Programs and Features
    • Select PostgreSQL program.
    • Click Uninstall.
  • Delete the entire LabKey installation directory.
  • Install LabKey again.

Graphviz

If you encounter the following error, or a similar error mentioning Graphviz:

Unable to display graph view: cannot run dot due to an error.

Cannot run program "dot" (in directory "./temp/ExperimentRunGraphs"): CreateProcess error=2, The system cannot find the file specified.

then install Graphviz according to the following instructions:

Installation: Third-Party Components

Support Forum

You can search for problems resolved through community support in the LabKey Support Forum.

If you don't see your issue listed there, you can post a new question. If the install seems successful, it is often helpful to submit debugging logs for diagnosis.

If the install failed to complete, please include the install.log and install-debug.log from your selected LabKey install directory.

PostgreSQL logs its installation process separately. If PostgreSQL installation/upgrade fails, please locate and include the PostgreSQL install logs as well.

Related Topics

Discussion

Was this content helpful?

Log in or register an account to provide feedback


previousnext
 
expand all collapse all