If you have encountered errors or other problems when installing and starting LabKey Server, first review this topic:
The list below includes some possible errors, example messages, and common problems with suggested solutions.
Premium edition subscribers can open a ticket on their support portal. Community users can
search the LabKey Community Support Forums to see if the issue you are seeing is listed there.
Database Startup
Error:
"Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections."
Problem: Tomcat (embedded in LabKey Server) cannot connect to the database.
Likely causes:
- The database is not running
- The database connection URL or user credentials in the application.properties file are wrong
- LabKey (Tomcat) was started before the database finished starting up
Solution: Make sure that database is started and fully operational before starting LabKey. Check the database connection URL, user name, and password in the <LABKEY_HOME>/config/application.properties file.
Unsupported JDK Version
Resource Not Available/Error Deploying Configuration Descriptor
Error:
"The requested resource () is not available." OR
"500: Unexpected server error"
Plus stack trace in the log file related to "Error deploying configuration descriptor..."
Solution:
- You may need to use a newer version of the JDK. See Supported Technologies.
2. Confirm that LabKey (i.e. Tomcat) is configured to use the correct version of Java, as it is possible to have multiple versions installed simultaneously. Check that <JAVA_HOME> points to the correct version.
Fatal Error in JRE
Error:
A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x0000000000000000, pid=23893, tid=39779
#
# JRE version: Java(TM) SE Runtime Environment (8.0_45-b14) (build 1.8.0_45-b14)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.45-b02 mixed mode bsd-amd64 compressed oops)
# Problematic frame:
# C 0x0000000000000000
#
# Failed to write core dump. Core dumps have been disabled.
# To enable core dumping, try "ulimit -c unlimited" before starting Java again
Cause: These are typically bugs in the Java Virtual Machine itself.
Solutions:
- Ensure that you are on the latest patched release for a supported Java version.
- If you have multiple versions of Java installed, be sure that <JAVA_HOME> and other configuration is pointing at the correct location.
- If you are running through the debugger in IntelliJ, check the JDK configuration:
- Under Project Structure > SDKs check the JDK home path and confirm it points to the newer version.
Postgres Join Collapse Limit
Error: If you see the following error when running complex queries on PostgreSQL:
org.postgresql.util.PSQLException: ERROR: failed to build any 8-way joins
Solution: Increase the join collapse limit. Edit postgresql.conf and change the following line:
# join_collapse_limit = 8
to
join_collapse_limit = 10
Font-related Errors
Sometimes font-related errors including but not limited to the following will need to be remedied on your system.
1. Problem: If you see an error containing messages similar to this:
Caused by: java.lang.RuntimeException: Fontconfig head is null, check your fonts or fonts configuration
at java.desktop/sun.awt.FontConfiguration.getVersion(FontConfiguration.java:1271) ~[?:?]
at java.desktop/sun.awt.FontConfiguration.readFontConfigFile(FontConfiguration.java:224) ~[?:?]
at java.desktop/sun.awt.FontConfiguration.init(FontConfiguration.java:106) ~[?:?]
at java.desktop/sun.awt.X11FontManager.createFontConfiguration(X11FontManager.java:706) ~[?:?]
at java.desktop/sun.font.SunFontManager$2.run(SunFontManager.java:358) ~[?:?]
at java.desktop/sun.font.SunFontManager$2.run(SunFontManager.java:315) ~[?:?]
at java.base/java.security.AccessController.doPrivileged(AccessController.java:318) ~[?:?]
at java.desktop/sun.font.SunFontManager.<init>(SunFontManager.java:315) ~[?:?]
Solution: you should install
fontconfig via your operating system's package manager.
2. Problem: Plots are not rendered as expected, or cannot be exported to formats like PDF or PNG. Excel exports fail or generate corrupted xlsx files. Possible Excel export failures include 500 errors or opening of an empty tab instead of a successful export.
Error:
java.lang.InternalError: java.lang.reflect.InvocationTargetException
at java.desktop/sun.font.FontManagerFactory$1.run(FontManagerFactory.java:86)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:312)
at java.desktop/sun.font.FontManagerFactory.getInstance(FontManagerFactory.java:74)
...
Caused by: java.lang.NullPointerException: Cannot load from short array because "sun.awt.FontConfiguration.head" is null
at java.desktop/sun.awt.FontConfiguration.getVersion(FontConfiguration.java:1260)
at java.desktop/sun.awt.FontConfiguration.readFontConfigFile(FontConfiguration.java:223)
...
or
java.lang.InternalError: java.lang.reflect.InvocationTargetException
at java.desktop/sun.font.FontManagerFactory$1.run(FontManagerFactory.java:86)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:312)
at java.desktop/sun.font.FontManagerFactory.getInstance(FontManagerFactory.java:74)
...
Caused by: java.lang.NullPointerException
at java.desktop/sun.awt.FontConfiguration.getVersion(FontConfiguration.java:1262)
at java.desktop/sun.awt.FontConfiguration.readFontConfigFile(FontConfiguration.java:225)
at java.desktop/sun.awt.FontConfiguration.init(FontConfiguration.java:107)
...
Solution:
Install the fonts required to generate plots, most typically missing on some Linux JDK installations.
WebSocket Errors
Error: One or more of:
- Banner message reading "The WebSocket connection failed. LabKey Server uses WebSockets to send notifications and alert users when their session ends. See the Troubleshooting Installation topic|troubleshootingAdmin#websocket] for more information.
- Error logged to the JS console in the browser:
"clientapi.min.js?452956478:1 WebSocket connection to 'wss://server.com/_websocket/notifications' failed.
- Unexpected errors in login/logout modals for Sample Manager and Biologics applications.
Problem: WebSocket connection errors are preventing a variety of actions from succeeding. Try using browser developer tools and checking the Console tab to see the specific errors.
Likely cause: A load balancer or proxy may be blocking websocket connections and needs to have its configuration updated.
Solution: Make sure that websockets are available and configured properly.
Pipeline Serialization Errors
Error: On Java 16 or later, pipeline jobs and some other operations fail with errors.
Problem: To be compatible with some libraries, such as Jackson, Java 16 and later needs to be configured with a command-line argument to allow for successful serialization and deserialization. Errors may take different forms, but include messages like:
java.lang.reflect.InaccessibleObjectException: Unable to make private java.io.File(java.lang.String,java.io.File) accessible: module java.base does not "opens java.io" to unnamed module @169268a7
or
java.lang.reflect.InaccessibleObjectException: Unable to make field private java.lang.String java.lang.Throwable.detailMessage accessible: module java.base does not "opens java.lang" to unnamed module @18f84035
Solution: Add to the Java command line:
--add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.desktop/java.awt.font=ALL-UNNAMED --add-opens=java.base/java.text=ALL-UNNAMED
Upgradable Versions
Error:
ERROR ModuleLoader 2022-04-07T10:09:07,707 main : Failure occurred during ModuleLoader init.
org.labkey.api.util.ConfigurationException: Can't upgrade from LabKey Server version 20.3; installed version must be 21.0 or greater.
...
Cause: Beginning in January 2022, you can only upgrade from versions less than a year old.
Solution: Following the guidance in
this topic, you will need to perform intermediate upgrades of LabKey Server to upgrade from older versions.
Excel POI Files
Problem: Disk filling with Tomcat temporary files. For example, Excel exports might be leaving one file for every export in "temp/poifiles" folder.
Error:
...| comment: Exported to Excel |
ERROR ExceptionUtil 2022-07-19T11:24:20,609 ps-jsse-nio-8443-exec-25 : Unhandled exception: No space left on device
java.io.IOException: No space left on device
Likely cause: The Excel export writer is leaving temporary "poi-sxssf-####.xml" files behind. These can be fairly large and may correlate 1:1 with Excel exports. This issue was reported in version 22.3.5 and fixed in version 22.11.
Solution: These temporary "poifiles" may be safely deleted after the export is completed. Administrators can periodically clean these up; when this issue is resolved, they will be automatically cleaned up within 10 minutes of an Excel export.
ETL Schedule Error
Error:
org.labkey.api.util.ConfigurationException: Based on configured schedule, the given trigger 'org.labkey.di.pipeline.ETLManager.222/{DataIntegration}/User_Defined_EtlDefId_36' will never fire.
...
org.quartz.SchedulerException: Based on configured schedule, the given trigger 'org.labkey.di.pipeline.ETLManager.222/{DataIntegration}/User_Defined_EtlDefId_36' will never fire.Problem: If an ETL is saved with an invalid schedule, the server may fail to start with an error similar to the above.
Solution: Correct the invalid schedule in the ETL. Learn more in this topic:
Graal.js Error on Startup
Error:
ScriptEngineManager providers.next(): javax.script.ScriptEngineFactory: Provider com.oracle.truffle.js.scriptengine.GraalJSEngineFactory could not be instantiated
and later this, with full stack trace again mentioning Graal:
ERROR CoreModule 2024-03-28T16:54:16,073 main : Exception registering MarkdownServiceImpl
org.labkey.api.util.ConfigurationException: Graal.js engine not found
Cause: The LabKey process does not have permission to write to a .cache directory under the user's home directory.
Solution: The fix is to set an environment variable, XDG_CACHE_HOME, pointing to a directory that the process has permission to write to, such as the <LABKEY_HOME>/labkey-tmp directory. Graal only supports this environment variable on Linux. It can be added to the [Service] section of the labkey_server.service configuration file, alongside other environment variables.
Environment="XDG_CACHE_HOME=/labkey/labkey/labkey-tmp"
Note: If you had previously addressed this problem by adding a "-Dployglot.engine.resourcePath=..." argument, you will need to remove it for the XDG_CACHE_HOME environment variable to be picked up.
Remember that any time you edit your Linux service file, you need to reload the service daemon:
sudo systemctl daemon-reload
Related Topics
