I suspect there's more detailed information in the log, beyond "Could not create the Java Virtual Machine". Can you post more of the logs?

A couple other things to note or check here:

• We strongly recommend Oracle OpenJDK 12, available here: https://jdk.java.net/12/. This is the only JDK implementation that we test with LabKey.
• What version of Tomcat are you running? We always recommend the latest point release (at the moment, 8.5.39 or 9.0.17). JDK 12 was just released last week; earlier versions of Tomcat were never tested with JDK 12.

Please consult our Supported Technologies page for more information about our recommendations and requirements.

Thanks,
Adam

Hi Adam,
Thanks for quick response. We have replace that Java (which is noted in the link from Labkey website I included in my previous email) with one you note Oracle OpenKD12 from the link you indicate. But we still get the same error on the browser access -
HTTP Status 404 - /labkey
type Status report
message /labkey
description The requested resource is not available.
Apache Tomcat/7.0.63
As you see we use Tomcat 7.0.63 so we will need to upgrade Tomcat as the next step.

In the meantime here is the full error message from the catalina.out:

Error: A fatal exception has occurred. Program will exit.
-Djava.endorsed.dirs=/usr/local/tomcat/endorsed is not supported. Endorsed standards and standalone APIs
in modular form will be supported via the concept of upgradeable modules.
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

Should I look for errors elsewhere?
Thanks,
Tomas

Actually - I attach whole catalina.out. Only 3/25 and 3/26 is relevant - events from around 2:41PM 3/25 are related to system reboot with the previous (wrong version of Java). Events from around 6:26AM 3/26 are related to system reboot with the recommended Java.

You definitely need to upgrade that old version of Tomcat. It came out in 2015, and pre-dates Java 9, 10, 11, and 12. The endorsed dirs concept was removed in Java 9. You might be able to get past this specific error by modifying the script or parameters you use to launch Tomcat, but you'll likely find other issues beyond this.

Our blog post mentions AdoptOpenJDK and several other OpenJDK distributions, but immediately goes on to say "...it’s important to understand that LabKey has not yet tested any of the non-Oracle distributions and, therefore, we do not support them." We're not saying that AdoptOpenJDK won't work, but we can't recommend it since we haven't tested it.

Adam

Hi Adam,

So by now I have installed the recommended OpenJDK-12 and I upgraded the Tomcat to 9.0.17. I moved all the labkey files to the folders per instructions. However, even after this I still cannot get the Labkey server working.
The error I get from the Tomcat when I try to launch the Labkey Server is the 404 error with description saying:
The origin server did not find a current representation for the target resource or is not willing to disclose that one exists.

Catalina.out shows errors I am not sure I can decipher.
Any help with troubleshooting is greatly appreciated.

Thanks,
Tomas

Actually,
I have got a bit further. Now I got the Tomcat working and I am seeing the Labkey Server actually trying to do something, but I get an error related to labkey.xml configuration. I have triple checked against my previous from version 18.2 and all variables edited in that one I have also edited in the 19.1 version labkey.xml:

Configuration Error Detected at LabKey Server Startup
DataSources are not properly configured in labkey.xml.

There is a problem with your configuration. Please contact your organization's server administrator for assistance or visit the LabKey Support Portal to view additional support options.

org.labkey.api.util.ConfigurationException: DataSources are not properly configured in labkey.xml.
at org.labkey.api.module.ModuleLoader.initializeDataSources(ModuleLoader.java:990)
at org.labkey.api.module.ModuleLoader.doInit(ModuleLoader.java:347)
at org.labkey.api.module.ModuleLoader.init(ModuleLoader.java:249)
at org.apache.catalina.core.ApplicationFilterConfig.initFilter(ApplicationFilterConfig.java:270)
at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:251)
at org.apache.catalina.core.ApplicationFilterConfig.<init>(ApplicationFilterConfig.java:102)
at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:4530)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5176)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:713)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:690)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:695)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:631)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1832)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:118)
at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:526)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:425)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1577)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:309)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:424)
at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:367)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:929)
at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:831)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1377)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1367)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:902)
at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:262)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.StandardService.startInternal(StandardService.java:423)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:928)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.startup.Catalina.start(Catalina.java:634)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:350)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:492)
Caused by: org.labkey.api.util.ConfigurationException: Can't connect to data source "labkeyDataSource".
at org.labkey.api.data.DbScope.ensureDataBase(DbScope.java:1321)
at org.labkey.api.module.ModuleLoader.ensureDatabase(ModuleLoader.java:1068)
at org.labkey.api.module.ModuleLoader.initializeDataSources(ModuleLoader.java:967)
... 46 more
Caused by: org.postgresql.util.PSQLException: FATAL: role "labkey" does not exist
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
at org.postgresql.core.v3.QueryExecutorImpl.readStartupMessages(QueryExecutorImpl.java:2559)
at org.postgresql.core.v3.QueryExecutorImpl.<init>(QueryExecutorImpl.java:133)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:250)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:195)
at org.postgresql.Driver.makeConnection(Driver.java:454)
at org.postgresql.Driver.connect(Driver.java:256)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228)
at org.labkey.api.data.DbScope.ensureDataBase(DbScope.java:1284)
... 48 more

"BindException: Address already in use" means something is running on the port you've configured... 8080, it looks like. Maybe your previous version of Tomcat is still running. You'll need to find and shut down that process, or else reconfigure Tomcat to use a different port.

Earlier in the log, "A docBase [/data/software/apache-tomcat-9.0.17/webapps/@@appDocBase@@] inside the host appBase has been specified, and will be ignored" means you need to replace this path with the correct location of your LabKey webapp. Either replace @@appDocBase@@ or replace the entire path, whichever is appropriate. Maybe you've already fixed this.

Adam

Hi Adam,

I think I got past those two issues. But still am getting errors - now it is in the labkey.log files.
I attach the labkey.log and labkey-errors.log.
I think I am close, but can't crack this issue.
Has anything changed since the 18.2 in the location of files or configuration of the labkey.xml. To me it seems all is in right place.

Thanks a lot for the help.

Tomas

Hi Adam et al,
Since Thursday I have repeated the installation of the 19.1 on our system and get to the same error as documented in my previous post. Tomcat (9.0.17) appears to be running fine and so does Java OpenJDK-12.
I attach here my labkey.xml, since that is now implicated as incorrectly configured.

I would really appreciate any suggestion that would allow me to finish the 19.1 installation.

Thanks,
Tomas

Hi Tomas,

It looks like your PostgreSQL database doesn't recognize the username and password that's configured in labkey.xml; I don't think this has anything to do with LabKey. Try connecting to PostgreSQL via psql command line and/or pgAdmin 4 using these same credentials. If you can't connect to PostgreSQL through those clients then LabKey won't be able to either.

Note that many installations use "postgres" as the PostgreSQL database user; of course, you may have changed the default when you installed PostgreSQL.

Adam

Thanks Adam,
I did not touch postgres in the process of upgrading from 18.2 to 19.1 and all worked fine with the indicated user and password in 18.2.
I will investigate more, but this is bizarre.

Best,
Tomas

Hi Adam,
So finally with Jimmy's and Vagisha's help I was able to resolve the issue. To summarize for the completeness here, it turned out that in preparation for the Java upgrade we also ran overall update of the system (yum update) that resulted in messing up the postgres database (clearly some upgrade of postgres was installed that resulted in corruption of the database). Once we restored the database from a backup everything fell in place. I've now upgraded to 19.1.1 and everything works.
Thanks for your patience and help.

Tomas