Unable to get 24.3.4 working on my institution's container-based platform for deploying apps

LabKey Support Forum
Unable to get 24.3.4 working on my institution's container-based platform for deploying apps bkieft  2024-06-17 16:42
Status: Active
 

I am attempting to upgrade from 23.11.11 to 24.3.4 on a container-based platform provided by my workplace. So, I have created a Dockerfile that containerizes the embedded LabKey LIMS distribution using the template provided in LabKey's github page. In doing so, I've completed all the instructions provided on the LabKey website for upgrading (migrating my web.xml and ROOT.xml (labkey.xml) configurations over to the application.properties file, moving the labkeyServer.jar into my image, creating the necessary new directories, etc.).

When I deploy my container, I get several errors during java execution of labkeyServer.jar (after the Labkey Server welcome message and Conditions Evaluation Report are printed). I will attach the log file here, but I believe the root cause of the crash is ModuleLoader. Some modules load just fine (API, Pipeline, Query, Experiment, Assay, Audit, etc.), but then I get several error messages during module loading from others (for example, see below).

Please tell me if this is a known issue and/or if I can get some assistance fixing my deployment. Thanks!

java.lang.NumberFormatException: For input string: "tcp://10.43.165.217:8080" ```

``` ERROR ModuleLoader             2024-06-17T16:30:38,986           Module Upgrade : Failure starting module: FileContent
java.lang.NullPointerException: Cannot invoke "org.labkey.api.message.settings.MessageConfigService.registerConfigType(org.labkey.api.message.settings.MessageConfigService$ConfigTypeProvider)" because the return value of "org.labkey.api.message.settings.MessageConfigService.get()" is null ```

``` ERROR ModuleLoader             2024-06-17T16:30:37,945           Module Upgrade : Failure starting module: Announcements
java.lang.NullPointerException: Cannot invoke "org.labkey.api.message.settings.MessageConfigService.registerConfigType(org.labkey.api.message.settings.MessageConfigService$ConfigTypeProvider)" because the return value of "org.labkey.api.message.settings.MessageConfigService.get()" is null ```
 
 
mohara responded:  2024-06-18 11:13

Hi and thanks for reaching out.

I'm assuming you mean the Community Edition (and not a LIMS distribution, which would only be available with a Premium subscription). Was the previous server running in a docker container? I ask because that installation is not typical, but if you were previously using it, it's important to note that you need to reclone the dockerfile repo - the old and new versions are not cross-compatible, though offhand I don't have the details of what specifically symptoms of that incompatibility would look like.

When I look at that log, I see these errors above the ones you mention:

INFO  AppPropsImpl             2024-06-17T16:39:07,615           Module Upgrade : Setting site settings startup property 'sslRequired' to 'false'
INFO  AppPropsImpl             2024-06-17T16:39:07,615           Module Upgrade : Setting site settings startup property 'sslPort' to 'tcp://10.43.165.217:8080'
ERROR ModuleLoader             2024-06-17T16:39:07,615           Module Upgrade : Failure starting module: Core
java.lang.NumberFormatException: For input string: "tcp://10.43.165.217:8080"
...

Are you setting the sslPort to the full URL rather than just the 8080 port somewhere? In application.properties, you should have a line like:
server.port=8080
And not need to also set that as a "site settings startup property"... From that message, the full URL value could be in the database, or possibly specified in the docker environment? I'm not sure exactly, but I'd investigate where you are setting that property first.

--Molly

 
bkieft responded:  2024-06-18 12:25

Thanks for the reply, Molly.

Yes, I am talking about the community edition of 24.3.4. Our server was previously running 23.11.11 on the same resource, and yes I've made sure the template Dockerfile is the most updated form from github.

My server.port variable in application.properties is just set to 8080 as you've indicated. I'm not sure where/why it's getting expanded to the longer version and causing a NumberFormatException.

I read an issue on Medium where it's possible that the kubernetes code behind my container hosting platform is superseding the port variable. Is this a known issue? I tried reapplying the variable in a java properties file in /startup/basic_properties by changing the SiteSettings.sslPort variable from $LABKEY_PORT to hard-coded '8080'. That dropped all the warnings/errors that I was getting from ModuleLoader. Is this the "site settings startup property" you're referring to? My application is still not running correctly, so I'm attaching another updated log here. Please let me know if any of the issues here seem familiar to you, I know that this is a bit of a complex use case for the community edition.

 
mohara responded:  2024-06-18 14:58

I'm not sure what other parts of the environment are setting things there - you shouldn't need to set that sslPort in /startup/basic_properties at all, as the port gets set in application.properties. I'm seeing lines like this near the top of that log, so somewhere you have something "meddling" with the properties, which is suspicious:

Adding secrets to config/application.properties from environment variables...
Adding database and smtp settings to config/application.properties from environment variables...
Purging secrets and other bits from environment variables...

I also wonder if you've tried accessing the server. It looks like you've added "debug=true" to your application.properties as well (or something did) and that's what's dumping that "Conditions Evaluation Report" into the log. When I start up my happy local machine, the "conditions" look the same as yours, so I don't think that indicates any problem necessarily. Try removing that and restarting, then checking to see if the site is up. If you have the logs from the prior instance, comparing those might help identify what is not working the same way now?

Another troubleshooting idea is to try starting the 24.3 server on a new/empty database to first get it running happily before addressing any possible upgrade issues, but I'm not seeing those in the logs, so not sure if that's a problem.

--Molly