Hi Diego,

Can you also send us a copy of your LabKey.log file and the LabKey.XML file (with the password removed)? This will give us a better idea of the problem.

Regards,

Jon

Jon,

Will add the requested files in the am!

Thanks,
Diego

Jon,

LabKey.log or labkey.log are not located on the server "find / -name labkey.log" returns no results. I have attached the labkey.xml file.

LabKey Server Libraries are installed in: /var/lib/tomcat/ "ant.jar, jtds.jar, mail.jar etc..."

Let me know if you have any other questions.

Thank you,
Diego

Jon,

At the moment there is no labkey directory or files in the tomcat "webapps/ROOT" folder --> /usr/share/tomcat/webapps/ROOT/index.html


When inputting the following url in my browser -> http://localhost/labkey/ a "HTTP Status 404 - /labkey/" error is produced.

From my observation the reason LabKey server is not running is because Tomcat is not able to reference the proper file path, hence producing errors...

Please Advise!

Thank you,
Diego

Hi Diego,

Thanks for providing your LabKey.xml file.

Based on the error from the Catalina log and your LabKey.xml file, it appears that the labkeyBootstrap.jar file isn't being found within your Tomcat directory.

According to our docs (https://www.labkey.org/home/Documentation/wiki-page.view?name=configTomcat#1), the following six files that were unzipped out of the zip file:

tomcat-lib/ant.jar
tomcat-lib/jtds.jar
tomcat-lib/mail.jar
tomcat-lib/mysql.jar
tomcat-lib/postgresql.jar
tomcat-lib/labkeyBootstrap.jar

should have all be put within the TOMCAT_HOME/lib directory

If your Tomcat installation resides in:

/var/lib/tomcat/

Then that would mean that within that directory you should have a bin, conf, logs, lib, webapps, and a few other subdirectories:

/var/lib/tomcat/bin/
/var/lib/tomcat/lib/
/var/lib/tomcat/conf/
/var/lib/tomcat/logs/
/var/lib/tomcat/webapps/

Does your Tomcat installation match up like this?

Regards,

Jon

have attached a screenshot of the "/var/lib/tomcat" directory... have also attached a screenshot of all the tomcat directories in the server.

found that "/usr/share/tomcat/" contains the correct folders...

[root@localhost share]# cd /usr/share/tomcat
[root@localhost tomcat]# ls
bin conf lib logs temp webapps work

Moved LabKey Server Libraries into "/usr/share/tomcat/lib" and the server is now loading but with errors. have attached the labkey.log and labkey-error.log

Thanks!

Hi Diego,

The labkey.xml file looks okay, which is making me question whether your username and password for your SQL server is correct.

Can you do the following for me?

1. Send me your newer Catalina log file. It should provide us with some more details with what was occurring when LabKey tried to start.

2. Attempt to login to your Postgres DB using the same username/password combination and verify that it allows successful access.

Regards,

Jon

I have attached the latest catalina.log

Getting the following error when trying to login into postgres -->

[root@localhost tomcat]# psql postgres postgres
psql: FATAL: Peer authentication failed for user "postgres

Hi Diego,

Looks like your pg_hba.conf file might be misconfigured.

Within your Postgres directory, there should be some kind of data subdirectory that will contain the pg_hba.conf file (note: you may need to do a sudo as the postgres user to get into that directory for access). If you run a 'less' on the file and go to the bottom of the screen, there should be a list of configurations.

Most likely, there is probably a method listed on one of the lines for "peer" when it should be set to "md5".

Can you check your file to confirm this and make the changes to the method? Then re-attempt to sign into psql again with your credentials? If your credentials pass through, then try to restart Tomcat to see if LabKey starts up correctly.

Regards,

Jon

Jon,

Able to log into postgres after making the "md5" change:

[root@localhost tomcat]# psql -U postgres
Password for user postgres:
psql (9.2.14)
Type "help" for help.

Restarted Tomcat and still getting -> DataSources are not properly configured in labkey.xml.

Most recent logfiles attached.

Thanks,
Diego

Hi Diego,

What version of Tomcat are you running and is this version via the package manager or one you downloaded directly from Apache's site?

I noticed this also in the Catalina logs regarding the path pointing to labkey.xml:

/etc/tomcat/Catalina/localhost/labkey.xml

Since you said that you moved those jar files to /usr/share/tomcat/lib, then logically you should have that labkey.xml file located under:

/usr/share/tomcat/conf/Catalina/localhost/labkey.xml

And that would also mean that the Tomcat service should be running from:

/usr/share/tomcat/bin/startup.sh

Regards,

Jon

Also, can you re-run the psql command using the -h flag and the localhost:5432 address to make sure it can connect there as well?

So basically:

>psql -h localhost -p 5434 -U postgres

Additionally, your labkey.xml file I noticed had the username of "dude" and a password of "foobar". I'm assuming that wasn't the actual username and password combo?

Regards,

Jon

Jon,

1.) confirmed "/usr/share/tomcat/conf/Catalina/localhost/labkey.xml" is located correctly...

2.) Tomcat is installed from package.

[root@localhost bin]# java -cp /usr/share/java/tomcat/catalina.jar org.apache.catalina.util.ServerInfo
Server version: Apache Tomcat/7.0.54
Server built: May 12 2015 08:07:35
Server number: 7.0.54.0
OS Name: Linux
OS Version: 3.10.0-327.4.5.el7.x86_64
Architecture: amd64
JVM Version: 1.8.0_71-b15
JVM Vendor: Oracle Corporation


3.) Running "# psql -h localhost -p 5432 -U postgres" produces error below...

[root@localhost bin]# psql -h localhost -p 5432 -U postgres
psql: FATAL: Ident authentication failed for user "postgres"

Thanks,
Diego

Hi Diego,

I think that pg_hba.conf file is still misconfigured.

Any chance you can send us a copy of the bottom section of that file with all the configurations inside so we can look at it?

It most likely is probably using the Ident method rather than md5 as well.

Regards,

Jon

Jon,

pg-hba.conf file is attached!

Diego

Hi Diego,

Thanks for supplying the file. You will need to swap out all of the methods listed as ident on those additional two lines to md5 as well.

So from this:

# TYPE DATABASE USER ADDRESS METHOD

# "local" is for Unix domain socket connections only
local all all md5
# IPv4 local connections:
host all all 127.0.0.1/32 ident
# IPv6 local connections:
host all all ::1/128 ident

To this:

# TYPE DATABASE USER ADDRESS METHOD

# "local" is for Unix domain socket connections only
local all all md5
# IPv4 local connections:
host all all 127.0.0.1/32 md5
# IPv6 local connections:
host all all ::1/128 md5

Can you give this a try, restart postgresql again, then restart Tomcat?

Regards,

Jon

Jon,

Changed the Address Methods... But still getting data source errors. have attached latest logs... Let me know if you need other files.

You think it can be the following path in labkey.log?

INFO ViewServlet 2016-02-11 16:21:41,573 localhost-startStop-1 : PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
ERROR ExceptionUtil 2016-02-11 16:21:42,714 http-bio-8080-exec-1 : Unhandled exception: 500: Unexpected server error

Not sure what "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin" is...

Thanks,
Diego

Hi Diego,

Did you also restart the postgresql service as well?

The changes you made will only kick in after the postgresql service is also restarted.

You can verify this by running the same psql command to login again to see if it works:

psql -h localhost -p 5432 -U postgres

Regards,

Jon

psql -h localhost -p 5432 -U postgres --> prompts for password and logs in without an issue...

Does the password in the labkey.xml have to be in plain text or md5 hash? currently in plain text...

Hi Diego,

The username and password should be left alone there.

I am seeing a common error between both your Catalina logs and your LabKey.log file.

Can you add this 'factory="org.apache.commons.dbcp.BasicDataSourceFactory"' line in your LabKey.xml file under the jdbc/labkeyDataSource section?


    <Resource name="jdbc/labkeyDataSource" auth="Container"
        type="javax.sql.DataSource"
        username="dude"
        password="foobar"
        driverClassName="org.postgresql.Driver"
        url="jdbc:postgresql://localhost:5432/labkey"
        maxActive="20"
        maxIdle="10"
        accessToUnderlyingConnectionAllowed="true"
        validationQuery="SELECT 1"
        factory="org.apache.commons.dbcp.BasicDataSourceFactory"
        />

The package manager version of Tomcat doesn't play well with LabKey and I believe that this weird behavior is attributed to it.

Update the labkey.xml file with this line, restart Tomcat again and let us know how this goes.

Regards,

Jon

Jon,

Greatly appreciate your help! adding -> factory="org.apache.commons.dbcp.BasicDataSourceFactory" worked! Have attached a screenshot!

Thanks,
Diego

Hi Diego,

Great! So I suspect that your specific .jar file that is tied to tomcat-dhcp may possibly be corrupted. Apparently, this has been spotted before in the wild by others unrelated to LabKey:

http://stackoverflow.com/questions/14712308/ubuntu-tomcat7-java-lang-classnotfoundexception-org-apache-tomcat-dbcp-dbcp-bas

So you may need to update your specific tomcat-dhcp jar file to correct the issue so you don't have to use the factory option in your XML file. (Note: The example mentioned in StackOverflow was for tomcat-dbcp-7.0.30.jar since that was what the user had, so yours should be tomcat-dbcp-7.0.54.jar based on your version of Tomcat you mentioned earlier).

Regards,

Jon

Jon,

Will read the stackoverflow post, and correct the issue! Will let you know if I encounter any other issues!

Thanks,
Diego