see

https://www.labkey.org/issues/home/Developer/issues/details.view?issueId=7061

thanks for the java sample pointer

sorry, should have poked around a bit more - OTOH it did turn up Josh's pointer to org.labkey.api.util.HttpsUtil.disableValidation(), maybe I'll take a poke at that if nobody else is planning to.

Hi Brian,

I just added that comment a few minutes ago - I wouldn't have known to share that tidbit without your posting, so it's a good thing you did.

Thanks,
Josh

I wasn't planning on addressing this in the 9.2 branch. If you do work out a fix, I'll happily move the fix to trunk.

I usually work in trunk and put things into branches once they're deemed important and not destabilizing, but I'll follow whatever model you guys do - branching policy is the stuff of jihads.

That statement probably makes more sense when I say that up till now I've been using nothing but LabKey binary distros on the EC2 side of things... I'm not working in the branch per se, just using its official build artifacts to be maximally customer-like.

Either way is fine, I just assumed you were working with the 9.2 build (hence a branch). Or if you just wait 9.3 will likely have a fix.

I'll go ahead and take a shot at it with Josh's hint. Probably sometime next week when IntelliJ has finally finished startup (my god, its slow to load - is there something I'm not understanding?).

It's slow. Use the latest version and a fast machine. But even better, the upload applet is in its own very small project under (labkey)/server/applets/applets.ipr.

We are running into this problem: exception from the java applet file uploader tool. This is our staging server. Initially, we used a self-signed certificate, and got both the browser warning, and the applet exception. We obtained a valid certificate from inCommon, and that made the browser warning go away, but we still get the applet exception - though the exception message is different - see below for stacktrace.

I imported the inCommon CA certificate into the cacerts file of my local JRE, but that didn't seem to help. Though it's a little unclear to me where the certificate should be imported to affect a java applet run from my browser.

I see that an issue was opened, but not fixed related to this: https://www.labkey.org/issues/home/Developer/issues/details.view?issueId=7061

Is there a work-around for this? We don't have a problem with our production server that uses a different certificate purchased from Verisign.

Here's the stack trace:

javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Untrusted Server Certificate Chain
                at com.sun.net.ssl.internal.ssl.Alerts.getSSLException(Unknown Source)
                at com.sun.net.ssl.internal.ssl.SSLSocketImpl.fatal(Unknown Source)
                at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Unknown Source)
                at com.sun.net.ssl.internal.ssl.Handshaker.fatalSE(Unknown Source)
                at com.sun.net.ssl.internal.ssl.ClientHandshaker.serverCertificate(Unknown Source)
                at com.sun.net.ssl.internal.ssl.ClientHandshaker.processMessage(Unknown Source)
                at com.sun.net.ssl.internal.ssl.Handshaker.processLoop(Unknown Source)
                at com.sun.net.ssl.internal.ssl.Handshaker.process_record(Unknown Source)
                at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(Unknown Source)
                at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(Unknown Source)
                at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(Unknown Source)
                at com.sun.net.ssl.internal.ssl.AppOutputStream.write(Unknown Source)
                at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
                at java.io.BufferedOutputStream.flush(Unknown Source)
                at org.apache.commons.httpclient.methods.StringRequestEntity.writeRequest(StringRequestEntity.java:146)
                at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:499)
                at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2114)
                at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1096)
                at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
                at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
                at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
                at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
                at org.labkey.applets.drop.FTPJobQueue.propFind(FTPJobQueue.java:982)
                at org.labkey.applets.drop.FTPJobQueue$DavRunnableJob._sendFile(FTPJobQueue.java:440)
                at org.labkey.applets.drop.FTPJobQueue$DavRunnableJob$1.run(FTPJobQueue.java:370)
                at java.security.AccessController.doPrivileged(Native Method)
                at org.labkey.applets.drop.FTPJobQueue$DavRunnableJob.run(FTPJobQueue.java:366)
                at org.labkey.applets.drop.FTPJobQueue$MyThreadPoolExecutor$2.call(FTPJobQueue.java:1253)
                at org.labkey.applets.drop.FTPJobQueue$Future.run(FTPJobQueue.java:1156)
                at org.labkey.applets.drop.FTPJobQueue$MyThreadPoolExecutor$1.run(FTPJobQueue.java:1202)
                at java.lang.Thread.run(Unknown Source)
 
Caused by: java.security.cert.CertificateException: Untrusted Server Certificate Chain
                at com.sun.net.ssl.X509TrustManagerJavaxWrapper.checkServerTrusted(Unknown Source)
                ... 27 more

I've not seen any response to my previous post (immediately before this one). We're now using an InCommon cert in production, and the mutli-file upload applet now fails in production with this error. (Previously, we had a Verisign cert in production, and an InCommon cert in dev/staging, and only had problems in dev/staging.)

Does anyone have any ideas on how we can fix or work around this? Why is the browser happy with the cert, but the applet not?

Thanks for any help anyone can provide. Our users must upload one file at a time, which, of course, can be quite tedious if you have lots of files to upload.

Andy

Hi Andy,

The problem is that the browser trusts InCommon as a certificate authority, but the JRE that's running the applet doesn't. Thus, the applet tries to make a connection to the server over HTTPS, it blows up with the exception during certificate validation.

The JRE stores the list of trusted certificate authorities in <JRE_HOME>/lib/security/cacerts. It's possible to manually add a new certificate to the file, but it would need to be done on all the computers that are running the applet.

http://docs.oracle.com/javase/1.5.0/docs/tooldocs/solaris/keytool.html

Different versions of the JRE have a different set of trusted CA certificates, so it's possible that installing a newer version of the JRE or Java PlugIn might fix the problem.

It would be possible to build a version of the applet that disables SSL certificate validation, but I wouldn't recommend doing this on a production server as it introduces a potential vulnerability.

Thanks,
Josh

Andy,

You can also import a new certificate on all computers running the applet via the Java Control Panel. See http://docs.oracle.com/javase/6/docs/technotes/guides/deployment/deployment-guide/jcp.html#security

When I did a little more research, it looks like InCommon certificates have been trusted by the JRE since v1.4.2. See https://spaces.internet2.edu/display/InCCollaborate/InCommon+Cert+Service+FAQ#InCommonCertServiceFAQ-Whatarethesupportedbrowsers%2Cdevicesandapplicationsuites%3F

I wonder if there is a SSL configuration problem on your server which might be causing this problem. The InCommon documentation says this type of behavior might occur if all the InCommon Intermediate certificates are not properly installed on the server See https://spaces.internet2.edu/display/InCCollaborate/InCommon+Cert+Service+FAQ#InCommonCertServiceFAQ-WhyisthebrowserrejectingmySSL%2FTLScertificate%3F for description of what the problem might be and steps on verifying that the SSL certificates are configured properly on your server.

Brian

Thanks for the responses.

Having each user install a certificate client-side is not feasible for us, nor should it be necessary, correct?

I upgraded my Java plug-in to 1.7 (was using 1.6) and that did not help. Based on Brian's comment, that should not have mattered anyway - we're all using at least Java 1.5.

I followed the link in Brian's last paragraph, and used openssl to verify our certificates. I ran this command (after downloading the AddTrustExternalCARoot.crt file):

openssl s_client -connect blis.urmc.rochester.edu:443 -CAfile AddTrustExternalCARoot.crt

Output of that command is attached. From what I could tell, it looks like our server is configured properly - but I have to admit, I don't know how to interpret most of the output from openssl. Since we've not seen any issues or pop-ups or warnings from our browsers - just from the applet - I'm not surprised openssl seemed to think the cert chain was okay, since the browsers seem to think so as well.

Is there something else we can do to help diagnose this problem? Are we sure the applet is doing the write thing to deal with the cert chain? I notice the stack trace has httpclient classes in it - could the problem be there?

Thanks.

Andy

Andy,
I believe there is a problem with your SSL configuration. At the top of the openssl output, you will notice

Certificate chain
 0 s:/C=US/postalCode=14627/ST=NY/L=Rochester/O=University of Rochester/OU=Center for Integrated Research Computing/CN=blis.urmc.rochester.edu
   i:/C=US/O=Internet2/OU=InCommon/CN=InCommon Server CA
 1 s:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
   i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
 2 s:/C=US/O=Internet2/OU=InCommon/CN=InCommon Server CA
   i:/C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root


The order of your certificate chain is incorrect. Certificate #2 should be above certificate #1. Most SSL client implementations, such as the ones in browsers, can handle having the certificate chain in the wrong order, but JAVA is much more picky and is erroring out due to this misconfiguration.

If you fix this, I believe that the file upload applet will correctly. I recommend creating a new keystore file with the correct order and then configuring your server to use the new keystore. Given that you have a test server, which is experiencing the same error, you can probably test the new keystore on that server.

Thank you,

Brian

PS when running the openssl command in the future, use `openssl s_client -connect blis.urmc.rochester.edu:443` without specifying the CAFile. This will give you a true test of your SSL configuration.

Thanks very much. (I actually noticed this difference, but the last line of the output said "ok", so I assumed it was ok.)

As soon as I can get my sys admin to re-configure the cert chain on our server, we'll try this fix, and let you know how that goes.

Andy

When we switched the order of the certs in the chain, that fixed the problem.

Thanks again for your help with this. I doubt we would have been able to solve this so quickly without your help.

Andy