Developers using the Java client API should be aware that we've made some significant changes to this library. Earlier this month, we migrated the underlying HTTP library it uses from Apache Commons HttpClient 3.1 to Apache HttpComponents HttpClient 4.3.5. The 4.x project was started (10 years ago) to replace 3.x, which is now "end of life" and hasn't seen any security, reliability, or feature updates since 2007. https://hc.apache.org/httpclient-legacy/index.htmlThe conversion was complicated by the fact that 4.x was a complete rewrite, introducing new concepts, usage patterns, and class hierarchy. We made every effort to hide these changes from mainstream users of our Java client API. However, a few of our public and protected methods exposed HttpClient 3.x classes that have been removed (HttpMethod, HttpConnectionManager, and the old HttpClient), making 100% compatibility impossible. We encourage early testing of this new version, especially if you've extended our classes (e.g., implemented your own Commands). Please report any issues you find with the new library. We will officially release the new client library alongside LabKey Server 15.1, but the changes can be tested today by building from trunk. Adam |
|
jeckels responded: |
2014-11-22 17:26 |
|
|
|
Andy Straw responded: |
2015-05-14 10:14 |
We are finally migrating to 15.1, and have some code in a standalone client app that is affected by this change in http library. We have a List in LabKey in which one field is of type File; we manually upload a text file to this field when we insert a new record to this List. Our app that uses the LabKey client API and the http library can calculate the URL to fetch this file, and wants to read its contents. Our code that works with LabKey 14.3 and HttpClient 3.x is the following: GetMethod get = new GetMethod( urlString ); labkeyConnection.executeMethod( get ); String fileContents = get.getResponseBodyAsString(); where urlString is the URL as a String, labkeyConnection is a org.labkey.remoteapi.Connection, properly authenticated via username and password. I think I figured out most of how to re-write this using the new API - except for authentication. Here is what I have so far (without error/exception handling): CloseableHttpClient httpClient = labkeyConnection.getHttpClient( 60 ); HttpGet get = new HttpGet( urlString ); CloseableHttpResponse response = httpClient.execute( get ); HttpEntity entity = response.getEntity(); String fileContents = EntityUtils.toString( httpEntity ); When I execute this, the fileContents appears to be the login page, rather the file I want. I started looking at the docs for how to do authentication with the new HttpClient APIs here: https://hc.apache.org/httpcomponents-client-ga/tutorial/html/authentication.html and it seems like rather a lot of code (CredentialsProvider, AuthCache, BasicScheme, HttpClientContext, etc.), and I'm wondering why I have to re-authenticate, if I'm using the CloseableHttpClient object gotten from the LabKey Connection object - to which I've already provided host, port, username and password. Am I doing something wrong? Is there an easier/different way to do this with the new APIs? Thanks. Andy Straw
University of Rochester |
|
Andy Straw responded: |
2015-05-14 12:29 |
I was able to get authentication to work, but this required another 10 lines of code to properly create the HttpClientContext that is needed as a second parameter the execute() method of CloseableHttpClient. Since the code to create the context object requires only the LabKey Connection object, it would be nice if your API somehow could do that for us. Here's the way I factored this into a helper method: private static HttpClientContext buildHttpContext( Connection labkeyConnection ) throws MalformedURLException { URL labkeyBaseUrl = new URL( labkeyConnection.getBaseUrl() ); HttpHost labkeyHost = new HttpHost( labkeyBaseUrl.getHost(), labkeyBaseUrl.getPort(), labkeyBaseUrl.getProtocol() ); CredentialsProvider credsProvider = new BasicCredentialsProvider(); credsProvider.setCredentials( new AuthScope(labkeyHost), new UsernamePasswordCredentials(labkeyConnection.getEmail(), labkeyConnection.getPassword()) ); AuthCache authCache = new BasicAuthCache(); authCache.put( labkeyHost, new BasicScheme()); HttpClientContext httpContext = HttpClientContext.create(); httpContext.setCredentialsProvider(credsProvider); httpContext.setAuthCache(authCache); return httpContext; } Andy |
|
jeckels responded: |
2015-05-15 10:44 |
Hi Andy, I think this is due to a change in the way that HttpClient 4.x deals with "preemptive" authentication, relative to the approach in 3.x. I have an idea of a way to make this easier within the Java client API, but would like to test it with the same URL that you're hitting in this case. From your code, I'm assuming that it's not an "API" URL, which means that it won't challenge the client for Basic Auth credentials, resulting in sending back the login page. Can you share the controller/action from your URL? Thanks,
Josh |
|
Andy Straw responded: |
2015-05-15 11:04 |
Here is an example URL. The URL is generated by LabKey as the value of a hidden column in the list, and corresponds to the download link for the File property. For example, the File property is called "TemplateFile", and the URL ends up in a property called "_labkeyurl_TemplateFile": /list/MyContainerPathGoesHere/download.view?entityId=a8cd2f13-a660-1032-96ac-a38b824db812&name=GateTemplate.txt I have run into another problem with this. The code above works from an external program where I can specify the username and password when I run the app. But it doesn't work when part of a transformation script where I'm authenticating by grabbing the JSESSIONID cookie name and value off the script command line, and calling addCookie() on the Connection. Calling addCookie() works if I'm using the Connection with the Labkey client API (authenticates fine), but does not work with the CloseableHttpClient object I get from that Connection. Just as when using username/password, that Client object does not have any of the authentication info that I gave the Connection, and so I have to (essentially) re-authenticate separately with the Client. So far, I haven't figure out how to do that using a cookie. As I said before: in 14.3, I didn't have to jump through any of these hoops. The executeMethod() method on Connection did all the hard work for me; but that method is no longer in the Connection API as of 15.1. Thanks. Andy |
|
Andy Straw responded: |
2015-05-15 11:29 |
Looking at the source for Connection, I see code similar to my buildHttpContext() method above in your executeRequest() method (works with username/password), and I see code similar to what I need to do for using the Cookie in addCookie(). But I have no way to use/reuse your code. Could you make something like your executeRequest() method public? That might be just what I need - though not sure about the Cookie case (i.e., case where getEmail() is null in executeRequest()). Andy |
|
jeckels responded: |
2015-05-20 09:19 |
Hi Andy, With revision 38038 in the trunk, I made a change that enables this usage pattern: Connection cn = Connection("http://localhost/labkey", "jeckels@labkey.com", "PASSWORD");
CloseableHttpClient httpClient = cn.ensureAuthenticated();
HttpGet get = new HttpGet(url);
CloseableHttpResponse response = httpClient.execute(get);
HttpEntity entity = response.getEntity();
String fileContents = EntityUtils.toString(entity); I think this fits what you've requested, but please let me know. Thanks,
Josh |
|
Andy Straw responded: |
2015-05-20 09:32 |
Thanks. Will it also work if the LabKey Connection is authenticated using a session cookie, like so? // Create connection and authenticate with cookie, instead of username + password. Connection cn = Connection("http://localhost/labkey"); cn.addCookie( "JSESSIONID", sessionId, "localhost", "/labkey/", null, false ); // Code is same from here. CloseableHttpClient httpClient = cn.ensureAuthenticated(); HttpGet get = new HttpGet(url); CloseableHttpResponse response = httpClient.execute(get); HttpEntity entity = response.getEntity(); String fileContents = EntityUtils.toString(entity); |
|
jeckels responded: |
2015-05-20 10:08 |
I haven't tested, but I expect that will work. The primary change is to make all of the HttpClient instances share the same CookieStore, so calling addCookie() should do the trick. Thanks,
Josh |
|
|
|