Improved Access Logging for the LabKey Server Brian Connolly  2010-06-02 17:08
 

As you know, the LabKey Server runs on Apache Tomcat. The Tomcat server has the ability to perform Access Logging similar the Apache web server. When configured, the Tomcat server will log each request to the server to a file on the file system. By default:

  • the log file is located in the $CATALINA_HOME/logs directory
  • the log file name will be of the form localhost_access_log.YYYY-MM-DD.txt
    • where YYYY = 4 digit year, MM = month and DD is the day of the month.
  • the log file is rotated daily at midnight

The Tomcat server comes with two preconfigured Access Log formats; common and combined. If I had to choose one of these format, I would recommend the combined format. The format of the combined format is

%h %l %u %t "%r" %s %b Referer User-Agent'

See below for a description of what each pattern code (ie %h) means. Complete documentation on the Tomcat Access Logging can be found at http://tomcat.apache.org/tomcat-5.5-doc/config/valve.html


For production installations of the LabKey server, I recommend a new format:

%h %l %u %t "%r" %s %b %D %S "%{Referer}i" "%{User-Agent}i" %{LABKEY.username}s

The difference between this format and the combined format is

  • %D = The time taken by the server to process the request. This is valuable when debugging performance problems
  • %S = Tomcat User SESSION ID
  • %{LABKEY.username}s = The username of the person accessing the LabKey server. If the person is not logged in, then this will be a “-“

There are 3 other LabKey specific fields that might be of interest in particular circumstances

  • %{LABKEY.container}r = The LabKey server container being accessed during the request
  • %{LABKEY.controller}r = The LabKey server controller being used during the request
  • %{LABKEY.action}r = The LabKey server action being used during the request


Example Access Log Valve configuration

Below is an example configuration for using the pattern above

<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="localhost_access_log." 
suffix=".txt" pattern='%h %l %u %t "%r" %s %b %D %S "%{Referrer}i" "%{User-Agent}i" %{LABKEY.username}s' 
resolveHosts="false"/>


Pattern Code description

This was taken from http://tomcat.apache.org/tomcat-5.5-doc/config/valve.html

  • %a - Remote IP address
  • %A - Local IP address
  • %b - Bytes sent, excluding HTTP headers, or ‘-’ if zero
  • %B - Bytes sent, excluding HTTP headers
  • %h - Remote host name (or IP address if resolveHosts is false)
  • %H - Request protocol
  • %l - Remote logical username from identd (always returns ‘-‘)
  • %m - Request method (GET, POST, etc.)
  • %p - Local port on which this request was received
  • %q - Query string (prepended with a ‘?’ if it exists)
  • %r - First line of the request (method and request URI)
  • %s - HTTP status code of the response
  • %S - User session ID
  • %t - Date and time, in Common Log Format
  • %u - Remote user that was authenticated (if any), else ‘-‘
  • %U - Requested URL path
  • %v - Local server name
  • %D - Time taken to process the request, in millis
  • %T - Time taken to process the request, in seconds
  • %I - current request thread name (can compare later with stacktraces)

There is also support to write information from the cookie, incoming header, outgoing response headers, the Session or something else in the ServletRequest. It is modeled after the apache syntax:

  • %{xxx}i for incoming request headers
  • %{xxx}o for outgoing response headers
  • %{xxx}c for a specific request cookie
  • %{xxx}r xxx is an attribute in the ServletRequest
  • %{xxx}s xxx is an attribute in the HttpSession