Site HTTP Access Logs: /Documentation

LabKey Server can log each HTTP request sent to the server, including requests from logged in users, anonymous guest users, and API calls into the server.

Overview
Recommended LabKey Server Access Log Format
Example Access Log Valve Configuration
Pattern Code Description
Obtaining Source IP Address When Using a Load Balancer

Overview

Information is written to a text file on the file system using Tomcat's native access logging. Access logs can include details such as queries requested, user information, server actions invoked, etc. By default:

the log file is located in the $CATALINA_HOME/logs directory
the log file name includes the date: localhost_access_log.YYYY-MM-DD.txt
the log file is rotated daily at midnight

The information written to the log, and its format, is controlled by the <Valve> element in the configuration file:

CATALINA_HOME/conf/server.xml

Tomcat's default log format has the following pattern code:

%h %l %u %t "%r" %s %b

which provides log entries like the following:

0:0:0:0:0:0:0:1 - - [16/Jul/2020:11:45:35 -0700] "GET /labkey/home/helloworld/
query-getQuery.api?dataRegionName=query&query.queryName=things&schemaName=helloworld
&query.color%2Fname~eq=Red&query.columns=name&apiVersion=9.1 HTTP/1.1" 200 2746

See below for a description of what each pattern code means. Complete documentation on the Tomcat Access Logging can be found at Access Log Valve

Recommended LabKey Server Access Log Format

For production installations of the LabKey server, we recommend the following format. Because Tomcat uses an XML configuration file, the quotes need to be encoded. See the <Valve> example below for guidance.

%h %l %u %t "%r" %s %b %D %S "%{Referer}i" "%{User-Agent}i" %{LABKEY.username}s

This format introduces the following elements:

%D = The time taken by the server to process the request. This is valuable when debugging performance problems
%S = Tomcat user session ID
%{Referer}i = Referring URL - Note that the misspelling of "Referer" is intentional
%{User-Agent}I = The name of the client program used to make the HTTP request
%{LABKEY.username}s = The username of the person accessing the LabKey server. If the person is not logged in, then this will be a "-"

Also consider 3 other LabKey specific fields that might be of interest in particular circumstances:

%{LABKEY.container}r = The LabKey server container being accessed during the request
%{LABKEY.controller}r = The LabKey server controller being used during the request
%{LABKEY.action}r = The LabKey server action being used during the request

Example Access Log Valve Configuration

Below is an example configuration for using the fields mentioned above:

<Valve className="org.apache.catalina.valves.AccessLogValve" 
directory="logs" prefix="localhost_access_log." suffix=".txt" 
pattern="%h %l %u %t &quot;%r&quot; %s %b %D %S &quot;%{Referer}i&quot; &quot;%{User-Agent}i&quot; %{LABKEY.username}s" 
resolveHosts="false"/>

This configuration provides log entries like the following:

0:0:0:0:0:0:0:1 - - [03/Aug/2020:10:11:16 -0700] "GET /labkey/HiWorld/query-getQuery.api?
dataRegionName=query&query.queryName=things&schemaName=helloworld&query.color%2Fname~eq=Red&quer
y.columns=name&apiVersion=9.1 HTTP/1.1" 200 2714 186 D9003A0B7132A3F98D1E63E613859230 "-" 
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) 
Chrome/84.0.4147.89 Safari/537.36" username@labkey.com

Pattern Code Description

Reference information below was taken from the Tomcat docs at: Access Log Valve

%a - Remote IP address
%A - Local IP address
%b - Bytes sent, excluding HTTP headers, or ‘-’ if zero
%B - Bytes sent, excluding HTTP headers
%h - Remote host name (or IP address if resolveHosts is false)
%H - Request protocol
%l - Remote logical username from identd (always returns ‘-‘)
%m - Request method (GET, POST, etc.)
%p - Local port on which this request was received
%q - Query string (prepended with a ‘?’ if it exists)
%r - First line of the request (method and request URI)
%s - HTTP status code of the response
%S - User session ID
%t - Date and time, in Common Log Format
%u - Remote user that was authenticated (if any), else ‘-‘
%U - Requested URL path
%v - Local server name
%D - Time taken to process the request, in millis. Note that in httpd %D is microseconds.
%T - Time taken to process the request, in seconds
%I - current request thread name (can compare later with stacktraces)

There is also support to write information from the cookie, incoming header, outgoing response headers, the Session or something else in the ServletRequest. It is modeled after the Apache syntax:

%{xxx}i for incoming request headers
%{xxx}o for outgoing response headers
%{xxx}c for a specific request cookie
%{xxx}r xxx is an attribute in the ServletRequest
%{xxx}s xxx is an attribute in the HttpSession

Obtaining Source IP Address When Using a Load Balancer

When using a load balancer, such as AWS ELB or ALB, you may notice that your logs are capturing the IP address of the load balancer itself, rather than the user's originating IP address. To obtain the source IP address, first confirm that your load balancer is configured to preserve source IPs:

One possible valve configuration pattern to use is:

pattern="%{org.apache.catalina.AccessLog.RemoteAddr}r %l %u %t "%r" %s %b %D %S "%{Referer}i" "%{User-Agent}i" %{LABKEY.username}s %q"

LabKey Support

LabKey Support