9.1.0.0 | Services | HTTP Logging Service

The HTTP Logging Service enables logging of operator-configurable messages for every HTTP interaction.

Usage of this service is just a matter of configuring messages. Messages tell this service what to log and where to log it. Messages include a template to define what should be written to a log. Templates will be populated and rendered using data relevant to each HTTP interaction.

This service can be used for auditing all interactions with an API.

Configuration

Default Configuration: http-logging.cfg.xml
Released: TBD
Schema

Full Configuration

This configuration sets every available setting explicitly for the sake of detailing the available settings.

http-logging.cfg.xml

<http-logging xmlns="http://docs.openrepose.org/repose/http-logging/v1.0">
    <message (1)
        log-to="plain-access-log" (2)
        format="plain"> (3)
        Request handled!
    </message>
</http-logging>

1	Declares a message to be logged for every HTTP interaction. The value of this element serves as the content of the message. The value will be processed as a template. See Templating for more information.
2	Specifies the name of the logger to which messages should be logged. See Logging for more information.
3	Specifies the format of the content. The format determines if and how to validate the content and how to escape values populated by the templating engine. Supported values: `plain`, `json` Default: `plain`

Common Log Format (CLF)

This configuration includes a content template that conforms to the Common Log Format. This template shows how some variables can be accessed, and makes use of some JTwig functions.

http-logging.cfg.xml

<http-logging xmlns="http://docs.openrepose.org/repose/http-logging/v1.0">
    <message log-to="common-log" format="plain">
        {{ remoteIpAddress }} - {{ default(first(outboundRequestHeaders['x-user-id']), '-') }} [{{ timeRequestReceived }}] "{{ inboundRequestMethod }} {{ inboundRequestPath }} {{ inboundRequestProtocol }}" {{ outboundResponseStatusCode }} {{ default(outboundResponseContentLength, '-') }}
    </message>
</http-logging>

JSON Format

This configuration includes a content template that contains much of the same data as the Common Log Format (CLF) formatted in JSON. Rather than rendering - for an undefined value, this template uses JTwig’s conditional control flow mechanism to render null. If desired, the same mechanism could be used to omit a field rather than rendering a null value for that field.

http-logging.cfg.xml

<http-logging xmlns="http://docs.openrepose.org/repose/http-logging/v1.0">
    <message log-to="json-access-log" format="json">
<![CDATA[{
    "remoteIpAddress": "{{ remoteIpAddress }}",
    "userId": {; if (defined(outboundRequestHeaders['x-user-id'])) ;}"{{ first(outboundRequestHeaders['x-user-id']) }}"{; else ;}null{; endif ;},
    "timeRequestReceived": "{{ timeRequestReceived }}",
    "requestLine": "{{ inboundRequestMethod }} {{ inboundRequestPath }} {{ inboundRequestProtocol }}",
    "responseStatusCode": {{ outboundResponseStatusCode }},
    "responseContentLength": {; if (defined(outboundResponseContentLength)) ;}{{ outboundResponseContentLength }}{; else ;}null{; endif ;}
}]]>
    </message>
</http-logging>

Additional Information

Templating

In order for this service to adapt to a myriad of potential use-cases, content templates are supported. These templates are rendered using the JTwig templating engine. For details on how to write a JTwig template, please refer to the JTwig reference documentation.

For compatibility reasons, this service uses different delimiters than JTwig for control flow code islands. The following table shows the difference:

Table 1. Template Delimiters
	JTwig Default	HTTP Logging Service
Code	`{% code %}`	`{; code ;}`

The following table details every variable available for use in a content template.

Table 2. Template Variables
Name	Data Type	Value
inboundRequestProtocol	String	The name and version of the inbound request protocol. For example, for a request with the request line `GET / HTTP/1.1`, this value would be `HTTP/1.1`. Note that the name of the protocol will always be `HTTP`, but the version may vary.
outboundRequestProtocol	String	The name and version of the outbound request protocol. For example, for a request with the request line `GET / HTTP/1.1`, this value would be `HTTP/1.1`. Note that the name of the protocol will always be `HTTP`, but the version may vary.
inboundResponseProtocol	String	The name and version of the inbound response protocol. For example, for a response with the status line `HTTP/1.1 200 OK`, this value would be `HTTP/1.1`. Note that the name of the protocol will always be `HTTP`, but the version may vary.
inboundRequestMethod	String	The HTTP method of the inbound request. For example, for a request with the request line `GET / HTTP/1.1`, this value would be `GET`.
outboundRequestMethod	String	The HTTP method of the outbound request. For example, for a request with the request line `GET / HTTP/1.1`, this value would be `GET`.
inboundRequestPath	String	The path portion of the inbound request URL. For example, for a request with the request line `GET /some/path HTTP/1.1`, this value would be `/some/path`.
outboundRequestPath	String	The path portion of the outbound request URL. For example, for a request with the request line `GET /some/path HTTP/1.1`, this value would be `/some/path`.
inboundRequestQueryString	String	The query portion of the inbound request URL. For example, for a request with the request line `GET /path?when=now&what=it HTTP/1.1`, this value would be `when=now&what=it`.
outboundRequestQueryString	String	The query portion of the outbound request URL. For example, for a request with the request line `GET /path?when=now&what=it HTTP/1.1`, this value would be `when=now&what=it`.
inboundRequestUrl	String	The full inbound request URL minus the query portion. For example, for a request with the request line `GET /path?when=now HTTP/1.1` and the header `Host: example.com`, this value would be `http://example.com/path`.
outboundRequestUrl	String	The full outbound request URL minus the query portion. For example, for a request with the request line `GET /path?when=now HTTP/1.1` and the header `Host: example.com`, this value would be `http://example.com/path`.
inboundRequestHeaders	Map<String, List<String>>	The headers on the inbound request. The keys in this map are the header names while the values are lists of corresponding header values. The keys (i.e., header names) are case-sensitive and will only contain lowercase letters. Headers with multiple values spanning multiple lines will have a value list with one item for each line. For example, given the header lines: `Test-Header: foo, bar Test-Header: baz` The value of this variable would be: `{ "test-header": ["foo, bar", "baz"] }`
outboundRequestHeaders	Map<String, List<String>>	The headers on the outbound request. The keys in this map are the header names while the values are lists of corresponding header values. The keys (i.e., header names) are case-sensitive and will only contain lowercase letters. Headers with multiple values spanning multiple lines will have a value list with one item for each line. For example, given the header lines: `Test-Header: foo, bar Test-Header: baz` The value of this variable would be: `{ "test-header": ["foo, bar", "baz"] }`
outboundResponseHeaders	Map<String, List<String>>	The headers on the outbound response. The keys in this map are the header names while the values are lists of corresponding header values. The keys (i.e., header names) are case-sensitive and will only contain lowercase letters. Headers with multiple values spanning multiple lines will have a value list with one item for each line. For example, given the header lines: `Test-Header: foo, bar Test-Header: baz` The value of this variable would be: `{ "test-header": ["foo, bar", "baz"] }`
outboundResponseStatusCode	Integer	The status code of the outbound response. For example, for a response with the status line `HTTP/1.1 200 OK`, this value would be `200`.
outboundResponseReasonPhrase	String	The reason phrase of the outbound response. For example, for a response with the status line `HTTP/1.1 200 OK`, this value would be `OK`.
outboundResponseContentLength	Integer	The size of the outbound response body in bytes. This value is taken from the `Content-Length` header of the response. For example, for a response with the header line `Content-Length: 1234`, this value would be `1234`.
outboundResponseBytesWritten	Integer	The number of bytes of the outbound response body written.
userId	String	The identifier which uniquely identifies a user. This value is taken from the `X-User-Id` header of the outbound request. For example, given the header `X-User-Id: 1234`, the value of this variable would be `1234`.
userName	String	The name of a user, which is not necessarily unique. This value is taken from the `X-User-Name` header of the outbound request. For example, given the header `X-User-Name: IAmAUser`, the value of this variable would be `IAmAUser`.
impersonatorUserId	String	The identifier which uniquely identifies the impersonator user. This value is taken from the `X-Impersonator-Id` header of the outbound request. For example, given the header `X-Impersonator-Id: 4321`, the value of this variable would be `4321`.
impersonatorUserName	String	The name of the impersonator user, which is not necessarily unique. This value is taken from the `X-Impersonator-Name` header of the outbound request. For example, given the header `X-Impersonator-Name: IAmAnImpersonator`, the value of this variable would be `IAmAnImpersonator`.
timeRequestReceived	Instant	The time at which the inbound request was received. Evaluating this variable in an expression (i.e., `{{ timeRequestReceived }}`) will result in the value of Instant.toString being rendered.
timeToHandleRequest	Duration	The amount of time between receiving an inbound request and sending an outbound response. Evaluating this variable in an expression (i.e., `{{ timeToHandleRequest }}`) will result in the value of Duration.toString being rendered. This value can be rendered as the amount of time in milliseconds by calling the Duration.toMillis method (i.e., `{{ timeToHandleRequest.toMillis }}`).
timeInOriginService	Duration	The amount of time between sending an outbound request to the origin service and receiving an inbound response. Evaluating this variable in an expression (i.e., `{{ timeInOriginService }}`) will result in the value of Duration.toString being rendered. This value can be rendered as the amount of time in milliseconds by calling the Duration.toMillis method (i.e., `{{ timeToHandleRequest.toMillis }}`).
localIpAddress	String	The Internet Protocol (IP) address of the interface on which the inbound request was received.
remoteIpAddress	String	The Internet Protocol (IP) address of the client or last proxy that sent the inbound request.
remoteHost	String	The fully qualified name of the client or the last proxy that sent the inbound request.
traceId	String	The ID associated with the inbound request used for tracing purposes. This value will match the value from the decoded `X-Trans-Id` header.
extensions	Map<String, Object>	Extension data that does not correspond to any of the other defined variables. The data types of the values of this map are unknown. Using this map in a template will require coordination between the template author and the developer that added the desired mapping. This variable serves as an extension point which allows third-party developers to inject arbitrary data into this service.

If a the value of a variable cannot be resolved, the variable will be left undefined. When rendering a template, an undefined variable has specific semantics which are detailed in the JTwig reference documentation.

A distinction is made between inbound and outbound requests and responses. This distinction allows template authors to retrieve data associated with a request or response at a particular point during the processing of the HTTP interaction. The inbound qualifier denotes the point at which an HTTP message is received by Repose. The outbound qualifier denotes the point at which an HTTP message is sent by Repose. An inbound request and an outbound response define the interaction with an end-user while an outbound request and inbound response define the interaction with the origin service.

Whitespace in templates is preserved. However, it is possible to modify whitespace when rendering a template by leveraging Jtwig whitespace control and the Jtwig trim function. Whitespace can also be modified when a log message is being written using the Log4J2 Pattern Layout replace pattern. Those mechanisms should help in cases where it is desirable to include whitespace for configuration formatting that is not desired in the rendered message.

For Developers

This service introduces the concept of an HTTP logging context. An HTTP logging context is a container for data related to an individual HTTP interaction. An HTTP interaction is the exchange of an HTTP request and response.

While this service provides the means to open and close a logging context, it does not do so automatically. Logging contexts are opened and closed by code which uses this service and is hooked into the container lifecycle. While it is possible to create a new logging context at any time using this service, it is not the expected usage. Rather, the logging context created when a new request is received is expected to be used. That logging context will be passed around in a request attribute with the key org.openrepose.core.services.httplogging.context.

To be clear, this service only manages:

Configuration of logging parameters to answer the questions of what to log and where to log it.
Handling the processing of the logging context (i.e., rendering messages and sending those messages to the logging system) which answers the question of how to log.

Notably, this service does not manage:

The lifecycle of the logging context for an HTTP interaction which answers the question of when to log.
Collection of data related to an HTTP interaction which assists in answering the question of what to log.
- However, this service does provide the logging context interface to be used for data collection.

Management of those facets is left to developers intent on interacting with the logging system supported by this service.

There may be instances when it is desired to add data to the logging context that is not part of the predefined data set. For this reason, the logging context supports extension data. Extension data can be accessed and modified via the extensions field of the logging context. Extension data is not validated by this service; it is passed as a map directly to the templating engine.