Red Hat SummitChapter 12. The Logging Subsystem
12.1. Introduction
12.1.1. Overview of Logging
JBoss EAP 6 provides highly configurable logging facilities for both its own internal use and for use by deployed applications. The logging subsystem is based on JBoss LogManager and it supports several third party application logging frameworks in addition to JBoss Logging.
The logging subsystem is configured using a system of log categories and log handlers. Log categories define what messages to capture, and log handlers define how to deal with those messages (write to disk, send to console etc).
Logging Profiles allow uniquely named sets of logging configuration to be created and assigned to applications independent of any other logging configuration. The configuration of logging profiles is almost identical to the main logging subsystem.
12.1.2. Application Logging Frameworks Supported By JBoss LogManager
JBoss LogManager supports the following logging frameworks:
- JBoss Logging - included with JBoss EAP 6
- Apache Commons Logging - http://commons.apache.org/logging/
- Simple Logging Facade for Java (SLF4J) - http://www.slf4j.org/
- Apache log4j - http://logging.apache.org/log4j/1.2/
- Java SE Logging (java.util.logging) - http://download.oracle.com/javase/6/docs/api/java/util/logging/package-summary.html
JBoss LogManager supports the following APIs:
- java.util.logging
- JBoss Logging
- Log4j
- SLF4J
- commons-logging
JBoss LogManager also supports the following SPIs:
- java.util.logging Handler
- Log4j Appender
Note
If you are using the
Log4j API and a Log4J Appender, then Objects will be converted to string before being passed.
12.1.3. Bootup Logging
During bootup JBoss EAP outputs log entries about the Java environment and the startup of each service. The log can be useful when troubleshooting. By default all log entries are written to the file
server.log, the location of which depends on the runtime mode.
- Standalone mode
EAP_HOME/standalone/log/server.log- Domain mode
EAP_HOME/domain/servers/SERVER_NAME/log/server.log
The configuration of bootup logging is specified in the configuration file
logging.properties, the location of which depends on the runtime mode.
- Standalone mode
EAP_HOME/standalone/configuration/logging.properties- Domain mode
- In domain mode there is a
logging.propertiesfile for the domain controller and each server.Domain controller:EAP_HOME/domain/configuration/logging.propertiesServer:EAP_HOME/domain/servers/SERVER_NAME/data/logging.properties
The configuration file
logging.properties is active until the logging subsystem is started and takes over.
Warning
It is recommended that you do not directly edit the
logging.properties file unless you know of a specific use case that requires you to do so. Before doing so, it is recommended that you raise a Support Case.
Changes made manually to the
logging.properties file are overwritten on startup.
12.1.4. View Bootup Errors
When troubleshooting JBoss EAP, checking for errors which occurred during bootup should be one of the first steps taken. There are two methods of viewing bootup errors, each with its advantages. Each method results in a list of any errors which occurred during bootup. Use the information provided to diagnose and resolve their causes. Contact Red Hat Customer Support for assistance in troubleshooting.
- Examine the
server.loglog file.This method allows you to see each error message together with possibly related messages, allowing you to get more information about why an error might have occurred. It also allows you to see error messages in plain text format. - From JBoss EAP 6.4, use the Management CLI command
read-boot-errors.This method does not require access to the server's file system, which is useful for anyone responsible for monitoring for errors who does not have file system access. Since it is a Management CLI command, it can be used in a script. For example, you could write a script which starts multiple JBoss EAP instances, then checks for errors which occurred on bootup.
Procedure 12.1. Examine server.log for Errors
- Open the file
server.login a file viewer. - Navigate to the end of the file.
- Search backward for the message identifier
JBAS015899, which marks the start of the latest bootup sequence. - Search the log from that point onward for instances of
ERROR. Each instance will include a description of the error and list the modules involved.
Example 12.1. Error Description from server.log
The following is an example error description from the
server.log log file.
13:23:14,281 ERROR [org.apache.coyote.http11.Http11Protocol] (MSC service thread 1-4) JBWEB003043: Error initializing endpoint: java.net.BindException: Address already in use /127.0.0.1:8080
Procedure 12.2. List Bootup Errors via the Management CLI
- Run the following Management CLI command.
/core-service=management:read-boot-errors
Any errors which occurred during bootup will be listed.The timestamp of each error uses the Java methodcurrentTimeMillis(), which is the difference, measured in milliseconds, between the current time and midnight, January 1, 1970 UTC(coordinated universal time).
Example 12.2. Output from read-boot-errors Command
{
"outcome" => "success",
"result" => [{
"failed-operation" => {
"operation" => "add",
"address" => [
("subsystem" => "web"),
("connector" => "http")
]
},
"failure-timestamp" => 1417560953245L,
"failure-description" => "{\"JBAS014671: Failed services\" => {\"jboss.web.connector.http\" => \"org.jboss.msc.service.StartException in service jboss.web.connector.http: JBAS018007: Error starting web connector
Caused by: LifecycleException: JBWEB000023: Protocol handler initialization failed\"}}",
"failed-services" => {"jboss.web.connector.http" => "org.jboss.msc.service.StartException in service jboss.web.connector.http: JBAS018007: Error starting web connector
Caused by: LifecycleException: JBWEB000023: Protocol handler initialization failed"}
}]
}12.1.5. About Garbage Collection Logging
Garbage collection logging logs all garbage collection activity to plain text log files. These log files can be useful for diagnostic purposes. From JBoss EAP 6 garbage collection logging is enabled by default for
standalone mode on all supported configurations except IBM Java development kit.
Logging is output to the file
EAP_HOME/standalone/log/gc.log.digit. Log rotation has been enabled, with the number of log files limited to five and each file limited to a maximum size of three MiB.
12.1.6. Implicit Logging API Dependencies
The JBoss EAP 6 logging subsystem has the
add-logging-api-dependencies attribute that controls whether the container adds implicit logging API dependencies to deployments. By default this attribute is set to true, which means that all implicit logging API dependencies are added to deployments. If set to false, implicit logging API dependencies will not be added.
The
add-logging-api-dependencies attribute can be configured using the Management CLI. For example:
/subsystem=logging:write-attribute(name=add-logging-api-dependencies, value=false)12.1.7. Default Log File Locations
These are the log files that get created for the default logging configurations. The default configuration writes the server log files using periodic log handlers
| Log File | Description |
|---|---|
EAP_HOME/standalone/log/server.log |
Server Log. Contains all server log messages, including server startup messages.
|
EAP_HOME/standalone/log/gc.log |
Garbage collection log. Contains details of all garbage collection.
|
| Log File | Description |
|---|---|
EAP_HOME/domain/log/host-controller.log |
Host Controller boot log. Contains log messages related to the startup of the host controller.
|
EAP_HOME/domain/log/process-controller.log |
Process controller boot log. Contains log messages related to the startup of the process controller.
|
EAP_HOME/domain/servers/SERVERNAME/log/server.log |
The server log for the named server. Contains all log messages for that server, including server startup messages.
|
12.1.8. Filter Expressions for Logging
Filter expressions are used to record log messages based on various criterion. Filter checking is always done on a raw unformatted message. You can include a filter for a logger or handler, the logger filter takes precedence over the filter put on a handler.
Note
A
filter-spec specified for the root logger is not inherited by other loggers. Instead a filter-spec must be specified per handler.
|
Filter Type
expression
| Description | Parameters |
|---|---|---|
|
Accept
accept
| Accept all log messages | accept
|
|
Deny
deny
| Deny all log messages | deny
|
|
Not
not[filter expression]
| Returns the inverted value of the filter expression |
Takes single filter expression as a parameter
not(match("JBAS"))
|
|
All
all[filter expression]
| Returns concatenated value from multiple filter expressions. |
Takes multiple filter expressions delimited by commas
all(match("JBAS"),match("WELD"))
|
|
Any
any[filter expression]
| Returns one value from multiple filter expressions. |
Takes multiple filter expressions delimited by commas
any(match("JBAS"),match("WELD"))
|
|
Level Change
levelChange[level]
| Modifies the log record with the specified level |
Takes single string-based level as an argument
levelChange("WARN")
|
|
Levels
levels[levels]
| Filters log messages with a level listed in the list of levels |
Takes multiple string-based levels delimited by commas as argument
levels("DEBUG","INFO","WARN","ERROR")
|
|
Level Range
levelRange[minLevel,maxLevel]
| Filters log messages within the specified level range. |
The filter expression uses
[ to indicate a minimum inclusive level and a ] to indicate a maximum inclusive level. Alternatively, one can use ( or ) respectively to indicate exclusive. The first argument for the expression is the minimum level allowed, the second argument is the maximum level allowed.
Examples are shown below.
|
Match (match["pattern"]) | A regular-expression based filter. The unformatted message is used against the pattern specified in the expression. |
Takes a regular expression as argument
match("JBAS\d+")
|
Substitute (substitute["pattern","replacement value"]) | A filter which replaces the first match to the pattern with the replacement value |
The first argument for the expression is the pattern the second argument is the replacement text
substitute("JBAS","EAP")
|
Substitute All (substituteAll["pattern","replacement value"]) | A filter which replaces all matches of the pattern with the replacement value |
The first argument for the expression is the pattern the second argument is the replacement text
substituteAll("JBAS","EAP")
|
Note
Escape the comma and quotation marks in the value (by preceding them with the '\' operator) and wrap the entire expression in quotation marks, so that the value is correctly interpreted to be a
string. Otherwise, it would be parsed as a list. An example of the correct format would be:
[standalone@localhost:9999 /] /subsystem=logging/console-handler=CONSOLE:write-attribute(name=filter-spec, value="substituteAll(\"JBAS\"\,\"SABJ\")")
12.1.9. About Log Levels
Log levels are an ordered set of enumerated values that indicate the nature and severity of a log message. The level of a given log message is specified by the developer using the appropriate methods of their chosen logging framework to send the message.
JBoss EAP 6 supports all the log levels used by the supported application logging frameworks. The most commonly used six log levels are (in order of lowest to highest):
TRACE, DEBUG, INFO, WARN, ERROR and FATAL.
Log levels are used by log categories and handlers to limit the messages they are responsible for. Each log level has an assigned numeric value which indicates its order relative to other log levels. Log categories and handlers are assigned a log level and they only process log messages of that level or higher. For example a log handler with the level of
WARN will only record messages of the levels WARN, ERROR and FATAL.
12.1.10. Supported Log Levels
| Log Level | Value | Description |
|---|---|---|
| FINEST | 300 |
-
|
| FINER | 400 |
-
|
| TRACE | 400 |
Use for messages that provide detailed information about the running state of an application. Log messages of
TRACE are usually only captured when debugging an application.
|
| DEBUG | 500 |
Use for messages that indicate the progress individual requests or activities of an application. Log messages of
DEBUG are usually only captured when debugging an application.
|
| FINE | 500 |
-
|
| CONFIG | 700 |
-
|
| INFO | 800 |
Use for messages that indicate the overall progress of the application. Often used for application startup, shutdown and other major lifecycle events.
|
| WARN | 900 |
Use to indicate a situation that is not in error but is not considered ideal. May indicate circumstances that may lead to errors in the future.
|
| WARNING | 900 |
-
|
| ERROR | 1000 |
Use to indicate an error that has occurred that could prevent the current activity or request from completing but will not prevent the application from running.
|
| SEVERE | 1000 |
-
|
| FATAL | 1100 |
Use to indicate events that could cause critical service failure and application shutdown and possibly cause JBoss EAP 6 to shutdown.
|
12.1.11. About Log Categories
Log categories define a set of log messages to capture and one or more log handlers which will process the messages.
The log messages to capture are defined by their Java package of origin and log level. Messages from classes in that package and of that log level or lower are captured by the log category and sent to the specified log handlers.
Log categories can optionally use the log handlers of the root logger instead of their own handlers.
12.1.12. About the Root Logger
The root logger captures all log messages sent to the server (of a specified level) that are not captured by a log category. These messages are then sent to one or more log handlers.
By default the root logger is configured to use a console and a periodic log handler. The periodic log handler is configured to write to the file
server.log. This file is sometimes referred to as the server log.
12.1.13. About Log Handlers
Log handlers define how captured log messages are recorded. The available log handlers are:
Console, File, Periodic, Size, Async, syslog, Periodic Size and Custom.
Note
A log handler must be added to at least one logger to be active.
12.1.14. Types of Log Handlers
- Console
- Console log handlers write log messages to either the host operating system's standard out (
stdout) or standard error (stderr) stream. These messages are displayed when JBoss EAP 6 is run from a command line prompt. The messages from a Console log handler are not saved unless the operating system is configured to capture the standard out or standard error stream. - File
- File log handlers write log messages to a specified file.
- Periodic
- Periodic log handlers write log messages to a named file until a specified period of time has elapsed. Once the time period has passed then the file is renamed by appending the specified timestamp and the handler continues to write into a newly created log file with the original name.
- Size
- Size log handlers write log messages to a named file until the file reaches a specified size. When the file reaches a specified size, it is renamed with a numeric suffix and the handler continues to write into a newly created log file with the original name. Each size log handler must specify the maximum number of files to be kept in this fashion.
- Periodic Size
- Available from JBoss EAP 6.4. This is a combination of the Periodic and Size handlers and supports their combined attributes.Once the current log file reaches the specified size, or the specified time period has passed, the file is renamed and the handler continues to write to a newly created log file with the original name.
- Async
- Async log handlers are wrapper log handlers that provide asynchronous behavior for one or more other log handlers. These are useful for log handlers that may have high latency or other performance problems such as writing a log file to a network file system.
- Custom
- Custom log handlers enable to you to configure new types of log handlers that have been implemented. A custom handler must be implemented as a Java class that extends
java.util.logging.Handlerand be contained in a module. - syslog
- Syslog handlers can be used to send messages to a remote logging server. This allows multiple applications to send their log messages to the same server, where they can all be parsed together.
12.1.15. About Log Formatters
A log formatter is the configuration property of a log handler that defines the appearance of log messages from that handler. It is a string that uses a syntax based on
java.util.Formatter class.
For example the log formatter string from the default configuration,
%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%E%n , creates log messages that look like:
15:53:26,546 INFO [org.jboss.as] (Controller Boot Thread) JBAS015951: Admin console listening on http://127.0.0.1:9990
12.1.16. Log Formatter Syntax
| Symbol | Description |
|---|---|
%c | The category of the logging event |
%p | The level of the log entry (info/debug/etc) |
%P | The localized level of the log entry |
%d | The current date/time (yyyy-MM-dd HH:mm:ss,SSS form) |
%r | The relative time (milliseconds since the log was initialized) |
%z | The time zone |
%k | A log resource key (used for localization of log messages) |
%m | The log message (including exception trace) |
%s | The simple log message (no exception trace) |
%e | The exception stack trace (no extended module information) |
%E | The exception stack trace (with extended module information) |
%t | The name of the current thread |
%n | A newline character |
%C | The class of the code calling the log method (slow) |
%F | The filename of the class calling the log method (slow) |
%l | The source location of the code calling the log method (slow) |
%L | The line number of the code calling the log method (slow) |
%M | The method of the code calling the log method (slow) |
%x | The Nested Diagnostic Context |
%X | The Message Diagnostic Context |
%% | A literal percent character (escaping) |
