Chapter 12. Exported fields

@timestamp

The UTC value marking when the log payload was created, or when the log payload was first collected if the creation time is not known. This is the log processing pipeline’s best effort determination of when the log payload was generated. Add the @ prefix convention to note a field as being reserved for a particular use. With Elasticsearch, most tools look for @timestamp by default. For example, the format would be 2015-01-24 14:06:05.071000.

geoip

This is geo-ip of the machine.

hostname

The hostname is the fully qualified domain name (FQDN) of the entity generating the original payload. This field is an attempt to derive this context. Sometimes the entity generating it knows the context. While other times that entity has a restricted namespace itself, which is known by the collector or normalizer.

ipaddr4

The IP address V4 of the source server, which can be an array.

ipaddr6

The IP address V6 of the source server, if available.

level

The logging level as provided by rsyslog (severitytext property), python’s logging module. Possible values are as listed at misc/sys/syslog.h plus trace and unknown. For example, alert crit debug emerg err info notice trace unknown warning. Note that trace is not in the syslog.h list but many applications use it.

* You should only use unknown when the logging system gets a value it does not understand, and note that it is the highest level.

* Consider trace as higher or more verbose, than debug.

* error is deprecated, use err.

* Convert panic to emerg.

* Convert warn to warning.

Numeric values from syslog/journal PRIORITY can usually be mapped using the priority values as listed at misc/sys/syslog.h.

Log levels and priorities from other logging systems should be mapped to the nearest match. See python logging for an example.

message

A typical log entry message, or payload. It can be stripped of metadata pulled out of it by the collector or normalizer, that is UTF-8 encoded.

pid

This is the process ID of the logging entity, if available.

service

The name of the service associated with the logging entity, if available. For example, the syslog APP-NAME and rsyslog programname property are mapped to the service field.

tags

Optionally provided operator defined list of tags placed on each log by the collector or normalizer. The payload can be a string with whitespace-delimited string tokens, or a JSON list of string tokens.

file

Optional path to the file containing the log entry local to the collector TODO analyzer for file paths.

offset

The offset value can represent bytes to the start of the log line in the file (zero or one based), or log line numbers (zero or one based), as long as the values are strictly monotonically increasing in the context of a single log file. The values are allowed to wrap, representing a new version of the log file (rotation).

namespace_name

Associate this record with the namespace that shares it’s name. This value will not be stored, but it is used to associate the record with the appropriate namespace for access control and visualization. Normally this value will be given in the tag, but if the protocol does not support sending a tag, this field can be used. If this field is present, it will override the namespace given in the tag or in kubernetes.namespace_name.

namespace_uuid

This is the uuid associated with the namespace_name. This value will not be stored, but is used to associate the record with the appropriate namespace for access control and visualization. If this field is present, it will override the uuid given in kubernetes.namespace_uuid. This will also cause the Kubernetes metadata lookup to be skipped for this log record.

collectd.interval

type: float

The collectd interval.

collectd.plugin

type: string

The collectd plug-in.

collectd.plugin_instance

type: string

The collectd plugin_instance.

collectd.type_instance

type: string

The collectd type_instance.

collectd.type

type: string

The collectd type.

collectd.dstypes

type: string

The collectd dstypes.

collectd.processes.ps_state

type: integer The collectd ps_state type of processes plug-in.

collectd.processes.ps_disk_ops.read

type: float

TODO

collectd.processes.ps_disk_ops.write

type: float

TODO

collectd.processes.ps_vm

type: integer

The collectd ps_vm type of processes plug-in.

collectd.processes.ps_rss

type: integer

The collectd ps_rss type of processes plug-in.

collectd.processes.ps_data

type: integer

The collectd ps_data type of processes plug-in.

collectd.processes.ps_code

type: integer

The collectd ps_code type of processes plug-in.

collectd.processes.ps_stacksize

type: integer

The collectd ps_stacksize type of processes plug-in.

collectd.processes.ps_cputime.user

type: float

TODO

collectd.processes.ps_cputime.syst

type: float

TODO

collectd.processes.ps_count.processes

type: integer

TODO

collectd.processes.ps_count.threads

type: integer

TODO

collectd.processes.ps_pagefaults.majflt

type: float

TODO

collectd.processes.ps_pagefaults.minflt

type: float

TODO

collectd.processes.ps_disk_octets.read

type: float

TODO

collectd.processes.ps_disk_octets.write

type: float

TODO

collectd.processes.fork_rate

type: float

The collectd fork_rate type of processes plug-in.

collectd.disk.disk_merged.read

type: float

TODO

collectd.disk.disk_merged.write

type: float

TODO

collectd.disk.disk_octets.read

type: float

TODO

collectd.disk.disk_octets.write

type: float

TODO

collectd.disk.disk_time.read

type: float

TODO

collectd.disk.disk_time.write

type: float

TODO

collectd.disk.disk_ops.read

type: float

TODO

collectd.disk.disk_ops.write

type: float

TODO

collectd.disk.pending_operations

type: integer

The collectd pending_operations type of disk plug-in.

collectd.disk.disk_io_time.io_time

type: float

TODO

collectd.disk.disk_io_time.weighted_io_time

type: float

TODO

collectd.interface.if_octets.rx

type: float

TODO

collectd.interface.if_octets.tx

type: float

TODO

collectd.interface.if_packets.rx

type: float

TODO

collectd.interface.if_packets.tx

type: float

TODO

collectd.interface.if_errors.rx

type: float

TODO

collectd.interface.if_errors.tx

type: float

TODO

collectd.interface.if_dropped.rx

type: float

TODO

collectd.interface.if_dropped.tx

type: float

TODO

collectd.virt.if_octets.rx

type: float

TODO

collectd.virt.if_octets.tx

type: float

TODO

collectd.virt.if_packets.rx

type: float

TODO

collectd.virt.if_packets.tx

type: float

TODO

collectd.virt.if_errors.rx

type: float

TODO

collectd.virt.if_errors.tx

type: float

TODO

collectd.virt.if_dropped.rx

type: float

TODO

collectd.virt.if_dropped.tx

type: float

TODO

collectd.virt.disk_ops.read

type: float

TODO

collectd.virt.disk_ops.write

type: float

TODO

collectd.virt.disk_octets.read

type: float

TODO

collectd.virt.disk_octets.write

type: float

TODO

collectd.virt.memory

type: float

The collectd memory type of virt plug-in.

collectd.virt.virt_vcpu

type: float

The collectd virt_vcpu type of virt plug-in.

collectd.virt.virt_cpu_total

type: float

The collectd virt_cpu_total type of virt plug-in.

collectd.CPU.percent

type: float

The collectd type percent of plug-in CPU.

collectd.df.df_complex

type: float

The collectd type df_complex of plug-in df.

collectd.df.percent_bytes

type: float

The collectd type percent_bytes of plug-in df.

collectd.entropy.entropy

type: integer

The collectd entropy type of entropy plug-in.

collectd.memory.memory

type: float

The collectd memory type of memory plug-in.

collectd.memory.percent

type: float

The collectd percent type of memory plug-in.

collectd.swap.swap

type: integer

The collectd swap type of swap plug-in.

collectd.swap.swap_io

type: integer

The collectd swap_io type of swap plug-in.

collectd.load.load.shortterm

type: float

TODO

collectd.load.load.midterm

type: float

TODO

collectd.load.load.longterm

type: float

TODO

collectd.aggregation.percent

type: float

TODO

collectd.statsd.host_cpu

type: integer

The collectd CPU type of statsd plug-in.

collectd.statsd.host_elapsed_time

type: integer

The collectd elapsed_time type of statsd plug-in.

collectd.statsd.host_memory

type: integer

The collectd memory type of statsd plug-in.

collectd.statsd.host_nic_speed

type: integer

The collectd nic_speed type of statsd plug-in.

collectd.statsd.host_nic_rx

type: integer

The collectd nic_rx type of statsd plug-in.

collectd.statsd.host_nic_tx

type: integer

The collectd nic_tx type of statsd plug-in.

collectd.statsd.host_nic_rx_dropped

type: integer

The collectd nic_rx_dropped type of statsd plug-in.

collectd.statsd.host_nic_tx_dropped

type: integer

The collectd nic_tx_dropped type of statsd plug-in.

collectd.statsd.host_nic_rx_errors

type: integer

The collectd nic_rx_errors type of statsd plug-in.

collectd.statsd.host_nic_tx_errors

type: integer

The collectd nic_tx_errors type of statsd plug-in.

collectd.statsd.host_storage

type: integer

The collectd storage type of statsd plug-in.

collectd.statsd.host_swap

type: integer

The collectd swap type of statsd plug-in.

collectd.statsd.host_vdsm

type: integer

The collectd VDSM type of statsd plug-in.

collectd.statsd.host_vms

type: integer

The collectd VMS type of statsd plug-in.

collectd.statsd.vm_nic_tx_dropped

type: integer

The collectd nic_tx_dropped type of statsd plug-in.

collectd.statsd.vm_nic_rx_bytes

type: integer

The collectd nic_rx_bytes type of statsd plug-in.

collectd.statsd.vm_nic_tx_bytes

type: integer

The collectd nic_tx_bytes type of statsd plug-in.

collectd.statsd.vm_balloon_min

type: integer

The collectd balloon_min type of statsd plug-in.

collectd.statsd.vm_balloon_max

type: integer

The collectd balloon_max type of statsd plug-in.

collectd.statsd.vm_balloon_target

type: integer

The collectd balloon_target type of statsd plug-in.

collectd.statsd.vm_balloon_cur

type: integer

The collectd balloon_cur type of statsd plug-in.

collectd.statsd.vm_cpu_sys

type: integer

The collectd cpu_sys type of statsd plug-in.

collectd.statsd.vm_cpu_usage

type: integer

The collectd cpu_usage type of statsd plug-in.

collectd.statsd.vm_disk_read_ops

type: integer

The collectd disk_read_ops type of statsd plug-in.

collectd.statsd.vm_disk_write_ops

type: integer

The collectd` disk_write_ops type of statsd plug-in.

collectd.statsd.vm_disk_flush_latency

type: integer

The collectd disk_flush_latency type of statsd plug-in.

collectd.statsd.vm_disk_apparent_size

type: integer

The collectd disk_apparent_size type of statsd plug-in.

collectd.statsd.vm_disk_write_bytes

type: integer

The collectd disk_write_bytes type of statsd plug-in.

collectd.statsd.vm_disk_write_rate

type: integer

The collectd disk_write_rate type of statsd plug-in.

collectd.statsd.vm_disk_true_size

type: integer

The collectd disk_true_size type of statsd plug-in.

collectd.statsd.vm_disk_read_rate

type: integer

The collectd disk_read_rate type of statsd plug-in.

collectd.statsd.vm_disk_write_latency

type: integer

The collectd disk_write_latency type of statsd plug-in.

collectd.statsd.vm_disk_read_latency

type: integer

The collectd disk_read_latency type of statsd plug-in.

collectd.statsd.vm_disk_read_bytes

type: integer

The collectd disk_read_bytes type of statsd plug-in.

collectd.statsd.vm_nic_rx_dropped

type: integer

The collectd nic_rx_dropped type of statsd plug-in.

collectd.statsd.vm_cpu_user

type: integer

The collectd cpu_user type of statsd plug-in.

collectd.statsd.vm_nic_rx_errors

type: integer

The collectd nic_rx_errors type of statsd plug-in.

collectd.statsd.vm_nic_tx_errors

type: integer

The collectd nic_tx_errors type of statsd plug-in.

collectd.statsd.vm_nic_speed

type: integer

The collectd nic_speed type of statsd plug-in.

collectd.postgresql.pg_n_tup_g

type: integer

The collectd type pg_n_tup_g of plug-in postgresql.

collectd.postgresql.pg_n_tup_c

type: integer

The collectd type pg_n_tup_c of plug-in postgresql.

collectd.postgresql.pg_numbackends

type: integer

The collectd type pg_numbackends of plug-in postgresql.

collectd.postgresql.pg_xact

type: integer

The collectd type pg_xact of plug-in postgresql.

collectd.postgresql.pg_db_size

type: integer

The collectd type pg_db_size of plug-in postgresql.

collectd.postgresql.pg_blks

type: integer

The collectd type pg_blks of plug-in postgresql.

rsyslog.facility

See syslog specification for more information on rsyslog.

rsyslog.protocol-version

This is the rsyslog protocol version.

rsyslog.structured-data

See syslog specification for more information on syslog structured-data.

rsyslog.msgid

This is the syslog msgid field.

rsyslog.appname

If app-name is the same as programname, then only fill top-level field service. If app-name is not equal to programname, this field will hold app-name. See syslog specifications for more information.

systemd.k.KERNEL_DEVICE

systemd.k.KERNEL_DEVICE is the kernel device name.

systemd.k.KERNEL_SUBSYSTEM

systemd.k.KERNEL_SUBSYSTEM is the kernel subsystem name.

systemd.k.UDEV_DEVLINK

systemd.k.UDEV_DEVLINK includes additional symlink names that point to the node.

systemd.k.UDEV_DEVNODE

systemd.k.UDEV_DEVNODE is the node path of the device.

systemd.k.UDEV_SYSNAME

systemd.k.UDEV_SYSNAME is the kernel device name.

systemd.t.AUDIT_LOGINUID

systemd.t.AUDIT_LOGINUID is the user ID for the journal entry process.

systemd.t.BOOT_ID

systemd.t.BOOT_ID is the kernel boot ID.

systemd.t.AUDIT_SESSION

systemd.t.AUDIT_SESSION is the session for the journal entry process.

systemd.t.CAP_EFFECTIVE

systemd.t.CAP_EFFECTIVE represents the capabilities of the journal entry process.

systemd.t.CMDLINE

systemd.t.CMDLINE is the command line of the journal entry process.

systemd.t.COMM

systemd.t.COMM is the name of the journal entry process.

systemd.t.EXE

systemd.t.EXE is the executable path of the journal entry process.

systemd.t.GID

systemd.t.GID is the group ID for the journal entry process.

systemd.t.HOSTNAME

systemd.t.HOSTNAME is the name of the host.

systemd.t.MACHINE_ID

systemd.t.MACHINE_ID is the machine ID of the host.

systemd.t.PID

systemd.t.PID is the process ID for the journal entry process.

systemd.t.SELINUX_CONTEXT

systemd.t.SELINUX_CONTEXT is the security context, or label, for the journal entry process.

systemd.t.SOURCE_REALTIME_TIMESTAMP

systemd.t.SOURCE_REALTIME_TIMESTAMP is the earliest and most reliable timestamp of the message. This is converted to RFC 3339 NS format.

systemd.t.SYSTEMD_CGROUP

systemd.t.SYSTEMD_CGROUP is the systemd control group path.

systemd.t.SYSTEMD_OWNER_UID

systemd.t.SYSTEMD_OWNER_UID is the owner ID of the session.

systemd.t.SYSTEMD_SESSION

systemd.t.SYSTEMD_SESSION, if applicable, is the systemd session ID.

systemd.t.SYSTEMD_SLICE

systemd.t.SYSTEMD_SLICE is the slice unit of the journal entry process.

systemd.t.SYSTEMD_UNIT

systemd.t.SYSTEMD_UNIT is the unit name for a session.

systemd.t.SYSTEMD_USER_UNIT

systemd.t.SYSTEMD_USER_UNIT, if applicable, is the user unit name for a session.

systemd.t.TRANSPORT

systemd.t.TRANSPORT is the method of entry by the journal service. This includes, audit, driver, syslog, journal, stdout, and kernel.

systemd.t.UID

systemd.t.UID is the user ID for the journal entry process.

systemd.t.SYSLOG_FACILITY

systemd.t.SYSLOG_FACILITY is the field containing the facility, formatted as a decimal string, for syslog.

systemd.t.SYSLOG_IDENTIFIER

systemd.t.systemd.t.SYSLOG_IDENTIFIER is the identifier for syslog.

systemd.t.SYSLOG_PID

SYSLOG_PID is the client process ID for syslog.

systemd.u.CODE_FILE

systemd.u.CODE_FILE is the code location containing the filename of the source.

systemd.u.CODE_FUNCTION

systemd.u.CODE_FUNCTION is the code location containing the function of the source.

systemd.u.CODE_LINE

systemd.u.CODE_LINE is the code location containing the line number of the source.

systemd.u.ERRNO

systemd.u.ERRNO, if present, is the low-level error number formatted in numeric value, as a decimal string.

systemd.u.MESSAGE_ID

systemd.u.MESSAGE_ID is the message identifier ID for recognizing message types.

systemd.u.RESULT

For private use only.

systemd.u.UNIT

For private use only.

kubernetes.pod_id

Kubernetes ID of the pod.

kubernetes.namespace_name

The name of the namespace in Kubernetes.

kubernetes.namespace_id

ID of the namespace in Kubernetes.

kubernetes.host

Kubernetes node name.

kubernetes.container_name

The name of the container in Kubernetes.

kubernetes.labels.deployment

The deployment associated with the Kubernetes object.

kubernetes.labels.deploymentconfig

The deploymentconfig associated with the Kubernetes object.

kubernetes.labels.component

The component associated with the Kubernetes object.

kubernetes.labels.provider

The provider associated with the Kubernetes object.

pipeline_metadata.collector.hostname

FQDN of the collector. It might be different from the FQDN of the actual emitter of the logs.

pipeline_metadata.collector.name

Name of the collector.

pipeline_metadata.collector.version

Version of the collector.

pipeline_metadata.collector.ipaddr4

IP address v4 of the collector server, can be an array.

pipeline_metadata.collector.ipaddr6

IP address v6 of the collector server, can be an array.

pipeline_metadata.collector.inputname

How the log message was received by the collector whether it was TCP/UDP, or imjournal/imfile.

pipeline_metadata.collector.received_at

Time when the message was received by the collector.

pipeline_metadata.collector.original_raw_message

The original non-parsed log message, collected by the collector or as close to the source as possible.

pipeline_metadata.normalizer.hostname

FQDN of the normalizer.

pipeline_metadata.normalizer.name

Name of the normalizer.

pipeline_metadata.normalizer.version

Version of the normalizer.

pipeline_metadata.normalizer.ipaddr4

IP address v4 of the normalizer server, can be an array.

pipeline_metadata.normalizer.ipaddr6

IP address v6 of the normalizer server, can be an array.

pipeline_metadata.normalizer.inputname

how the log message was received by the normalizer whether it was TCP/UDP.

pipeline_metadata.normalizer.received_at

Time when the message was received by the normalizer.

pipeline_metadata.normalizer.original_raw_message

The original non-parsed log message as it is received by the normalizer.

pipeline_metadata.trace

The field records the trace of the message. Each collector and normalizer appends information about itself and the date and time when the message was processed.

ovirt.entity

The type of the data source, hosts, VMS, and engine.

ovirt.host_id

The oVirt host UUID.

aushape.serial

Audit event serial number.

aushape.node

Name of the host where the audit event occurred.

aushape.error

The error aushape encountered while converting the event.

aushape.trimmed

An array of JSONPath expressions relative to the event object, specifying objects or arrays with the content removed as the result of event size limiting. An empty string means the event removed the content, and an empty array means the trimming occurred by unspecified objects and arrays.

aushape.text

An array log record strings representing the original audit event.

aushape.data.avc

type: nested

aushape.data.execve

type: string

aushape.data.netfilter_cfg

type: nested

aushape.data.obj_pid

type: nested

aushape.data.path

type: nested

tlog.ver

Message format version number.

tlog.user

Recorded user name.

tlog.term

Terminal type name.

tlog.session

Audit session ID of the recorded session.

tlog.id

ID of the message within the session.

tlog.pos

Message position in the session, milliseconds.

tlog.timing

Distribution of this message’s events in time.

tlog.in_txt

Input text with invalid characters scrubbed.

tlog.in_bin

Scrubbed invalid input characters as bytes.

tlog.out_txt

Output text with invalid characters scrubbed.

tlog.out_bin

Scrubbed invalid output characters as bytes.