17.1.1. Eve JSON Output
The EVE output facility outputs alerts, anomalies, metadata, file info and protocol specific records through JSON.
The most common way to use this is through 'EVE', which is a firehose approach where all these logs go into a single file.
outputs:
# Extensible Event Format (nicknamed EVE) event log in JSON format
- eve-log:
enabled: yes
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
filename: eve.json
# Enable for multi-threaded eve.json output; output files are amended with
# an identifier, e.g., eve.9.json
#threaded: false
#prefix: "@cee: " # prefix to prepend to each log entry
# the following are valid when type: syslog above
#identity: "suricata"
#facility: local5
#level: Info ## possible levels: Emergency, Alert, Critical,
## Error, Warning, Notice, Info, Debug
#ethernet: no # log ethernet header in events when available
#redis:
# server: 127.0.0.1
# port: 6379
# async: true ## if redis replies are read asynchronously
# mode: list ## possible values: list|lpush (default), rpush, channel|publish, xadd|stream
# ## lpush and rpush are using a Redis list. "list" is an alias for lpush
# ## publish is using a Redis channel. "channel" is an alias for publish
# ## xadd is using a Redis stream. "stream" is an alias for xadd
# key: suricata ## string denoting the key/channel/stream to use (default to suricata)
# stream-maxlen: 100000 ## Automatically trims the stream length to at most
## this number of events. Set to 0 to disable trimming.
## Only used when mode is set to xadd/stream.
# stream-trim-exact: false ## Trim exactly to the maximum stream length above.
## Default: use inexact trimming (inexact by a few
## tens of items)
## Only used when mode is set to xadd/stream.
# Redis pipelining set up. This will enable to only do a query every
# 'batch-size' events. This should lower the latency induced by network
# connection at the cost of some memory. There is no flushing implemented
# so this setting should be reserved to high traffic Suricata deployments.
# pipelining:
# enabled: yes ## set enable to yes to enable query pipelining
# batch-size: 10 ## number of entries to keep in buffer
# Include top level metadata. Default yes.
#metadata: no
# include the name of the input pcap file in pcap file processing mode
pcap-file: false
# Community Flow ID
# Adds a 'community-id' field to EVE records. These are meant to give
# records a predictable flow ID that can be used to match records to
# output of other tools such as Zeek (Bro).
#
# Takes a 'seed' that needs to be same across sensors and tools
# to make the id less predictable.
# enable/disable the community id feature.
community-id: false
# Seed value for the ID output. Valid values are 0-65535.
community-id-seed: 0
# HTTP X-Forwarded-For support by adding an extra field or overwriting
# the source or destination IP address (depending on flow direction)
# with the one reported in the X-Forwarded-For HTTP header. This is
# helpful when reviewing alerts for traffic that is being reverse
# or forward proxied.
xff:
enabled: no
# Two operation modes are available: "extra-data" and "overwrite".
mode: extra-data
# Two proxy deployments are supported: "reverse" and "forward". In
# a "reverse" deployment the IP address used is the last one, in a
# "forward" deployment the first IP address is used.
deployment: reverse
# Header name where the actual IP address will be reported. If more
# than one IP address is present, the last IP address will be the
# one taken into consideration.
header: X-Forwarded-For
types:
- alert:
# payload: yes # enable dumping payload in Base64
# payload-buffer-size: 4kb # max size of payload buffer to output in eve-log
# payload-printable: yes # enable dumping payload in printable (lossy) format
# payload-length: yes # enable dumping payload length, including the gaps
# packet: yes # enable dumping of packet (without stream segments)
# metadata: no # enable inclusion of app layer metadata with alert. Default yes
# If you want metadata, use:
# metadata:
# Include the decoded application layer (ie. http, dns)
#app-layer: true
# Log the current state of the flow record.
#flow: true
#rule:
# Log the metadata field from the rule in a structured
# format.
#metadata: true
# Log the raw rule text.
#raw: false
#reference: false # include reference information from the rule
# http-body: yes # Requires metadata; enable dumping of HTTP body in Base64
# http-body-printable: yes # Requires metadata; enable dumping of HTTP body in printable format
# websocket-payload: yes # Requires metadata; enable dumping of WebSocket Payload in Base64
# websocket-payload-printable: yes # Requires metadata; enable dumping of WebSocket Payload in printable format
# Enable the logging of tagged packets for rules using the
# "tag" keyword.
tagged-packets: yes
# Enable logging the final action taken on a packet by the engine
# (e.g: the alert may have action 'allowed' but the verdict be
# 'drop' due to another alert. That's the engine's verdict)
# verdict: yes
# app layer frames
- frame:
# disabled by default as this is very verbose.
enabled: no
# payload-buffer-size: 4kb # max size of frame payload buffer to output in eve-log
- anomaly:
# Anomaly log records describe unexpected conditions such
# as truncated packets, packets with invalid IP/UDP/TCP
# length values, and other events that render the packet
# invalid for further processing or describe unexpected
# behavior on an established stream. Networks which
# experience high occurrences of anomalies may experience
# packet processing degradation.
#
# Anomalies are reported for the following:
# 1. Decode: Values and conditions that are detected while
# decoding individual packets. This includes invalid or
# unexpected values for low-level protocol lengths as well
# as stream related events (TCP 3-way handshake issues,
# unexpected sequence number, etc).
# 2. Stream: This includes stream related events (TCP
# 3-way handshake issues, unexpected sequence number,
# etc).
# 3. Application layer: These denote application layer
# specific conditions that are unexpected, invalid or are
# unexpected given the application monitoring state.
#
# By default, anomaly logging is enabled. When anomaly
# logging is enabled, applayer anomaly reporting is
# also enabled.
enabled: yes
#
# Choose one or more types of anomaly logging and whether to enable
# logging of the packet header for packet anomalies.
types:
# decode: no
# stream: no
# applayer: yes
#packethdr: no
- http:
extended: yes # enable this for extended logging information
# custom allows additional HTTP fields to be included in eve-log.
# the example below adds three additional fields when uncommented
#custom: [Accept-Encoding, Accept-Language, Authorization]
# set this value to one and only one from {both, request, response}
# to dump all HTTP headers for every HTTP request and/or response
# dump-all-headers: none
- dns:
# This configuration uses the new DNS logging format,
# the old configuration is still available:
# https://docs.suricata.io/en/latest/output/eve/eve-json-output.html#dns-v1-format
# As of Suricata 5.0, version 2 of the eve dns output
# format is the default.
#version: 2
# Enable/disable this logger. Default: enabled.
#enabled: yes
# Control logging of requests and responses:
# - requests: enable logging of DNS queries
# - responses: enable logging of DNS answers
# By default both requests and responses are logged.
#requests: no
#responses: no
# Format of answer logging:
# - detailed: array item per answer
# - grouped: answers aggregated by type
# Default: all
#formats: [detailed, grouped]
# DNS record types to log, based on the query type.
# Default: all.
#types: [a, aaaa, cname, mx, ns, ptr, txt]
- tls:
extended: yes # enable this for extended logging information
# output TLS transaction where the session is resumed using a
# session id
#session-resumption: no
# custom controls which TLS fields that are included in eve-log
# WARNING: enabling custom disables extended logging.
#custom: [subject, issuer, session_resumed, serial, fingerprint, sni, version, not_before, not_after, certificate, chain, ja3, ja3s, ja4, subjectaltname, client, client_certificate, client_chain, client_alpns, server_alpns]
- files:
force-magic: no # force logging magic on all logged files
# force logging of checksums, available hash functions are md5,
# sha1 and sha256
#force-hash: [md5]
#- drop:
# alerts: yes # log alerts that caused drops
# flows: all # start or all: 'start' logs only a single drop
# # per flow direction. All logs each dropped pkt.
# Enable logging the final action taken on a packet by the engine
# (will show more information in case of a drop caused by 'reject')
# verdict: yes
- smtp:
#extended: yes # enable this for extended logging information
# this includes: bcc, message-id, subject, x_mailer, user-agent
# custom fields logging from the list:
# reply-to, bcc, message-id, subject, x-mailer, user-agent, received,
# x-originating-ip, in-reply-to, references, importance, priority,
# sensitivity, organization, content-md5, date
#custom: [received, x-mailer, x-originating-ip, relays, reply-to, bcc]
# output md5 of fields: body, subject
# for the body you need to set app-layer.protocols.smtp.mime.body-md5
# to yes
#md5: [body, subject]
#- dnp3
- websocket
- ftp
- ftp-data
- rdp
- nfs
- smb
- tftp
- ike
- dcerpc
- krb5
- bittorrent-dht
- ssh
- arp:
enabled: no
- snmp
- rfb
- sip
- quic
- dhcp:
enabled: yes
# When extended mode is on, all DHCP messages are logged
# with full detail. When extended mode is off (the
# default), just enough information to map a MAC address
# to an IP address is logged.
extended: no
- mqtt:
# passwords: yes # enable output of passwords
# string-log-limit: 1kb # limit size of logged strings in bytes.
# Can be specified in kb, mb, gb. Just a number
# is parsed as bytes. Default is 1KB.
# Use a value of 0 to disable limiting.
# Note that the size is also bounded by
# the maximum parsed message size (see
# app-layer configuration)
- http2
- pgsql:
enabled: no
# passwords: yes # enable output of passwords. Disabled by default
- stats:
totals: yes # stats for all threads merged together
threads: no # per thread stats
deltas: no # include delta values
# Don't log stats counters that are zero. Default: true
#null-values: false # False will NOT log stats counters: 0
# bi-directional flows
- flow
# uni-directional flows
#- netflow
# Metadata event type. Triggered whenever a pktvar is saved
# and will include the pktvars, flowvars, flowbits and
# flowints.
#- metadata
# EXPERIMENTAL per packet output giving TCP state tracking details
# including internal state, flags, etc.
# This output is experimental, meant for debugging and subject to
# change in both config and output without any notice.
#- stream:
# all: false # log all TCP packets
# event-set: false # log packets that have a decoder/stream event
# state-update: false # log packets triggering a TCP state update
# spurious-retransmission: false # log spurious retransmission packets
Each alert, http log, etc will go into this one file: 'eve.json'. This file can then be processed by 3rd party tools like Logstash (ELK) or jq.
If ethernet
is set to yes, then ethernet headers will be added to events
if available.
17.1.1.1. Output types
EVE can output to multiple methods. regular
is a normal file. Other
options are syslog
, unix_dgram
, unix_stream
and redis
.
Output types:
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
filename: eve.json
# Enable for multi-threaded eve.json output; output files are amended
# with an identifier, e.g., eve.9.json. Default: off
#threaded: off
#prefix: "@cee: " # prefix to prepend to each log entry
# the following are valid when type: syslog above
#identity: "suricata"
#facility: local5
#level: Info ## possible levels: Emergency, Alert, Critical,
## Error, Warning, Notice, Info, Debug
#ethernet: no # log ethernet header in events when available
#redis:
# server: 127.0.0.1
# port: 6379
# async: true ## if redis replies are read asynchronously
# mode: list ## possible values: list|lpush (default), rpush, channel|publish, xadd|stream
# ## lpush and rpush are using a Redis list. "list" is an alias for lpush
# ## publish is using a Redis channel. "channel" is an alias for publish
# ## xadd is using a Redis stream. "stream" is an alias for xadd
# key: suricata ## string denoting the key/channel/stream to use (default to suricata)
# stream-maxlen: 100000 ## Automatically trims the stream length to at most
## this number of events. Set to 0 to disable trimming.
## Only used when mode is set to xadd/stream.
# stream-trim-exact: false ## Trim exactly to the maximum stream length above.
## Default: use inexact trimming (inexact by a few
## tens of items)
## Only used when mode is set to xadd/stream.
# Redis pipelining set up. This will enable to only do a query every
# 'batch-size' events. This should lower the latency induced by network
# connection at the cost of some memory. There is no flushing implemented
# so this setting as to be reserved to high traffic suricata.
# pipelining:
# enabled: yes ## set enable to yes to enable query pipelining
# batch-size: 10 ## number of entry to keep in buffer
17.1.1.2. Alerts
Alerts are event records for rule matches. They can be amended with metadata, such as the application layer record (HTTP, DNS, etc) an alert was generated for, and elements of the rule.
Metadata:
- alert:
#payload: yes # enable dumping payload in Base64
#payload-buffer-size: 4kb # max size of payload buffer to output in eve-log
#payload-printable: yes # enable dumping payload in printable (lossy) format
#payload-length: yes # enable dumping payload length, including the gaps
#packet: yes # enable dumping of packet (without stream segments)
#http-body: yes # Requires metadata; enable dumping of http body in Base64
#http-body-printable: yes # Requires metadata; enable dumping of http body in printable format
# metadata:
# Include the decoded application layer (ie. http, dns)
#app-layer: true
# Log the current state of the flow record.
#flow: true
#rule:
# Log the metadata field from the rule in a structured
# format.
#metadata: true
# Log the raw rule text.
#raw: false
# Include the rule reference information
#reference: false
17.1.1.3. Anomaly
Anomalies are event records created when packets with unexpected or anomalous
values are handled. These events include conditions such as incorrect protocol
values, incorrect protocol length values, and other conditions which render the
packet suspect. Other conditions may occur during the normal progression of a stream;
these are termed stream
events are include control sequences with incorrect
values or that occur out of expected sequence.
Anomalies are reported by and configured by type:
Decode
Stream
Application layer
Metadata:
- anomaly:
# Anomaly log records describe unexpected conditions such as truncated packets,
# packets with invalid IP/UDP/TCP length values, and other events that render
# the packet invalid for further processing or describe unexpected behavior on
# an established stream. Networks which experience high occurrences of
# anomalies may experience packet processing degradation.
#
# Anomalies are reported for the following:
# 1. Decode: Values and conditions that are detected while decoding individual
# packets. This includes invalid or unexpected values for low-level protocol
# lengths as well.
# 2. Stream: This includes stream related events (TCP 3-way handshake issues,
# unexpected sequence number, etc).
# 3. Application layer: These denote application layer specific conditions that
# are unexpected, invalid or are unexpected given the application monitoring
# state.
#
# By default, anomaly logging is disabled. When anomaly logging is enabled,
# application-layer anomaly reporting is enabled.
#
# Choose one or both types of anomaly logging and whether to enable
# logging of the packet header for packet anomalies.
types:
#decode: no
#stream: no
#applayer: yes
#packethdr: no
17.1.1.4. HTTP
HTTP transaction logging.
Config:
- http:
extended: yes # enable this for extended logging information
# custom allows additional http fields to be included in eve-log
# the example below adds three additional fields when uncommented
#custom: [Accept-Encoding, Accept-Language, Authorization]
# set this value to one among {both, request, response} to dump all
# http headers for every http request and/or response
# dump-all-headers: [both, request, response]
List of custom fields:
Yaml Option |
HTTP Header |
---|---|
accept |
accept |
accept_charset |
accept-charset |
accept_encoding |
accept-encoding |
accept_language |
accept-language |
accept_datetime |
accept-datetime |
authorization |
authorization |
cache_control |
cache-control |
cookie |
cookie |
from |
from |
max_forwards |
max-forwards |
origin |
origin |
pragma |
pragma |
proxy_authorization |
proxy-authorization |
range |
range |
te |
te |
via |
via |
x_requested_with |
x-requested-with |
dnt |
dnt |
x_forwarded_proto |
x-forwarded-proto |
x_authenticated_user |
x-authenticated-user |
x_flash_version |
x-flash-version |
accept_range |
accept-range |
age |
age |
allow |
allow |
connection |
connection |
content_encoding |
content-encoding |
content_language |
content-language |
content_length |
content-length |
content_location |
content-location |
content_md5 |
content-md5 |
content_range |
content-range |
content_type |
content-type |
date |
date |
etag |
etags |
expires |
expires |
last_modified |
last-modified |
link |
link |
location |
location |
proxy_authenticate |
proxy-authenticate |
referer |
referer |
refresh |
refresh |
retry_after |
retry-after |
server |
server |
set_cookie |
set-cookie |
trailer |
trailer |
transfer_encoding |
transfer-encoding |
upgrade |
upgrade |
vary |
vary |
warning |
warning |
www_authenticate |
www-authenticate |
true_client_ip |
true-client-ip |
org_src_ip |
org-src-ip |
x_bluecoat_via |
x-bluecoat-via |
In the custom
option values from both columns can be used. The
HTTP Header
column is case insensitive.
17.1.1.5. DNS
Note
As of Suricata 7.0 the v1 EVE DNS format has been removed.
Version 2 EVE DNS will be removed in Suricata 9.
DNS records are logged as one entry for the request, and one entry for the response.
YAML:
- dns:
#version: 3
# Enable/disable this logger. Default: enabled.
#enabled: yes
# Control logging of requests and responses:
# - requests: enable logging of DNS queries
# - responses: enable logging of DNS answers
# By default both requests and responses are logged.
#requests: no
#responses: no
# Format of answer logging:
# - detailed: array item per answer
# - grouped: answers aggregated by type
# Default: all
#formats: [detailed, grouped]
# Types to log, based on the query type.
# Default: all.
#types: [a, aaaa, cname, mx, ns, ptr, txt]
17.1.1.6. TLS
TLS records are logged one record per session.
YAML:
- tls:
extended: yes # enable this for extended logging information
# custom allows to control which tls fields that are included
# in eve-log
#custom: [subject, issuer, serial, fingerprint, sni, version, not_before, not_after, certificate, chain, ja3, ja3s, ja4]
The default is to log certificate subject and issuer. If extended
is
enabled, then the log gets more verbose.
By using custom
it is possible to select which TLS fields to log.
Note that this will disable ``extended`` logging.
17.1.1.7. ARP
ARP records are logged as one entry for the request, and one entry for the response.
YAML:
- arp:
enabled: no
The logger is disabled by default since ARP can generate a large number of events.
17.1.1.8. MQTT
EVE-JSON output for MQTT consists of one object per MQTT transaction, with some common and various type-specific fields. Two aspects can be configured:
YAML:
- mqtt:
# passwords: yes # enable output of passwords
# string-log-limit: 1kb # limit size of logged strings in bytes.
# Can be specified in kb, mb, gb. Just a number
# is parsed as bytes. Default is 1KB.
# Use a value of 0 to disable limiting.
# Note that the size is also bounded by
# the maximum parsed message size (see
# app-layer configuration)
The default is to output passwords in cleartext and not to limit the size of
message payloads. Depending on the kind of context the parser is used in (public
output, frequent binary transmissions, ...) this can be configured for regular
mqtt
events.
17.1.1.9. Drops
Drops are event types logged when the engine drops a packet.
Config:
- drop:
alerts: yes # log alerts that caused drops
flows: all # start or all: 'start' logs only a single drop
# per flow direction. All logs each dropped pkt.
# Enable logging the final action taken on a packet by the engine
# (will show more information in case of a drop caused by 'reject')
verdict: yes
17.1.1.10. Stats
17.1.1.10.1. Zero-valued Counters
While the human-friendly stats.log output will only log out non-zeroed counters, by default EVE Stats logs output all enabled counters, which may lead to fairly verbose logs.
To reduce log file size, one may set null-values to false. Do note that this may impact on the visibility of information for which a stats counter as zero is relevant.
Config:
- stats:
# Don't log stats counters that are zero. Default: true
#null-values: false # False will NOT log stats counters: 0
17.1.1.11. Date modifiers in filename
It is possible to use date modifiers in the eve-log filename.
outputs:
- eve-log:
filename: eve-%s.json
The example above adds epoch time to the filename. All the date modifiers from the
C library should be supported. See the man page for strftime
for all supported
modifiers.
17.1.1.12. Threaded file output
By default, all output is written to the named filename in the outputs section. The threaded
option enables
each output thread to write to individual files. In this case, the filename
will include a unique identifier.
With threaded
enabled, the output will be split among many files -- and
the aggregate of each file's contents must be treated together.
outputs:
- eve-log:
filename: eve.json
threaded: on
This example will cause each Suricata thread to write to its own "eve.json" file. Filenames are constructed
by adding a unique identifier to the filename. For example, eve.7.json
.
17.1.1.13. Rotate log file
Eve-log can be configured to rotate based on time.
outputs:
- eve-log:
filename: eve-%Y-%m-%d-%H:%M.json
rotate-interval: minute
The example above creates a new log file each minute, where the filename contains
a timestamp. Other supported rotate-interval
values are hour
and day
.
In addition to this, it is also possible to specify the rotate-interval
as a
relative value. One example is to rotate the log file each X seconds.
outputs:
- eve-log:
filename: eve-%Y-%m-%d-%H:%M:%S.json
rotate-interval: 30s
The example above rotates eve-log each 30 seconds. This could be replaced with
30m
to rotate every 30 minutes, 30h
to rotate every 30 hours, 30d
to rotate every 30 days, or 30w
to rotate every 30 weeks.
17.1.1.14. Multiple Logger Instances
It is possible to have multiple 'EVE' instances, for example the following is valid:
outputs:
- eve-log:
enabled: yes
type: file
filename: eve-ips.json
types:
- alert
- drop
- eve-log:
enabled: yes
type: file
filename: eve-nsm.json
types:
- http
- dns
- tls
So here the alerts and drops go into 'eve-ips.json', while http, dns and tls go into 'eve-nsm.json'.
With the exception of drop
, you can specify multiples of the same
logger type, however, drop
can only be used once.
Note
The use of independent json loggers such as alert-json-log, dns-json-log, etc. has been deprecated and will be removed by June 2020. Please use multiple eve-log instances as documented above instead. Please see the deprecation policy for more information.
17.1.1.15. File permissions
Log file permissions can be set individually for each logger. filemode
can be used to
control the permissions of a log file, e.g.:
outputs:
- eve-log:
enabled: yes
filename: eve.json
filemode: 600
The example above sets the file permissions on eve.json
to 600, which means that it is
only readable and writable by the owner of the file.
17.1.1.16. JSON flags
Several flags can be specified to control the JSON output in EVE:
outputs:
- eve-log:
json:
# Sort object keys in the same order as they were inserted
preserve-order: yes
# Make the output more compact
compact: yes
# Escape all unicode characters outside the ASCII range
ensure-ascii: yes
# Escape the '/' characters in string with '\/'
escape-slash: yes
All these flags are enabled by default, and can be modified per EVE instance.
17.1.1.17. Community Flow ID
Often Suricata is used in combination with other tools like Bro/Zeek. Enabling
the community-id option in the eve-log section adds a new community_id
field to each output.
Example:
{
"timestamp": "2003-12-16T13:21:44.891921+0000",
"flow_id": 1332028388187153,
"pcap_cnt": 1,
"event_type": "alert",
...
"community_id": "1:LQU9qZlK+B5F3KDmev6m5PMibrg=",
"alert": {
"action": "allowed",
"gid": 1,
"signature_id": 1,
},
}
{
"timestamp": "2003-12-16T13:21:45.037333+0000",
"flow_id": 1332028388187153,
"event_type": "flow",
"flow": {
"pkts_toserver": 5,
"pkts_toclient": 4,
"bytes_toserver": 338,
"bytes_toclient": 272,
"start": "2003-12-16T13:21:44.891921+0000",
"end": "2003-12-16T13:21:45.346457+0000",
"age": 1,
"state": "closed",
"reason": "shutdown",
"alerted": true
},
"community_id": "1:LQU9qZlK+B5F3KDmev6m5PMibrg=",
}
17.1.1.17.1. Options
The output can be enabled per instance of the EVE logger.
The community-id
option is boolean. If set to true
it is enabled.
The community-id-seed
option specifies a unsigned 16 bit value that
is used a seed to the hash that is calculated for the community-id
output. This must be set to the same value on all tools that output this
record.
YAML:
- eve-log:
# Community Flow ID
# Adds a 'community_id' field to EVE records. These are meant to give
# a records a predictable flow id that can be used to match records to
# output of other tools such as Bro.
#
# Takes a 'seed' that needs to be same across sensors and tools
# to make the id less predictable.
# enable/disable the community id feature.
community-id: false
# Seed value for the ID output. Valid values are 0-65535.
community-id-seed: 0
17.1.1.17.1.1. Multi Tenancy
Suricata can be configured to support multiple tenants with different detection
engine configurations. When these tenants are configured and the detection
engine is running then all EVE logging will also report the tenant_id
field
for traffic matching a specific tenant.