8.58. Rule Processing

Suricata rules have many elements that influence how they are processed by Suricata and matched against network traffic.

This section explains some key aspects of how Suricata handles rules internally, so it can be easier to understand/predict how different rules may interact in specific scenarios.

This material is intended for: rule writers; developers.

Possible questions one should be better equipped to answer after reading this document:

  • What happens if two rules have the same priority value (the keyword)?

  • What type of rules will be evaluated first, given a set of rules?

  • How does Suricata decide what rule is "more important" when matching traffic?

Important

Rules processing is also heavily affected by rule types, as mentioned in this chapter. You may want to read more on Rule Types and Categorization.

Note

Throughout this documentation and for Suricata, the terms "rule" and "signature" are mainly interchangeable, unless context indicates otherwise.

8.58.1. Overview

Rules are provided to Suricata via rules files. Starting from those, the Detection Engine loader will:

  1. Load all the signatures;

  2. Check for validity (non-existing keywords; duplicated sid etc);

  3. Report stats on loaded, good and bad signatures;

  4. Sort all valid signatures and store them in a list in the Detect Engine Context (DetectEngineCtx) structure, taking into consideration several rule aspects according to their order of relevance to the Detection Engine;

  5. Attribute internal rule IDs that reflect this rule prioritization;

  6. During inspection, match rules against the inspected traffic, according to rule and traffic type.

8.58.2. Rule Prioritization

Suricata registers several different ordering functions (with SCSigRegisterSignatureOrderingFuncs()), which are then used to compare the rules, sort them, and define their priority. The elements taken into consideration for such are the signature's:

  1. Action

  2. Usage of flowbits

  3. Usage of flowint

  4. Usage of flowvar

  5. Usage of pktvar

  6. Usage of hostbits

  7. Usage of ippair

  8. "Priority" keyword

In this order. Once signatures are ordered, they are attributed a unique internal ID (Signature::iid) which symbolizes their priority (the lower the iid, the higher the priority). This could mean that a rule with a keyword-defined priority of 1 could have lower priority than another rule that had flowbits set and a rule action with higher priority, for instance.

Note

this list isn't fully comprehensive, in the sense that each item has extra logic for prioritization. For example, considering flowbits, the priority is write (highest) > write + read > read (lowest) > no flowbits.

Another important element when considering rule parsing, processing and matching is that the ruleset is optimized into signature group heads based on the signature elements (thus, for instance, a TCP rule and an UDP rule would be loaded into different groups, and their internal ids will not interfere between one another, as they're matched against different traffic). For more on this, see Detection engine.

8.58.3. Inspection Process

Once it is time to inspect network traffic against the loaded rules, the Detect Engine will match against - if applicable:

  1. IP Only rules;

  2. Packet/payload-related rules;

  3. Frame keywords;

  4. Application layer protocol transaction rules.

During packet inspection, if the signature uses the last two in this list, inspection is left to those steps.

Tip

With the introduction of Firewall mode, it is possible to explicitly control to which step of the detection engine flow a rule will be hooked. This is done with Explicit rule hooks.

For each rule that is matched, a PacketAlert is created. After all matches for a packet have been processed, and the alert queue limit is taken into account, the remaining PacketAlerts become the alerts in Suricata logs.

8.58.4. Considerations on Inspection Steps

8.58.4.1. IP Only rules

Without optimization, IP Only signatures would match on every packet on a flow. To improve performance, what Suricata does is to evaluate rules that are ip-only only once per flow direction, for the first packet in each direction.

8.58.4.2. Application layer protocol transactions

Each parser has its own state machine, and uses a per-direction parsing state "progress". Keywords can be registered for each progress value. So, for instance, http has a value "request line available" for which there are keywords like http.uri, http.method etc. registered. While parsing the traffic, if the engine reaches this state, the signatures with those keywords may be already evaluated, even if they have a lower priority than an http body inspecting signature.

Relatedly, a rule with two keywords matching at two different progress stages may be evaluated against two different packets.

8.58.5. Implications

8.58.5.1. Action precedence and interaction with ip-only rules

To illustrate what may be counter-intuitive implications of how inspection steps, action prioritization and rule keywords interact and affect the engine behavior, we will use a real case example. Consider these three rules:

pass tcp 0.0.0.0/0 any <> 0.0.0.0/0 443 (msg:"Allow TCP in port 443"; flow: not_established; sid:1; rev:1;)

pass tcp 0.0.0.0/0 any <> 0.0.0.0/0 80 (msg:"Allow TCP in port 80"; flow: not_established; sid:2; rev:1;)

drop ip 0.0.0.0/0 any -> 0.0.0.0/0 any (msg:"No outbound internet access from host"; sid:3; rev:1;)

The first two are signatures that analyze individual packets and match only if the flow has not been established (flow:not_established): the rules grant PASS to the matched packet - but not to its flow.

The third signature is considered ip-only. This means it will be evaluated for the first packet in both directions of a flow, in addition to rules 1 and 2. By extension, the other packets in the same flow will not be evaluated against this rule.

With an action order configuration that prioritizes PASS over DROP, this means that rules 1 and 2 will have a higher internal priority over rule 3, therefore nullifying the DROP outcome. The result: a flow for outbound internet traffic from the host, expected to be dropped, wouldn't be.

If the expected behavior with those three signatures was to allow traffic on ports 80 and 443 only, while dropping everything else, the simplest way to achieve this would be to remove the flow:not_established portion from rules sid:1 and sid:2. This ensures that the PASS action would be applied to the whole flow following the match on the first packet and that all other traffic would be dropped.

Following that, all three rules will be evaluated on the same step, and if a flow isn't flagged with pass, it will be dropped with the third rule.

Tip

A more straightforward way to achieve that in Suricata 8 is using the firewall more. See Firewall Mode Design.

8.58.5.2. Alerts not seen

Another aspect of rule prioritization combined with the alerts queue size is that, in corner case scenarios, if a packet matches against too many rules, signatures with lower priority could be discarded from the PacketAlert queue (see the section on alert queue overflow impact for more).

The stats counter detect.alert_queue_overflow will be higher than zero if an alert was discarded due to Alert Queue overflow (cf. Discarded and Suppressed Alerts Stats).