How to look for two values that occur at the same time per host?

[AssureSec]

Hello all,

I am trying to figure out the following:

1. If an alert for rule_id1 occurs at the same time on the same host as an alert for rule_id2 then don’t report the alert on rule_id2.
2. Otherwise report alerts on rule_id2

I have triend the if(match) and if(like) method and neither are able to yield the results I am hoping for.  Also not sure how to incorporate the time check as well to ensure the fired at the same-ish time.

Any and all help greatly appreciated!

Thanks!

[AssureSec]

So basically ruled_id1 and 2 are two different events.  What we want to do is make sure that if there is a event for rule_id1 and an event for rule_id2 on the same host, at the same time, we don't display those and only display the events where only rule_id2 has an event.  If both rule_id1 and rule_id2 have an event at the same time for the same host, those are false positives.

rule_id1 and rule_id2 are the same field just different values. How to find when they occur at the same-ish time and on the same host/user and then only display the unique occurrences of rule_id2 or what is left.

[yuanliu]

Assume that the field with values rule_id1 and 2 is called "rule", and the condition "same-ish" can be implemented with search in 5-minute bins, this should work:

index=myindex rule IN (rule_id1, rule_id2)
| bin span=5m _time
| stats values(rule) as rule by _time host
| where rule == "rule_id2" AND NOT rule == "rule_id1"

The last filter reads a little silly if it is in another language.  But SPL's equality operator returns true when any value in a multivalue if the other value is single valued.  A more semantically explicit expression can be

| where isnotnull(mvfind(rule, "rule_id2")) AND isnull(mvfind(rule, "rule_id1"))

In plain English, the search says: give me data containing values of both rule_id1 and rule_id2 in each 5-minute calendar intervals for each host, then find out which host and interval combinations contain only rule_id2 and not rule_id1.  A key test of suitability for this solution will be whether x-calendar interval is a good enough approximation of "same-ish". (What I am getting at is that a calendar interval is not a rolling time window.)

[yuanliu]

Because SPL is a streaming language, you'll have to explain and illustrate raw data (anonymize as needed), illustrate how rule_id1 and rule_id2 relate to such data.

For example, suppose your raw data contains some events with a field rule_id1, some others with a field rule_id2; suppose the first alert fires up when rule_id1 exists, and the second alert fires up when rule_id2 exists plus the simultaneity condition you described.  Further assume that "same-ish time" means test in 5-minute bins.  Then, your second alert can be

| bin span=5m _time
| stats values(rules_id1) values(rules_id2) by _time host
| where isnull('values(rules_id1)') AND isnotnull('values(rules_id2)')

This example shows how a solution is closely related to details of data and individual criteria.