topic Transaction not grouping subsequent filter strings in Splunk Search

Transaction not grouping subsequent filter strings

Naren26 — Mon, 23 Aug 2021 12:01:23 GMT

I am trying to find the occurrence whenever the state changes due to the error. Below are my sample events:

2021/08/01 07:12:12.098 host=12345 In
2021/08/01 07:13:12.098 host=12345 In
2021/08/01 07:14:12.098 host=12345 Out
2021/08/01 07:15:12.098 host=12345 Out
2021/08/01 07:16:12.098 host=12345 In
2021/08/01 07:17:12.098 host=12345 In
2021/08/01 07:18:12.098 host=12345 Out
2021/08/01 07:18:35.098 host=12345 ERROR
2021/08/01 07:19:12.098 host=12345 In
2021/08/01 07:20:12.098 host=12345 Out

I need to group the events when the state (In/Out) changed due an ERROR event. For the above sample events, I should not get any result. Because, when the ERROR event happened, the host is already in "Out" stage. We need to monitor only when a "In" host changes to "Out" due to an ERROR.

I tried the below search

index=myindex ("Cut-In" OR "Cut-Out" OR "ERROR") | rex "host=(?<host>\d+) (?<State>.*)" | transaction host startswith="State=In" endswith="Out" maxspan=24h | where searchmatch("ERROR") | table _time host

But the above query returns a result by grouping the "In" state which logged at "07:16:12" as start of the transaction and "07:20:12" as end of the transaction. This is not a valid scenario.

Please help me in framing the logic.

Re: Transaction not grouping subsequent filter strings

ITWhisperer — Mon, 23 Aug 2021 12:29:30 GMT

Try something like this - note that 07:13 - 07:14 appears as a transaction and 07:12 - 07:15 is an outer transaction which may or may not be what you want, but given this is dummy data, this may just be a quirk of the example you made up.

| makeresults | eval _raw="2021/08/01 07:12:12.098 host=12345 In 2021/08/01 07:13:12.098 host=12345 In 2021/08/01 07:14:12.098 host=12345 Out 2021/08/01 07:15:12.098 host=12345 Out 2021/08/01 07:16:12.098 host=12345 In 2021/08/01 07:17:12.098 host=12345 In 2021/08/01 07:18:12.098 host=12345 Out 2021/08/01 07:18:35.098 host=12345 ERROR 2021/08/01 07:19:12.098 host=12345 In 2021/08/01 07:20:12.098 host=12345 Out" | multikv noheader=t | table _raw | rex "(?<datetime>.*)\shost=(?<host>\d+)\s(?<State>.*)" | eval _time=strptime(datetime,"%Y/%m/%d %H:%M:%S.%Q") | sort 0 - _time | transaction host startswith="State=In" endswith="Out" maxspan=24h

Re: Transaction not grouping subsequent filter strings

Naren26 — Wed, 25 Aug 2021 12:55:48 GMT

This is not working too. I am still getting the event which should not be picked

Re: Transaction not grouping subsequent filter strings

ITWhisperer — Wed, 25 Aug 2021 12:59:12 GMT

Given that the runanywhere example works, how does your real data differ from the example?