Splunk Search

Transaction not grouping subsequent filter strings

Naren26
Path Finder

I am trying to find the occurrence whenever the state changes due to the error. Below are my sample events:

2021/08/01 07:12:12.098 host=12345 In
2021/08/01 07:13:12.098 host=12345 In
2021/08/01 07:14:12.098 host=12345 Out
2021/08/01 07:15:12.098 host=12345 Out
2021/08/01 07:16:12.098 host=12345 In
2021/08/01 07:17:12.098 host=12345 In
2021/08/01 07:18:12.098 host=12345 Out
2021/08/01 07:18:35.098 host=12345 ERROR
2021/08/01 07:19:12.098 host=12345 In
2021/08/01 07:20:12.098 host=12345 Out

I need to group the events when the state (In/Out) changed due an ERROR event. For the above sample events, I should not get any result. Because, when the ERROR event happened, the host is already in "Out" stage. We need to monitor only when a "In" host changes to "Out" due to an ERROR.

I tried the below search

 

index=myindex ("Cut-In" OR "Cut-Out" OR "ERROR")
| rex "host=(?<host>\d+) (?<State>.*)"
| transaction host startswith="State=In" endswith="Out" maxspan=24h
| where searchmatch("ERROR")
| table _time host

 

But the above query returns a result by grouping the "In" state which logged at "07:16:12" as start of the transaction and "07:20:12" as end of the transaction. This is not a valid scenario.

Please help me in framing the logic.

Labels (2)
0 Karma

Naren26
Path Finder

This is not working too. I am still getting the event which should not be picked

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Given that the runanywhere example works, how does your real data differ from the example?

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Try something like this - note that 07:13 - 07:14 appears as a transaction and 07:12 - 07:15 is an outer transaction which may or may not be what you want, but given this is dummy data, this may just be a quirk of the example you made up.

| makeresults 
| eval _raw="2021/08/01 07:12:12.098 host=12345 In
2021/08/01 07:13:12.098 host=12345 In
2021/08/01 07:14:12.098 host=12345 Out
2021/08/01 07:15:12.098 host=12345 Out
2021/08/01 07:16:12.098 host=12345 In
2021/08/01 07:17:12.098 host=12345 In
2021/08/01 07:18:12.098 host=12345 Out
2021/08/01 07:18:35.098 host=12345 ERROR
2021/08/01 07:19:12.098 host=12345 In
2021/08/01 07:20:12.098 host=12345 Out"
| multikv noheader=t 
| table _raw



| rex "(?<datetime>.*)\shost=(?<host>\d+)\s(?<State>.*)"
| eval _time=strptime(datetime,"%Y/%m/%d %H:%M:%S.%Q")
| sort 0 - _time
| transaction host startswith="State=In" endswith="Out" maxspan=24h
0 Karma
Get Updates on the Splunk Community!

3 Ways to Make OpenTelemetry Even Better

My role as an Observability Specialist at Splunk provides me with the opportunity to work with customers of ...

What's New in Splunk Cloud Platform 9.2.2406?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2406 with many ...

Enterprise Security Content Update (ESCU) | New Releases

In August, the Splunk Threat Research Team had 3 releases of new security content via the Enterprise Security ...