Splunk Search

Transaction not grouping subsequent filter strings

Naren26
Path Finder

I am trying to find the occurrence whenever the state changes due to the error. Below are my sample events:

2021/08/01 07:12:12.098 host=12345 In
2021/08/01 07:13:12.098 host=12345 In
2021/08/01 07:14:12.098 host=12345 Out
2021/08/01 07:15:12.098 host=12345 Out
2021/08/01 07:16:12.098 host=12345 In
2021/08/01 07:17:12.098 host=12345 In
2021/08/01 07:18:12.098 host=12345 Out
2021/08/01 07:18:35.098 host=12345 ERROR
2021/08/01 07:19:12.098 host=12345 In
2021/08/01 07:20:12.098 host=12345 Out

I need to group the events when the state (In/Out) changed due an ERROR event. For the above sample events, I should not get any result. Because, when the ERROR event happened, the host is already in "Out" stage. We need to monitor only when a "In" host changes to "Out" due to an ERROR.

I tried the below search

 

index=myindex ("Cut-In" OR "Cut-Out" OR "ERROR")
| rex "host=(?<host>\d+) (?<State>.*)"
| transaction host startswith="State=In" endswith="Out" maxspan=24h
| where searchmatch("ERROR")
| table _time host

 

But the above query returns a result by grouping the "In" state which logged at "07:16:12" as start of the transaction and "07:20:12" as end of the transaction. This is not a valid scenario.

Please help me in framing the logic.

Labels (2)
0 Karma

Naren26
Path Finder

This is not working too. I am still getting the event which should not be picked

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Given that the runanywhere example works, how does your real data differ from the example?

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Try something like this - note that 07:13 - 07:14 appears as a transaction and 07:12 - 07:15 is an outer transaction which may or may not be what you want, but given this is dummy data, this may just be a quirk of the example you made up.

| makeresults 
| eval _raw="2021/08/01 07:12:12.098 host=12345 In
2021/08/01 07:13:12.098 host=12345 In
2021/08/01 07:14:12.098 host=12345 Out
2021/08/01 07:15:12.098 host=12345 Out
2021/08/01 07:16:12.098 host=12345 In
2021/08/01 07:17:12.098 host=12345 In
2021/08/01 07:18:12.098 host=12345 Out
2021/08/01 07:18:35.098 host=12345 ERROR
2021/08/01 07:19:12.098 host=12345 In
2021/08/01 07:20:12.098 host=12345 Out"
| multikv noheader=t 
| table _raw



| rex "(?<datetime>.*)\shost=(?<host>\d+)\s(?<State>.*)"
| eval _time=strptime(datetime,"%Y/%m/%d %H:%M:%S.%Q")
| sort 0 - _time
| transaction host startswith="State=In" endswith="Out" maxspan=24h
0 Karma
Get Updates on the Splunk Community!

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...