Splunk Search

How to edit my search to detect a significant change in events frequency over a sliding window?

pm771
Communicator

Based on an event log we would like to find event type which frequency changed by 50% or more over a 5 min window.

Using my limited Splunk knowledge and heavy Googling, I came up with something like this:

index=index_of_events | eval cnt=1 | timechart span=20s limit=40 per_second(cnt) as ev  by ev_typeuseother=f usenull=f |
streamstats window=40 global=false first(ev) as start last(ev) as end by ev_type | 
eval diff=abs(start-end) | eval max_val=max(start, end) | 
where diff > 0 AND max > 0 | eval prc=100*diff/max_val | where prc > 50

I'm getting reasonable output after streamstats, but then I'm losing the data.

Was it OK to pipe timechart directly into streamstats? Did I need untable (or something) in between?

How do I get it right?

0 Karma

sundareshr
Legend

See if this gets you what you're looking for

index=index_of_events | bin span=5m _time | stats count by _time evtype | streamstats window=1 current=f global=f first(count) as start by evtype | eval diff=(abs(start-count)/start)*100 | where diff>50
0 Karma

pm771
Communicator

Unfortunately I do not believe it solves my problem, as it compares adjacent number of events, while I need to compare frequencies. In physics terms, I need to measure drops of speed and not distance. That's why I used per_second().

Have I missed something in your solution?

What is wrong with my query? Can it be somehow salvaged?

0 Karma
Get Updates on the Splunk Community!

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...