Splunk Search

A value remains for a length of time

troyhochsprung
New Member

I am looking for a search that returns an events(s) when the searched value remains for a set length of time. Using Windows performance monitoring searches I am searching for CPU "Peak" values but would like to add a time condition for length.

Example: 

eventtype="perfmon_windows" (Host="hostname") object="Processor" counter="% Processor Time" instance="*" | stats sparkline(avg(Value)) as Trend avg(Value) as Average, max(Value) as Peak, latest(Value) as Current, latest(_time) as "Last Updated" by Host | convert ctime("Last Updated") | sort - Current | eval Average=round(Average, 2) | eval Peak=round(Peak, 2) | eval Current=round(Current, 2) | where Peak>85 

Right now that search works very well but if I get a quick spike of CPU it will go off. I would like to add something like when the CPU hits that Peak of 85 for something like 5 minutes or more. 

Labels (1)
0 Karma

troyhochsprung
New Member

Spikes can be somewhat common so really would like to see the Peak value that is set hold that level for a set time like 5 minutes to be creating the event

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Change your stats so that you check every minute, then use autoregress to capture the previous 4 values, then Eval if all 5 values are above your threshold

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

You could use autoregress and count the number of times you have had spikes in the last X periods

0 Karma
Get Updates on the Splunk Community!

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...