Alerting

Help with query, transactions and percentage. Need it for Alerting.

Rialf1959
Explorer

Hello, I need help with this query. Cpu_percent field return values in percentage, so it might be a problem.
Basically i need to find all containers where cpu_percent usage is above 50% during 5mins and print some fields. I need it for realtime alerting. I am using ta-dockerstats addon.
This is my first query 😞

My example:

index=main source="docker_stats" host=* |stats last(cpu_percent) as CPU by container_id |where CPU> 30 | transaction host, container_id, CPU > 30 maxpause=5m | where duration>300 | fields _time, duration, host, CPU, container_id

Thanks

Tags (1)
0 Karma

woodcock
Esteemed Legend

DO NOT use realtime for this. Not only will it not work properly, but it kills your performance.

0 Karma

cmerriman
Super Champion

Alright, so from what i can see, it looks like you're filtering out the cpu_percent that's greater than 30% by container_id and then running a transaction where the events are 5 minutes apart and also only last 5 minutes.

Is this an alert running every 5 minutes? if so, try something like this:

index=main source="docker_stats" host=* earliest=-5m|stats last(cpu_percent) as CPU by container_id |search CPU>50| fields _time, host, CPU, container_id

if you need it ran for a longer duration of time:

index=main source="docker_stats" host=*|eventstats last(cpu_percent) as CPU by container_id| transaction host, container_id maxspan=5m maxpause=5m|search CPU>50 | fields _time, duration, host, CPU, container_id
0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

.conf25 Global Broadcast: Don’t Miss a Moment

Hello Splunkers, .conf25 is only a click away.  Not able to make it to .conf25 in person? No worries, you can ...

Observe and Secure All Apps with Splunk

 Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

What's New in Splunk Observability - August 2025

What's New We are excited to announce the latest enhancements to Splunk Observability Cloud as well as what is ...