Alerting

Timechart CPU by Process

mxanareckless
Path Finder

I'm interested in creating an alert scheduled to run every 60 minutes, that will search for hosts which have had > 85% CPU load over a span of 5 minutes. Here's the search:

index=index sourcetype=cpu
| streamstats time_window=5min latest(cpu_load_percent) count by host
| eval cpu_load_percent=if(count<18,null,round(cpu_load_percent, 2))
| where cpu_load_percent>85
| dedup host
| table host, _time, cpu_load_percent

From there, I would like a report generated, wherein for each host a timechart is provided for the last 60 minutes, showing CPU %s for each of the processes run on that host. Ideally this will be a line chart, with a line for each of the top 10 CPU-heavy processes. I've tried using | transaction, and this is what I have so far:

index=index sourcetype=cpu AND sourcetype=top host=$host$
| timechart latest(cpu_load_percent) by COMMAND

I'd really appreciate any guidance on how to implement an alert of this type.

Labels (3)
0 Karma
1 Solution

tscroggins
Influencer

@to4kawa 

It should be a field alias in recent versions of Splunk_TA_nix. I just verified 8.2.0 and 8.3.0. It's not present in 5.2.4, which is the only other version I have handy.

[top]
...
FIELDALIAS-cpu_load_percent = pctCPU as cpu_load_percent

View solution in original post

to4kawa
Ultra Champion

I don't know the result of sourcetype=top at all, so it's hard to say.

mxanareckless
Path Finder
0 Karma

to4kawa
Ultra Champion
index=index sourcetype=cpu AND sourcetype=top host=$host$
| timechart latest(cpu_load_percent) by COMMAND

sourcetype=top doesn't have cpu_load_percent.

try coalesce

tscroggins
Influencer

@to4kawa 

It should be a field alias in recent versions of Splunk_TA_nix. I just verified 8.2.0 and 8.3.0. It's not present in 5.2.4, which is the only other version I have handy.

[top]
...
FIELDALIAS-cpu_load_percent = pctCPU as cpu_load_percent

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...