All Apps and Add-ons

Splunk App for Unix and Linux: Help creating a dashboard that shows servers using 20% more CPU than previous week

daniel333
Builder

All,

I have 400+ servers with Splunk for Nix installed and collecting metrics to index=os. What I'd like to do is create a dashboard which determines which servers are showing 20% more CPU than they were last week.

That the final result is just a table of servers which have showed 20% increase or more CPU compare the previous week. I really have no idea where to start. Any ideas?

0 Karma

kyaparla
Path Finder

This search can dynamically look for data 7 days back same hour and minute,

|makeresults count=2 | streamstats count |eval count=count-1 | addinfo |eval timediff=(24*3600*7*count) | eval latest=info_max_time-timediff| eval earliest=info_min_time-timediff | map search="search earliest=$earliest$ latest=$latest$ index=os sourcetype=cpu | bin span=5m _time | stats avg(cpu) as cpu by _time,host | eval _time=_time+$timediff$ | eval pw_no=$count$" | eval w{pw_no}=cpu | stats values(w*) as w* by _time,host | eval pct_increase=((w0-w1)/w1)*100 | where pct_increase>20

0 Karma

anthonymelita
Contributor

This maybe accomplishes what you want. Here I'm doing today versus same day last week. Using appendcols to add a stat for the previous time period then doing calculations against the two stats.

index=os sourcetype=cpu earliest=@d latest=now
| multikv fields pctIdle  
| eval Percent_CPU_Load = 100 - pctIdle  
| stats avg(Percent_CPU_Load) as avgLoad by host
| appendcols [search index=os sourcetype=cpu earliest=-8d@d latest=-7d
| multikv fields pctIdle  
| eval Percent_CPU_Load = 100 - pctIdle  
| stats avg(Percent_CPU_Load) as newAVG by host]
| eval Change_Percentage=(newAVG-avgLoad)*100/avgLoad
| where Change_Percentage >= 20
| table host avgLoad Change_Percentage newAVG

As you can see, I also made the CPU usage calculation based on 100 minus pctIdle, which is something I read to do. Someone may feel free to advise on an alternative to that calculation.

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...