Alerting

How to setup email alerts when linux servers are 80% full?

mike000
New Member

As always I know you will be able to answer my question.
So using this query:

index=_nix_xxxx sourcetype=df host=abdhw003 OR host=n OR host=n OR host=n or host=n MountedOn="/doc"
 | eval TotalGBytes= TotalMBytes/1024 | eval UsedGBytes=UsedMbytes/1024 |eval used_pct=100(UsedGBytes/TotalGBytes) 
 | stats max(TotalGBytes) as "MaxSize(GB) max(UsedGBytes) as "UsedSize(GB) as "percentUsed" by host, MountedOn 
 | search PercentUsed>05| Sort PercentUsed

I am able to see the space used by each server, is there a way wherein the dashboard once any server hits 80% or 90% used- the color of that server changes to red and an email is triggered to the support team that a certain server has reached 90% capacity?

Is that a query or something to be parameterized in the dashboard itself? Trying to understand Splunk, I appreciate all the help.

Thanks,
Mike

Labels (2)
0 Karma

woodcock
Esteemed Legend

Something like this:

index="_nix_xxxx " AND sourcetype="df" 
| stats avg(PercentUsedSpace) AS avgPctUsed BY host Filesystem 
| eval threshold = case( 
   match(Filesystem, "foo"), 85, 
   ...
   true(), 90) 
| where avgPctUsed > threshold
| sort 0 - avgPctUsed 
0 Karma

mike000
New Member

Hi woodcock, thanks for your reply. If I may ask, What will this do? Does this part just change the color or send email trigger? I cant see either in the snippet.

0 Karma

woodcock
Esteemed Legend

Just save it as an alert, add the alert action of email, done.

0 Karma
Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...