Splunk Search

setting up alert for search on index when counts for parameter are 0

jdepp
Path Finder

I am able to set up alerts for an index when the count = 0 for a specific parameter value. Since I have over 50 over these values, it is cumbersome to create so many alerts and wondered if I can create one alert that is able to monitor across these values. Let me explain with example:

source="/statsData.newIndex/tcp/10014" 

This is the general search for the main index but for the specific parameter value:

source="/statsData.newIndex/tcp/10014" PStream=864

So I set up alerts for multiple values of this PStream field that monitors the count, and alerts when count =0 for the last hour. Is there a way to construct the search so that it monitors all values of this field PStream and for each one if the count =0, I will get the appropriate alert?

Tags (1)
0 Karma

woodcock
Esteemed Legend

Like this:

... source="/statsData.newIndex/tcp/10014" | stats count(eval(PStream="864")) AS countPstream864 ... count(eval(otherField="otherValue")) AS countOtherValue | transpose| rename "row 1" AS count | where count=0

Then setup alert for Number of Events > 0.

jdepp
Path Finder

Thanks for your help. Does this search apply to all where PSTream=* or do I add for each PStream='864', '865', '866'

countPstream864 ... count

why are there dots in between?

0 Karma

woodcock
Esteemed Legend

The dots are showing you where you need to add your "over 50 of these" (as you put it in your question). So for the 3 values you just noted, it would look like this:

... source="/statsData.newIndex/tcp/10014" | stats count(eval(PStream="864")) AS countPstream864 count(eval(PStream="865")) AS countPstream865 count(eval(PStream="866")) AS countPstream866 | transpose| rename "row 1" AS count | where count=0
0 Karma

jeffland
SplunkTrust
SplunkTrust

How about this:

source="/statsData.newIndex/tcp/10014" | stats count by PStream | where count=0

This search should give you a result with a line for each value of PStream with zero count. If you create an alert on that, you could either go easy and contain the search result in that one email (so the recipient has to look at it to determine which PStream(s) are affected).

Or, if you need to explicitly send an email for each PStream, you could create a lookup for the appropriate address for each PStream and use the sendemail command with a map command (docs here and here). I have to admit I'm not sure about that second method, it appears that it has been asked before, I only skimmed the answers though.

jdepp
Path Finder

thanks for your suggestion.

0 Karma

woodcock
Esteemed Legend

The problem is that you will only be counting things that have non-zero instances (and he needs the opposite).

0 Karma

jeffland
SplunkTrust
SplunkTrust

Oh. You're right. Didn't think this through apparently.
In that case, ignore the first part of this answer - but the second one might still be useful if you intend to send emails depending on which PStream is affected.

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...