Alerting

What is the best and most efficient way to write alert for index with no events?

dannyze
Explorer

What is the best and most efficient way to write alert for index with no events?

I have the following

index=_internal earliest=60m | where count=0

or

| metadata type=sources index=* | eval flatline=round((now()-recentTime)/60,0)

Thank You

Tags (1)
0 Karma
1 Solution

manjunathmeti
SplunkTrust
SplunkTrust

To find indexes with no events use both eventcount and tstats.

| eventcount summarize=false index=* | fields index | dedup index | join type=left [ | tstats count as event_count WHERE (index=* earliest=-60m) by index] | fillnull value=0 | where event_count=0

View solution in original post

woodcock
Esteemed Legend

This has been solved many times including:

Meta Woot!: https://splunkbase.splunk.com/app/2949/
TrackMe: https://splunkbase.splunk.com/app/4621/,
Broken Hosts App for Splunk: https://splunkbase.splunk.com/app/3247/
Alerts for Splunk Admins ("ForwarderLevel" alerts): https://splunkbase.splunk.com/app/3796/
Splunk Security Essentials(https://docs.splunksecurityessentials.com/features/sse_data_availability/): https://splunkbase.splunk.com/app/3435/
Monitoring Console: https://docs.splunk.com/Documentation/Splunk/latest/DMC/Configureforwardermonitoring
Deployment Server: https://docs.splunk.com/Documentation/DepMon/latest/DeployDepMon/Troubleshootyourdeployment#Forwarde...
0 Karma

dannyze
Explorer

Thank you, was looking more for a way to do it with built-in capabilities

0 Karma

woodcock
Esteemed Legend

Most of these are built-in capabilities (searches) with some gift-wrapping around that. My point is: don't reinvent the wheel: download some of these apps (or in the case of #6 and #7, just turn them on) and tear apart their searches and copy what you need.

gjanders
SplunkTrust
SplunkTrust

If you would prefer to go down the apps path
TrackMe
Meta Woot!
Broken Hosts App for Splunk

Or see previous answers for missing indexes/sourcetypes such as this one

0 Karma

niketn
Legend

@dannyze if you want a different approach with REST API you can try the following however commands like tstats, metadata, eventcount and dbinspect are specifically useful while trying to query index related stuff. Since the following approach uses REST API you can output results similar to how you see it in Settings> Data > Indexes view (bring in or filter based on other fields like app name, access etc.

| rest /servicesNS/-/-/data/indexes 
| fields title maxTime
| rename title as index
| eval _time=strptime(maxTime,"%Y-%m-%dT%H:%M:%S+%z"), "Last event indexed age"=now()-_time 
| where 'Last event indexed age'>=3600 OR isnull('Last event indexed age')
| eval "Last event indexed age"=if(isnull('Last event indexed age'),"No Data",
                                   replace(replace(tostring('Last event indexed age',"duration"),"\+"," days "),"(\d+)\:(\d+)\:(\d+)\.\d+","\1 hr \2 min \3 sec"))
| fields - maxTime _time
____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

to4kawa
Ultra Champion
| tstats count where index=* earliest=-60m by index 
| append [| eventcount summarize=f index=* |fields index]
| fillnull count
| where count=0

To fire alert: event count > 0
hi, @manjunathmeti

I modify your query.

0 Karma

manjunathmeti
SplunkTrust
SplunkTrust

To find indexes with no events use both eventcount and tstats.

| eventcount summarize=false index=* | fields index | dedup index | join type=left [ | tstats count as event_count WHERE (index=* earliest=-60m) by index] | fillnull value=0 | where event_count=0
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...