What is the best and most efficient way to write alert for index with no events?

[dannyze]

What is the best and most efficient way to write alert for index with no events?

I have the following

index=_internal earliest=60m | where count=0

or

| metadata type=sources index=* | eval flatline=round((now()-recentTime)/60,0)

Thank You

[woodcock]

This has been solved many times including:

Meta Woot!: https://splunkbase.splunk.com/app/2949/
TrackMe: https://splunkbase.splunk.com/app/4621/,
Broken Hosts App for Splunk: https://splunkbase.splunk.com/app/3247/
Alerts for Splunk Admins ("ForwarderLevel" alerts): https://splunkbase.splunk.com/app/3796/
Splunk Security Essentials(https://docs.splunksecurityessentials.com/features/sse_data_availability/): https://splunkbase.splunk.com/app/3435/
Monitoring Console: https://docs.splunk.com/Documentation/Splunk/latest/DMC/Configureforwardermonitoring
Deployment Server: https://docs.splunk.com/Documentation/DepMon/latest/DeployDepMon/Troubleshootyourdeployment#Forwarde...

[dannyze]

Thank you, was looking more for a way to do it with built-in capabilities

[woodcock]

Most of these are built-in capabilities (searches) with some gift-wrapping around that. My point is: don't reinvent the wheel: download some of these apps (or in the case of #6 and #7, just turn them on) and tear apart their searches and copy what you need.

[gjanders]

If you would prefer to go down the apps path
TrackMe
Meta Woot!
Broken Hosts App for Splunk

Or see previous answers for missing indexes/sourcetypes such as this one

[niketn]

@dannyze if you want a different approach with REST API you can try the following however commands like tstats, metadata, eventcount and dbinspect are specifically useful while trying to query index related stuff. Since the following approach uses REST API you can output results similar to how you see it in Settings> Data > Indexes view (bring in or filter based on other fields like app name, access etc.

| rest /servicesNS/-/-/data/indexes 
| fields title maxTime
| rename title as index
| eval _time=strptime(maxTime,"%Y-%m-%dT%H:%M:%S+%z"), "Last event indexed age"=now()-_time 
| where 'Last event indexed age'>=3600 OR isnull('Last event indexed age')
| eval "Last event indexed age"=if(isnull('Last event indexed age'),"No Data",
                                   replace(replace(tostring('Last event indexed age',"duration"),"\+"," days "),"(\d+)\:(\d+)\:(\d+)\.\d+","\1 hr \2 min \3 sec"))
| fields - maxTime _time

[to4kawa]

| tstats count where index=* earliest=-60m by index 
| append [| eventcount summarize=f index=* |fields index]
| fillnull count
| where count=0

To fire alert: event count > 0
hi, @manjunathmeti

I modify your query.

[manjunathmeti · Accepted solution]

To find indexes with no events use both eventcount and tstats.

| eventcount summarize=false index=* | fields index | dedup index | join type=left [ | tstats count as event_count WHERE (index=* earliest=-60m) by index] | fillnull value=0 | where event_count=0