How do you create a dashboard that indicates data availability?

[Karlit0]

Dear all,

I am looking for a way to create a dashboard where I could check the data status and its availability. For now I cannot tell if I receive data from all indexes, hosts, source types, until I check it manually. For other tools, I can have a dashboard with green/red fields indicating the data availability — just wondering if this is also possible with Splunk.

Thank you in advance!

[gcusello]

Hi Karlit0,
you have to define a perimeter of your ingestion, in other words a lookup containing the list of the sources or the hosts that you're waiting for.
In few words, if you need to check if all your hosts are sending logs, you have to create a lookup called e.g. Perimeter.csv, in which there's at least the host column.
Then you have to run a search like the following:

index=_internal
| host=upper(host)
| stats count BY host
| append [ 
     | inputlookup Perimeter.csv
     | eval host=upper(host), count=0
     | fields host count
     ]
| stats sum(count) AS Total BY host

In this way all the hosts where Total = 0 are missing, instead host where Total > 0 are present.
You can use this approach also to check othes things as sources or sourcetypes.

Bye.
Giuseppe

[joshualarkins]

Sounds like a combination of tstats BY host, sourcetype piped into a timechart might be useful here? I'm going off memory here, so I'm not sure if that'll lead to what you want.

You still need to consider what ultimate action you want to take - do you want a dashboard that you manually check? Is the lack of a particular host / sourcetype something you need to be alerted on? Basically, what action will you potentially take after looking at the dashboard? Use that action to drive how you solve the use case.