topic Re: Reporting the number of events in an index in Splunk Search

Reporting the number of events in an index

bhiley — Tue, 19 Jul 2011 03:32:05 GMT

I want to report the number of events in a given index using a scheduled overnight report and send the PDF output to myself.
So in 'Manager>Searches and reports' I define a search :-

Search = 'index=tal' # name of index
No time range
Tick 'Schedule this search'
Type = Basic
Run every day at 6pm
and enter email details for myself including the PDF report option

The report doesn't return the expected number of events (> 150M) but instead gives a small total (around 220,000) - it seems to be limited by some threshold value that I don't understand.
How do I get the report run for as long as required to tranverse the whole index and send me a report ? (leaving aside for the moment whether it's sensible to do this).

Can supply any further info as required.

Re: Reporting the number of events in an index

sdwilkerson — Tue, 19 Jul 2011 09:10:37 GMT

Bhiley,

Yes, there are limits in splunk, which you may be hitting up against. See, in your search you are telling Splunk to RETURN all of this data and not just count it.
Splunk has many analytic features, such as "stats" that have arguments like count.

You probably want to do something like this:
search = index=tal | stats count

If you want to be sure to just run this for the previous day you can use the timepicker in the search or do this:
search = index=tal earliest=-24h@h latest=@d-1s | stats count

If you had multiple indexes you can do ... | stats count by index

Find more about stats here: http://www.splunk.com/base/Documentation/latest/SearchReference/stats

Best,
Sean