I want to report the number of events in a given index using a scheduled overnight report and send the PDF output to myself.
So in 'Manager>Searches and reports' I define a search :-
Search = 'index=tal' # name of index
No time range
Tick 'Schedule this search'
Type = Basic
Run every day at 6pm
and enter email details for myself including the PDF report option
The report doesn't return the expected number of events (> 150M) but instead gives a small total (around 220,000) - it seems to be limited by some threshold value that I don't understand.
How do I get the report run for as long as required to tranverse the whole index and send me a report ? (leaving aside for the moment whether it's sensible to do this).
Yes, there are limits in splunk, which you may be hitting up against. See, in your search you are telling Splunk to RETURN all of this data and not just count it.
Splunk has many analytic features, such as "stats" that have arguments like count.
You probably want to do something like this: search = index=tal | stats count
If you want to be sure to just run this for the previous day you can use the timepicker in the search or do this: search = index=tal earliest=-24h@h latest=@d-1s | stats count
If you had multiple indexes you can do ... | stats count by index