On the summary page, under Global summary, there is an "Events indexed" number. I'd like to be able to see the number of events indexed per indexer over time.
Is there a daily summary number for indexed events per indexer?
This would show me if there is any missing data and also if the indexers are taking an equal number of events overtime.
so, the search that populates that view is this:
| metadata type=sourcetypes | search totalCount>0 | rename totalCount as Count recentTime as "Last Update" | stats sum(Count)
You can setup a saved search that writes to a summary index every night at 00:00
Then you will have single values for each 24 hour period and see how your data grows everyday.
Again, this will be number of events, so if you wanted to know actual size you would want to run other searches.
Hope this helps,
This has a couple of problems. First, it only shows a total, so you can only get daily counts by storing and computing the difference between runs. Second and more importantly, the
metadata command does not split out counts by indexer.
You might consider using bytes indexed rather than the count of events if you're just looking for patterns and comparisons. Splunk records this in license_audit.log on each indexer each day:
index=_internal source=*license_audit.log | timechart span=1d sum(todaysBytesIndexed) as prevDaysBytesIndexed by splunk_server
If you must use the event count, you can use:
index=_internal source=*metrics.log "group=thruput" "name=index_thruput" | timechart span=1d sum(ev) as events by splunk_server
This is a somewhat more expensive search, though not really that bad.
Is the "Events Indexed" number telling me how many events were indexed up to that point, or the number of events that are indexed currently (as in the buckets and available for searching)?
I'd like to track that number, but per indexer, in the case of file corruption or something similar.
I'll try your second query and see what I get.