I am using this search to find volume for systems reporting to one index
index="_internal" source="*metrics.log" per_index_thruput series="Customer_Index_group" | stats sum(kb) as kb_indexed | eval GB = round(kb_indexed/1024/1024,2) | gauge GB 0 7 14 21 28
I can then search the metrics logs reported from the systems like this
index="Customer_Index" source="*metrics.log" per_index_thruput series="Customer_Index_group" | stats sum(kb) as kb_indexed | eval GB = round(kb_indexed/1024/1024,2) | gauge GB 0 7 14 21 28
However these two numbers are very different.
Granted the search on the _internal index runes much faster, but my users do not have access to the _internal index and they would like to know who much data there index is using. I see a volume that is much larger on the search form index=_internal than they can see using index="Customer_Index".
Why would the _internal index show more than the info from the $splunk/etc/var/log/splunk/metrics.log?
You give their role access to _internal
and add this to their search restriction terms:
(index!=_internal OR (source=*metrics.log series="Customer_Index"))
In this fashion you could also give them access to their entire UF logs by adding their hosts (or a host tag) here.
http://docs.splunk.com/Documentation/Splunk/4.3.1/Admin/Addandeditroles#Search_filter_format