topic Re: Index Volume, Licence Use Question in Security

Index Volume, Licence Use Question

hartfoml — Mon, 28 Sep 2020 16:19:48 GMT

I am using this search to find volume for systems reporting to one index

index="_internal" source="*metrics.log" per_index_thruput series="Customer_Index_group" | stats sum(kb) as kb_indexed | eval GB = round(kb_indexed/1024/1024,2) | gauge GB 0 7 14 21 28

I can then search the metrics logs reported from the systems like this

index="Customer_Index" source="*metrics.log" per_index_thruput series="Customer_Index_group" | stats sum(kb) as kb_indexed | eval GB = round(kb_indexed/1024/1024,2) | gauge GB 0 7 14 21 28

However these two numbers are very different.
Granted the search on the _internal index runes much faster, but my users do not have access to the _internal index and they would like to know who much data there index is using. I see a volume that is much larger on the search form index=_internal than they can see using index="Customer_Index".

Why would the _internal index show more than the info from the $splunk/etc/var/log/splunk/metrics.log?

Re: Index Volume, Licence Use Question

martin_mueller — Mon, 07 Apr 2014 12:38:05 GMT

Why does your customer index contain Splunk metrics logs?

Re: Index Volume, Licence Use Question

hartfoml — Mon, 07 Apr 2014 12:47:48 GMT

So that the customer [who did not want to install the splunk UF] can see and troubleshoot splunk UF issues.

Re: Index Volume, Licence Use Question

martin_mueller — Mon, 07 Apr 2014 12:49:38 GMT

Are those metrics from the UFs or from the Indexers?

Re: Index Volume, Licence Use Question

hartfoml — Mon, 07 Apr 2014 12:53:45 GMT

they are for the UF. I know this is maybe not best practice because the metrics.log's put in the customers index count against the license.

Where is the best place to record the UF Metrics Logs so that they don't count agents the license and how could I give this info to the customer without letting them see too much.

Re: Index Volume, Licence Use Question

martin_mueller — Mon, 07 Apr 2014 13:08:34 GMT

You could give them access to _internal but restrict that to metrics about their index.

Re: Index Volume, Licence Use Question

hartfoml — Mon, 28 Sep 2020 16:19:52 GMT

Thanks for helping Martin, I really appreciate it.

So How would I do that. All the customer users are in a group/Role. The group has access to there index.

I would give them access to the _internal but how do I restrict access in only the _internal to the search term [series="Customer_Index_group"]

I am on version 4.3.1, build 119532 PS will be onsite to help with the upgrade in 8 weeks. Until then I can not upgrade.

Re: Index Volume, Licence Use Question

martin_mueller — Mon, 07 Apr 2014 13:22:42 GMT

You give their role access to _internal and add this to their search restriction terms:

(index!=_internal OR (source=*metrics.log series="Customer_Index"))

In this fashion you could also give them access to their entire UF logs by adding their hosts (or a host tag) here.

http://docs.splunk.com/Documentation/Splunk/4.3.1/Admin/Addandeditroles#Search_filter_format

Re: Index Volume, Licence Use Question

hartfoml — Mon, 07 Apr 2014 13:27:18 GMT

I will try this out as soon as I can. Could you add this as your answer and if it works I can give you credit for the answer 🙂

Re: Index Volume, Licence Use Question

hartfoml — Tue, 08 Apr 2014 20:33:02 GMT

this is the final filer if anyone is interested. Thanks Martin for getting me there

index=customer_index OR (index=_internal AND series="customer_index")

Re: Index Volume, Licence Use Question

martin_mueller — Tue, 08 Apr 2014 20:35:51 GMT

I recommend keeping the restriction on the source field in _internal - else they'll be able to see random events that happen to contain series=customer_index caught by default key-value extraction.