We have a fairly large Splunk environment with several 1000 hosts reporting in. Within our business we have requirements around access, such that some folks can see just their hosts, some can see all hosts in a region and some others can see everything.
To date, we have used host tagging to provision access controls. For example, a new host starts reporting into Splunk and we give it a host tag in the format of something like biz-subbiz-subsubbiz. This then allows me to provision someone with a role that can see all hosts tagged with biz* and someone else with a more restrictive role of biz-subbiz-subsubiz.
For the most part, this strategy has worked ok. However, maintaining the tags in the tags.conf file is a pain. Additionally, we will soon be expecting a lot more hosts (8000+) which means each one has to be tagged appropriately and I am worried/conscious that our tagging strategy may not scale.
I came up with a new approach using a lookup file that listed all the hosts and then a role tag. We implemented this as an automated lookup that would bring in the host role tag and then we created a role to match the role tag that is brought in by the lookup. This also works, but searching is much slower than using the host tagging approach. This is because the search seems to be searching through all the log events for the particular host role. Whereas with the host tagging approach it was only searching the data from the hosts with the correct host tag.
So, do we continue with the host tagging approach (i.e. will it scale?), or is there some other method. Putting the data into different indexes is not really a viable option as even within the index we would need granular controls, so i am trying to live with what we have. I should also note that even though we have 8000 hosts we only have about 100 or so tags and corresponding roles.
Thanks!
... View more