How to detect Splunk searches ran by users without justification?

[sahiltcs]

We are looking for query to detect Splunk queries without business justification and also random validation of business justification for splunk queries

[woodcock]

There is no good way to do this and it really is backwards to general splunk philosophy of Show everybody everything. Your PS engineers during installation should have had a very serious conversation about index values and roles because these 2 things are the only tools that you have to limit access. PCI should not be in Splunk anyway, so nobody should care if anybody is poking around anywhere, unless he is ignoring his "real job".

[johnvr · Accepted solution]

There's plenty you can do in Splunk to monitor user activity - I would recommend the top answer to this previous question in how to do that:

https://answers.splunk.com/answers/12477/get-users-search-history.html

It shouldn't be difficult to build a dashboard looking at those, search performance, and other statistics like total # of searches, searches with poor SPL like "index=*", etc.

However, as @richgalloway mentioned, Splunk doesn't really have a concept of business justification.

You're better off looking at user and role permissions and limits to determine who has access to what datasets based on who would have a business justification for doing so.

It's my experience that "micro-managing" user searches in Splunk can lead to a negative work environment, especially if Splunk is being used in a security capacity. Especially for threat-hunting or IR, where searches can be run in parallel or quick succession, and a long process to justify each one can become prohibitive to the end goal.

[FrankVl]

What I have seen before is that people using Splunk are obliged to register their justification in some ticketing system and then for each search related to that add something like | eval ticket=1234 to the search.

You can then check the _audit index for people running ad-hoc searches without a ticket number. Or even do spot checks whether searches make sense with respect to the ticket that is mentioned.

Not perfect, apart from that it is pretty annoying for the users, as there are various ways in splunk to run searches other than ad-hoc (e.g. in a dashboard panel etc.) and there are also activities (e.g. drilldown) that trigger a search to be started without the user being able to put in a ticket number.

[richgalloway]

Splunk has no concept of "business justification". Any user with access to Splunk can query any index for which they have read permission.
Please explain your use case more completely.

[sahiltcs]

We need to reduce monitoring efforts, the correlation search in Splunk enterprise security will be executed once a week and will aggregate all queries without justification for a given user. Once a week one notable event per user shall be created containing all queries for that given user that do not contain a ticket or notable event ID as justification

Also RANDOM VALIDATION OF BUSINESS JUSTIFICATION FOR SPLUNK QUERIES, Once a week generate N notable events for randomly selected Splunk queries containing a business justification

Any thoughts?