topic How to keep only certain users events from Windows event log security? in Splunk Search

How to keep only certain users events from Windows event log security?

cafissimo — Wed, 29 Nov 2017 14:14:31 GMT

Hello,
I would like to filter, at the indexers, events coming from WinEventLog:Security to keep only certain users .
The problem is that the list of users is really huge and contains, more or less, 1200 entries. The customer I am working for has set up a REGEX with 1199 pipes(!) for 1200 entries.
A little sample of what is inside the transforms.conf file is:

REGEX = (?i)(A111111)|(A111112)|(A111118)... and so on with 1199 pipes

The final result is that the indexers (2 with 12 cores and 12 GB of RAM) become unresponsive, there is huge indexing lag and a lot of broken pipe connections from Windows Universal Forwarders.
How can I keep, in a better way, only those users' events?

Thanks in advance.

Re: How to keep only certain users events from Windows event log security?

harsmarvania57 — Wed, 29 Nov 2017 14:28:11 GMT

Hi @cafissimo,

Have you tried with filtering those Users on forwarder instead of at Indexer. Please refer http://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Data/MonitorWindowseventlogdata and search for Create advanced filters with 'whitelist' and 'blacklist', I have tried this config with SourceName and it is working fine but with small number of SourceName. And I am not sure how Splunkforwarder behaves with 1200 User filtering and also if you want to configure Username explicitly then approach which I have provided will not work because you need to add 1200 whitelist.

Re: How to keep only certain users events from Windows event log security?

gcusello — Wed, 29 Nov 2017 14:30:00 GMT

Hi cafissimo,
I think that probably you riched the limit of the lenght of a regex and there a very higl overload on the Indexers caused by the regex!
In addition this solution isn't well manageable because for every update you have to restart all your Splunk indexers!

Did you checked what's the difference (in license use) indexing events for all the users or if the not interesting users are less that the interesting ones?
maybe there a little difference!

In addition: are you sure that you need all the Windows EventCodes, maybe not indexing some events you reach the same goal of users reducing.

Anyway, to semplificate you regex, you should try to identify some common username parts (e.g.: A111) so you could limit the regex lenght, but not the work for the Indexers!

I hope to be useful.

Bye.
Giuseppe

Re: How to keep only certain users events from Windows event log security?

cafissimo — Wed, 29 Nov 2017 14:33:23 GMT

I have tried whitelisting @ forwarder side for Windows Event Codes (the most typical are 4624, 4625, 4634, ...) but never tried with a so big list and since I am working in a production environment with Domain Controllers I cannot make such test.
I need a proven solution.
Thank you.

Re: How to keep only certain users events from Windows event log security?

cafissimo — Wed, 29 Nov 2017 14:36:58 GMT

I was thinking about writing a python or AWK program to optimize the REGEX, but it's not that simple in my scenario.
I am working in a PCI environment and the customer prefers to index "more" data than needed.
For the time being I have ended up writing a short REGEX that keeps all users beginning with "X" followed by 6 chars.
I was just wondering if anyone has ever faced a similar situation.
Thank you.

Re: How to keep only certain users events from Windows event log security?

gcusello — Wed, 29 Nov 2017 14:52:18 GMT

Hi cafissimo,I think that, if you can, the best way is to index all users (eventually reducing EventCodes) and then filter them in search, using a lookup.
Bye.
Giuseppe

Re: How to keep only certain users events from Windows event log security?

cafissimo — Wed, 29 Nov 2017 14:57:28 GMT

Yes, I know it and I agree with you, but I should change some correlation searches in Splunk App for PCI Compliance.
Other than this I'll have to check next days if indexing volume is not increasing too much.
Thanks.

Re: How to keep only certain users events from Windows event log security?

aholzel — Thu, 07 Dec 2017 08:26:57 GMT

If all the usernames start with an "A" and then have 6 numbers you can use this:

([aA][0-9]{6})

if you just want A111111 to A112311 you can use:

([aA]11[12][0-9]{3})