topic Re: Index some of a certain event code in Getting Data In

Index some of a certain event code

tmontney — Tue, 08 Oct 2019 18:27:33 GMT

Follow-up (ish) to https://answers.splunk.com/answers/757315/why-isnt-my-transforms-working.html as I let it sit idle for a while.

I want to index Event Code 4688: https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4688 Problem is that it generates a high volume of events. I determined I can filter the field Security_ID for values starting with S-1-5-21- (includes real users only). I want this to apply only to event code 4688. All other events should process normally.

inputs.conf (forwarder app)

[WinEventLog://Security]
disabled = 0
index = test
interval = 60
blacklist = 4688

[WinEventLog://Security]
disabled = 0
index = test
interval = 60
whitelist = 4688
sourcetype = Security4688

\etc\apps\myapp\local\props.conf

[Security4688]
TRANSFORMS-set = setnull,useronly4688

\etc\apps\myapp\local\transforms.conf

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[useronly4688]
REGEX = S-1-5-21-
DEST_KEY = queue
FORMAT = indexQueue

My thought was to have the forwarder change the sourcetype of event 4688, and leave all others alone. On the server, props can use that sourcetype to only use the regular expression on event 4688, and send non-matches to nullQueue. What's wrong here? Is it my use of duplicate stanzas?

Re: Index some of a certain event code

richgalloway — Tue, 08 Oct 2019 18:43:07 GMT

They're not really duplicate stanzas, just the same stanza with multiple definitions. Splunk merges all attributes with the same stanza name into a single stanza. So in this case you'll end up with EventCode 4688 on both the whitelist and blacklist.

Re: Index some of a certain event code

tmontney — Tue, 08 Oct 2019 21:25:42 GMT

So how would I index the Security log, where 4688 becomes a different sourcetype, and the rest of the Security log indexes like normal? Separate apps?

Re: Index some of a certain event code

ivanreis — Wed, 09 Oct 2019 00:36:27 GMT

The blacklist has precedence over whitelist, so in this case the blacklist will not index any data from security event 4688.

"It is not necessary to define both a whitelist and a blacklist in a stanza. They are independent settings. If you do define both and a file matches both, the forwarder does not index that file as blacklist overrides whitelist." this note are from this document

https://docs.splunk.com/Documentation/SplunkCloud/7.2.7/Data/Whitelistorblacklistspecificincomingdata

My suggestion is to use this inputs.conf only
[WinEventLog://Security]
disabled = 0
index = test
interval = 60
whitelist = 4688
sourcetype = Security4688

This props.conf
[Security4688]
TRANSFORMS-set = setnull,useronly4688

and the transforms, because here you are send all the security event 4688 and others to null queue, except those ones starting with S-1-5-21-

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[useronly4688]
REGEX = S-1-5-21-
DEST_KEY = queue
FORMAT = indexQueue