topic Re: Why isn't my transforms working? in Getting Data In

Why isn't my transforms working?

tmontney — Thu, 11 Jul 2019 20:02:52 GMT

\etc\system\local\transforms.conf

[drop4768OK]
REGEX = EventCode=4768(.|\t|\r|\n)*Result.*Code.*0x0
DEST_KEY = queue
FORMAT = nullQueue

\etc\system\local\props.conf

[source::WinEventLog:Security]
TRANSFORMS-set = drop4768OK

After a reboot, events with Event Code 4768 and Result Code 0x0 are still being indexed. What am I doing wrong?

Re: Why isn't my transforms working?

woodcock — Wed, 30 Sep 2020 01:13:27 GMT

You shouldn't be putting stuff in $SPLUNK_HOME/etc/system/local; you should be creating your own app based on either the sourcetype or the splunk-node type (e.g. Indexer). In any case, if you are doing a sourcetype override/overwrite, you must use the ORIGINAL values NOT the new value (you are not, so that's not your problem), then you must deploy this to the first full instance(s) of Splunk that handles the events (usually either the HF-tier, if you use that, or your Indexer tier), restart all Splunk instances there, send in new events (old events will stay broken), then test using _index_earliest=-5m to be absolutely certain that you are only examining the newly indexed events. I think that last bit is your problem.

Re: Why isn't my transforms working?

oscar84x — Thu, 11 Jul 2019 20:30:48 GMT

Could you provide a few sample events containing both; events you'd like to keep and events you'd like to discard?
The events have to meet both criteria? i.e. Event code AND result code?

Thanks

Re: Why isn't my transforms working?

tmontney — Thu, 11 Jul 2019 20:50:15 GMT

https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventid=4768

Yes, discard events with EventID 4768 AND result code 0x0, keep the rest.

Re: Why isn't my transforms working?

tmontney — Thu, 11 Jul 2019 20:53:09 GMT

So you're saying the transforms should go in the deployed app? The UF discards the event before even sending it?

Re: Why isn't my transforms working?

woodcock — Thu, 11 Jul 2019 21:01:22 GMT

Nope, I am saying it has to go on the Indexers (probably), but DEFINITELY NOT on the UF.

Re: Why isn't my transforms working?

tmontney — Thu, 11 Jul 2019 21:15:53 GMT

Ok, it's on the Indexer. Everything's been restarted. I understand only new events will be affected by the change.

Re: Why isn't my transforms working?

woodcock — Fri, 12 Jul 2019 02:37:27 GMT

Right, so is it working? If so, come back and click Accept to close the question.

Re: Why isn't my transforms working?

tmontney — Fri, 12 Jul 2019 19:59:41 GMT

It is not working. I'll try putting the transforms in the search app.

Re: Why isn't my transforms working?

woodcock — Sun, 14 Jul 2019 02:06:04 GMT

NO! Not in anybody else's stuff. Create your own app in `$SPLUNK_HOME/etc/apps//default/{transforms,props}.conf.

Re: Why isn't my transforms working?

woodcock — Sun, 14 Jul 2019 22:59:33 GMT

First of all, you should have a better RegEx, like ^EventCode=4768[\S\s\r\n]+Result\s*Code:\s+0x0\D. Even so, yours should work. I would try using a sourcetype-based stanza header, instead of your source-based one. Again, what you have should work but since it isn't, let's try something else.

Re: Why isn't my transforms working?

tmontney — Wed, 17 Jul 2019 14:09:07 GMT

Regex isn't my strong suit. I usually mess around with it in a regex "calculator" to ensure it matches correctly before deploying it.

Re: Why isn't my transforms working?

tmontney — Wed, 17 Jul 2019 14:11:16 GMT

Alright. I've never seen this recommended in the 2 years I've worked with Splunk Engineers, Splunk Answers, or Splunk Support. Unless it's specifically related to certain config changes?