topic Re: routing window system logs to a different index in Getting Data In

How to route window system logs to a different index

usup_rajbahak — Fri, 08 Apr 2022 14:20:27 GMT

Hey there,

I have a windows forwarder sending the servers's application, system and security logs to the indexers. I need to route only the security logs to a different index. I've tried a few different things but none seem to be working. This is my latest config

props.conf
[WinEventLog]
TRANSFORMS-FIELDS = WinEventLog

transforms.conf
[WinEventLog]
SOURCE_KEY=sourcetype
REGEX=source=WinEventLog:Security
DEST_KEY=_Metadata:Index
FORMAT=Security

What am I doing wrong here? Thanks a lot

Re: routing window system logs to a different index

Ayn — Mon, 28 Sep 2020 13:21:35 GMT

I see a couple of issues with how you've set things up.

SOURCE_KEY shouldn't be just "sourcetype", it should be "MetaData:Sourcetype".
DEST_KEY should start with "_MetaData", not "_Metadata" (note the capital D).
The "MetaData:Sourcetype" value will be simply the string specifying the sourcetype, so "source=WinEventLog:Security" will not match. Perhaps you want to match on just "WinEventLog:Security"?

Re: routing window system logs to a different index

usup_rajbahak — Mon, 28 Sep 2020 13:21:39 GMT

hey Ayn,

thanks for yoour reply. The logs are still going to the main index.

Here's my latest transforms.conf config

[WinEventLog]
SOURCE_KEY=MetaData:Sourcetype
REGEX=WinEventLog:Security
DEST_KEY=_MetaData:Index
FORMAT=Security

And I restarted splunkd after making the changes.

Re: routing window system logs to a different index

jonasmeier — Fri, 08 Apr 2022 10:49:15 GMT

Years later 🙂

According to https://docs.splunk.com/Documentation/Splunk/latest/Admin/Transformsconf#KEYS: the correct solution was:

[WinEventLog]
SOURCE_KEY=MetaData:Sourcetype
REGEX=sourcetype::WinEventLog:Security
DEST_KEY=_MetaData:IndexFORMAT=Security

Probs:

- Syntax for indexes fields (:: instead of 😃 has to be used in REGEX

-There was a mix between sourcetype (in SOURCE_KEY) and source (in REGEX)

-case sensitivity of indexes is delicate, so I would always only use lowercase (FORMAT)

As of Splunk Add-on for Windows >=5.0.0 sourcetype contains only "WinEventLog" (or XmlWinEventLog) for all EventLogs, so the correct solution to specifically route WinEventLog:Security is:

[WinEventLogSecurityRouting]
SOURCE_KEY=MetaData:Source
REGEX=source::WinEventLog:Security
DEST_KEY=_MetaData:Index
FORMAT=security

Re: routing window system logs to a different index

PickleRick — Fri, 08 Apr 2022 11:27:04 GMT

Why don't you set the proper index on input in the first place?

Re: routing window system logs to a different index

jonasmeier — Fri, 08 Apr 2022 12:18:16 GMT

Any case where a deployed inputs.conf can not be accessed or changed. my szenario was attaching new indexers to an existing infrastructure as preparation for a migration. But index names also changed so for a specific period we had to write events to two different indexes on different indexers. Kind of special case though.

Re: routing window system logs to a different index

PickleRick — Fri, 08 Apr 2022 12:22:30 GMT

Baaah, I didn't notice it was a "golden shovel" post 😉

Of course, in some special cases one can use the index-time manipulation of metadata fields but in general if you can set the metadata right from the start, you should use that functionality. It's good to keep things simple and consistent. Forgotten custom solutions tend to bite you in the rear end in the least appropriate moment 🙂