Removing subset of logs from index while maintaining route to syslog server

timori — Thu, 17 Dec 2020 16:07:35 GMT

Windows security logs are sent to a heavy forwarder, which is configured to send these logs to a syslog server in addition to sending to the indexers. (We have additional tools that require the Windows security logs).

So, right now the config is almost exactly by the book following these docs:

https://docs.splunk.com/Documentation/SplunkCloud/8.1.2011/Forwarding/Forwarddatatothird-partysystemsd

What we're trying to do is move these logs into a different tool and slowly remove them from Splunk indexes, going by region as each domain controller has a specific naming scheme, but retaining the forwarding to syslog.

The problem I'm running into is that adding a nullroute also stops logs from coming into syslog. I think I understand why it's happening. The hostname matches in both stanzas, so the nullroute overwrites the syslog route. I've tried changing the order, making sure the syslog route is last, but that doesn't change anything.

I've been looking and I can't figure out a way to avoid this. The only unique thing I can match on is hostname. I also don't see that transforms or props has a logical NOT, i.e., for these do this, for these do not do this.

props.conf

[source::WinEventLog:Security] TRUNCATE = 0 TRANSFORMS-routing = routeAll, routeSubset, routeSubset2, routeNull

transforms.conf

[routeAll] REGEX=(.) DEST_KEY=_TCP_ROUTING FORMAT=splunkssl [routeSubset] SOURCE_KEY=MetaData:Host REGEX=(?i)(.*dc[0-9][0-9].*) DEST_KEY=_TCP_ROUTING FORMAT=splunkssl [routeSubset2] SOURCE_KEY=MetaData:Host REGEX=(?i)(.*dc[0-9][0-9].*) DEST_KEY=_SYSLOG_ROUTING FORMAT=my_syslog_server [routeNull] SOURCE_KEY=MetaData:Host REGEX=(?i)(.*region1dc[0-9][0-9].*|.*region2dc[0-9][0-9].*) DEST_KEY = queue FORMAT = nullQueue

topic Removing subset of logs from index while maintaining route to syslog server in Getting Data In

Removing subset of logs from index while maintaining route to syslog server