Getting Data In

Removing subset of logs from index while maintaining route to syslog server

timori
Loves-to-Learn Lots

Windows security logs are sent to a heavy forwarder, which is configured to send these logs to a syslog server in addition to sending to the indexers. (We have additional tools that require the Windows security logs).

So, right now the config is almost exactly by the book following these docs:

https://docs.splunk.com/Documentation/SplunkCloud/8.1.2011/Forwarding/Forwarddatatothird-partysystem...

What we're trying to do is move these logs into a different tool and slowly remove them from Splunk indexes, going by region as each domain controller has a specific naming scheme, but retaining the forwarding to syslog.

The problem I'm running into is that adding a nullroute also stops logs from coming into syslog. I think I understand why it's happening. The hostname matches in both stanzas, so the nullroute overwrites the syslog route. I've tried changing the order, making sure the syslog route is last, but that doesn't change anything.

I've been looking and I can't figure out a way to avoid this. The only unique thing I can match on is hostname. I also don't see that transforms or props has a logical NOT, i.e., for these do this, for these do not do this.

props.conf

 

[source::WinEventLog:Security]
TRUNCATE = 0
TRANSFORMS-routing = routeAll, routeSubset, routeSubset2, routeNull

 

transforms.conf

 

[routeAll]
REGEX=(.)
DEST_KEY=_TCP_ROUTING
FORMAT=splunkssl

[routeSubset]
SOURCE_KEY=MetaData:Host
REGEX=(?i)(.*dc[0-9][0-9].*)
DEST_KEY=_TCP_ROUTING
FORMAT=splunkssl

[routeSubset2]
SOURCE_KEY=MetaData:Host
REGEX=(?i)(.*dc[0-9][0-9].*)
DEST_KEY=_SYSLOG_ROUTING
FORMAT=my_syslog_server

[routeNull]
SOURCE_KEY=MetaData:Host
REGEX=(?i)(.*region1dc[0-9][0-9].*|.*region2dc[0-9][0-9].*)
DEST_KEY = queue
FORMAT = nullQueue

 

 

Labels (3)
0 Karma
Get Updates on the Splunk Community!

What the End of Support for Splunk Add-on Builder Means for You

Hello Splunk Community! We want to share an important update regarding the future of the Splunk Add-on Builder ...

Solve, Learn, Repeat: New Puzzle Channel Now Live

Welcome to the Splunk Puzzle PlaygroundIf you are anything like me, you love to solve problems, and what ...

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...