Getting Data In

Removing subset of logs from index while maintaining route to syslog server

timori
Loves-to-Learn Lots

Windows security logs are sent to a heavy forwarder, which is configured to send these logs to a syslog server in addition to sending to the indexers. (We have additional tools that require the Windows security logs).

So, right now the config is almost exactly by the book following these docs:

https://docs.splunk.com/Documentation/SplunkCloud/8.1.2011/Forwarding/Forwarddatatothird-partysystem...

What we're trying to do is move these logs into a different tool and slowly remove them from Splunk indexes, going by region as each domain controller has a specific naming scheme, but retaining the forwarding to syslog.

The problem I'm running into is that adding a nullroute also stops logs from coming into syslog. I think I understand why it's happening. The hostname matches in both stanzas, so the nullroute overwrites the syslog route. I've tried changing the order, making sure the syslog route is last, but that doesn't change anything.

I've been looking and I can't figure out a way to avoid this. The only unique thing I can match on is hostname. I also don't see that transforms or props has a logical NOT, i.e., for these do this, for these do not do this.

props.conf

 

[source::WinEventLog:Security]
TRUNCATE = 0
TRANSFORMS-routing = routeAll, routeSubset, routeSubset2, routeNull

 

transforms.conf

 

[routeAll]
REGEX=(.)
DEST_KEY=_TCP_ROUTING
FORMAT=splunkssl

[routeSubset]
SOURCE_KEY=MetaData:Host
REGEX=(?i)(.*dc[0-9][0-9].*)
DEST_KEY=_TCP_ROUTING
FORMAT=splunkssl

[routeSubset2]
SOURCE_KEY=MetaData:Host
REGEX=(?i)(.*dc[0-9][0-9].*)
DEST_KEY=_SYSLOG_ROUTING
FORMAT=my_syslog_server

[routeNull]
SOURCE_KEY=MetaData:Host
REGEX=(?i)(.*region1dc[0-9][0-9].*|.*region2dc[0-9][0-9].*)
DEST_KEY = queue
FORMAT = nullQueue

 

 

Labels (3)
0 Karma
Get Updates on the Splunk Community!

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

Splunk Enterprise Security 8.x: The Essential Upgrade for Threat Detection, ...

Watch On Demand the Tech Talk on November 6 at 11AM PT, and empower your SOC to reach new heights! Duration: ...

Splunk Observability as Code: From Zero to Dashboard

For the details on what Self-Service Observability and Observability as Code is, we have some awesome content ...