Are you a member of the Splunk Community?
- Find Answers
- :
- Splunk Administration
- :
- Deployment Architecture
- :
- Re: Can you help me with my routing and filtering ...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have a requirement to send certain filtered log events on to a 3rd party in addition to indexing the events locally.
I've followed this example http://docs.splunk.com/Documentation/Splunk/7.2.0/Forwarding/Routeandfilterdatad#Replicate_a_subset_...
My solution works great. However, I don't quite understand why. To use Splunk's example, why don't I see duplicate (SYSTEM|CONFIG|THREAT) events in my local indexer?? Doesn't the 'routeAll' transform get hit for every event and therefore forwards all events to the 'Everything' destinations? Why does the 'routeSubset' transform need to forward matching events to both 'Subsidiary' AND 'Everything'?
props.conf
[syslog]
TRANSFORMS-routing = routeAll, routeSubset
transforms.conf
[routeAll]
REGEX=(.)
DEST_KEY=_TCP_ROUTING
FORMAT=Everything
[routeSubset]
REGEX=(SYSTEM|CONFIG|THREAT)
DEST_KEY=_TCP_ROUTING
FORMAT=Subsidiary,Everything
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The two transforms happen in order.
The first one sets the value of _TCP_ROUTING to Everything.
The second one overrides the value of _TCP_ROUTING for only those items that match the REGEX, and sets it to both Subsidiary and Everything (two places). There is no duplication, because the prior instruction is overwritten, not added to.
This is similar to the various examples that either route everything to NULLQUEUE and then override what you want to keep, or route everything normally and then override only the ones you want to blacklist. The difference is that the override is sending the event both places.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The two transforms happen in order.
The first one sets the value of _TCP_ROUTING to Everything.
The second one overrides the value of _TCP_ROUTING for only those items that match the REGEX, and sets it to both Subsidiary and Everything (two places). There is no duplication, because the prior instruction is overwritten, not added to.
This is similar to the various examples that either route everything to NULLQUEUE and then override what you want to keep, or route everything normally and then override only the ones you want to blacklist. The difference is that the override is sending the event both places.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
My confusion then comes in how I understand the two transforms are used. I thought each transform is applied to each event. You're saying they are NOT. If an event matches the 'routeSubset' transform, it won't be evaluated against the 'routeAll' transform?? Isn't ordering important? How does the ordering of the transforms in the props.conf file work then?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I get it now. We're building a single transform stanza out of the two based on ordering and precedence rules. Thank you!!!! Your help made all the difference.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
happy to help.
Reduce and Transform Your Firewall Data with Splunk Data Management
Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...
Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions
