Getting Data In

Is it possible to filter, route, and change the index of specific events?

keithyap
Path Finder

Hi All,

Basically the data (WinEventLogs) flow is UF -> HF -> indexer Group 1/ Indexer Group 2.

All the data will go to Indexer Group 1, while subset/filtered data will go to Indexer Group 2 and to another index.

I have managed to configure the HF to send the data to multiple indexer groups but I can't seem to change the index of the events for data that go to Indexer Group 2.

Any advice?

Additionally if I only want to change the index of events coming from a specific HF would that be possible?

Heavy Forwarder Configuration:

outputs.conf

[tcpout]
defaultGroup = none

[tcpout:group1]
disabled = false
server = group1IDX:9997

[tcpout:group2]
disabled = false
server = group2IDX:9997

 

props.conf

[source::WinEventLog:Security]
TRANSFORMS-routing =routeGroup1, routeGroup2


transforms.conf

[routeGroup1]
REGEX = . 
DEST_KEY = _TCP_ROUTING
FORMAT = group1

[routeGroup2]
REGEX = Filter
DEST_KEY = _TCP_ROUTING
FORMAT = group1, group2

 

Indexer Configurations:

props.conf

[source::WinEventLog:Security]
TRANSFORMS-changeIndex = changeIndex

 

transforms.conf

[changeWinIndex]
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = newIndexName
Labels (3)
0 Karma
1 Solution

scelikok
SplunkTrust
SplunkTrust

@keithyap, since you are using HF between UF and Indexers changing the index name will happen on HF. 

You must put all transforms into HF like below. And also be careful, there is a typo on the transforms name on props. 

HF

props.conf
[source::WinEventLog:Security]
TRANSFORMS-changeIndex = changeWinIndex
TRANSFORMS-routing =routeGroup1, routeGroup2

transforms.conf
[routeGroup1]
REGEX = . 
DEST_KEY = _TCP_ROUTING
FORMAT = group1

[routeGroup2]
REGEX = Filter
DEST_KEY = _TCP_ROUTING
FORMAT = group1, group2

[changeWinIndex]
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = newIndexName 

Since you are changing the index on HF, your specific HF question will be also resolved.

 

If this reply helps you an upvote is appreciated.

If this reply helps you an upvote and "Accept as Solution" is appreciated.

View solution in original post

scelikok
SplunkTrust
SplunkTrust

You are right, but I cannot think of a way to change the index for only one output.

 

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

scelikok
SplunkTrust
SplunkTrust

@keithyap, since you are using HF between UF and Indexers changing the index name will happen on HF. 

You must put all transforms into HF like below. And also be careful, there is a typo on the transforms name on props. 

HF

props.conf
[source::WinEventLog:Security]
TRANSFORMS-changeIndex = changeWinIndex
TRANSFORMS-routing =routeGroup1, routeGroup2

transforms.conf
[routeGroup1]
REGEX = . 
DEST_KEY = _TCP_ROUTING
FORMAT = group1

[routeGroup2]
REGEX = Filter
DEST_KEY = _TCP_ROUTING
FORMAT = group1, group2

[changeWinIndex]
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = newIndexName 

Since you are changing the index on HF, your specific HF question will be also resolved.

 

If this reply helps you an upvote is appreciated.

If this reply helps you an upvote and "Accept as Solution" is appreciated.

keithyap
Path Finder

@scelikok  Thanks for the help.

wouldn't the configurations for changing the index do so for both indexer groups? 

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...