Getting Data In

How to route and filter data from HEC?

martaBenedetti
Path Finder

Hi Community,

I have the need to filter data based on a specific field value and route to a different group of indexers.

Data is coming through HEC configured on a Heavy Forwarder like this:

 

[http://tokenName]
index = main
indexes = main
outputgroup = my_indexers
sourcetype = _json
token = <string>
source = mysource

 

I'd like to use props.conf and transforms.conf as suggested here like this:

 

props.conf
[source::mysource]
TRANSFORMS-routing=otherIndexersRouting

transforms.conf
[otherIndexersRouting]
REGEX=\"domain\"\:\s\"CARD\"
DEST_KEY=_TCP_ROUTING
FORMAT=other_indexers

 

In outputs.conf I'd add the stanza [tcpOut:other_indexers]

 

Is this possible? Is there another way to achieve this goal?

 

Thank you

Marta

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

The configuration provided in the link should be the way to go. Just make sure that you chose appropriate source ([source::YourSource]) OR sourcetype in props.conf.

Another example:

https://community.splunk.com/t5/Getting-Data-In/How-to-route-and-filter-data-on-the-Heavy-Forwarder-...

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

The configuration provided in the link should be the way to go. Just make sure that you chose appropriate source ([source::YourSource]) OR sourcetype in props.conf.

Another example:

https://community.splunk.com/t5/Getting-Data-In/How-to-route-and-filter-data-on-the-Heavy-Forwarder-...

martaBenedetti
Path Finder

I was afraid the solution couldn't work with HTTP event collector since I've only used this configuration with classic monitor inputs.

 

The source stanza was just a typo, I've corrected it 🙂

 

Thank you

Marta

0 Karma

PickleRick
SplunkTrust
SplunkTrust

Close.

You need to specify the stanza in props.conf as

[source::mysource]

Then you can call appropriate transforms from there.

Keep in mind though that the hierarchy is source->host->sourcetype so if you have - for example - your host field overwritten based on data from the raw event in a transform called from sourcetype-based stanza, you won't be able to use this host value as selector.

martaBenedetti
Path Finder

The source stanza was just a typo, I've corrected it 🙂

 

Thank you

Marta

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...