Getting Data In

heavy forwarder to route base on _raw values

gooza
Communicator

I'm trying to use the heavy forwarder to route data to different indexes based on values in _raw , is this possible ?

the configuration files are:

inputs.conf:

[tcp://9997]

sourcetype = FromFooandBarbysinglehost

props.conf:

[FromFooandBarbysinglehost]

BREAK_ONLY_BEFORE = ^

TRANSFORMS-routing = FromFoo,FromBar

transforms.conf:

[FromFoo]

REGEX = (?i) From|.+?Foo

DEST_KEY = _TCP_ROUTING

FORMAT = outtoFoo

[FromBar]

REGEX = (?i) From|.+?Bar

DEST_KEY = _TCP_ROUTING

FORMAT = outtoBar

outputs.conf:

[tcpout:outtoFoo]

server = 10.10.10.10:1111

sendCookedData = false

[tcpout:outtoBar]

server = 10.10.10.10:2222

sendCookedData = false

in the indexer 10.10.10.10 the TCP port 1111 is indexed to foo index , and 2222 is indexed to Bar index)

me problem is that I see both foo data and bar data in both indexes , it is like there is no termination to the transforms process and both are sent to both ports.

I double checked my REGEX in the search bar in splunk and it does show only the relevant data

what am I missing ?

0 Karma

Jason
Motivator

_TCP_ROUTING is used to send data to another indexer. If the data is already at the indexer you want, just use the _MetaData:Index key to change the index for that event.

[FromFoo]
REGEX = (?i) From|.+?Foo
DEST_KEY = _MetaData:Index
FORMAT = foo_index

[FromBar]
REGEX = (?i) From|.+?Bar
DEST_KEY = _MetaData:Index
FORMAT = bar_index
0 Karma

gooza
Communicator

no , the tcpout redirect the data to different TCP ports on the indexer and each port has its own source type and index

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

when your data arrives in the indexers, what is the sourcetype? Is it FromFooandBarbysinglehost?

0 Karma
Get Updates on the Splunk Community!

Observability | How to Think About Instrumentation Overhead (White Paper)

Novice observability practitioners are often overly obsessed with performance. They might approach ...

Cloud Platform | Get Resiliency in the Cloud Event (Register Now!)

IDC Report: Enterprises Gain Higher Efficiency and Resiliency With Migration to Cloud  Today many enterprises ...

The Great Resilience Quest: 10th Leaderboard Update

The tenth leaderboard update (11.23-12.05) for The Great Resilience Quest is out >> As our brave ...