Getting Data In

Cloning Data on a Heavy Forwarder

redgoat
Engager
 
Labels (1)
0 Karma
1 Solution

shivanshu1593
Builder

Hello @redgoat ,

It can be done like this (Assuming that each S3 has it's own separate input and host or source or sourcetype, preferably sourcetype)

Identify the input, which you want to send to a separate set of Indexers, obtain it's sourcetype and then On your HF, go to props.conf or create one in the local directory of the AWS add on and put the following:

[host/source/sourcetype_name_here]
TRANSFORMS-routing=newRouting

 

Now, under transforms.conf under the same directory (create one if its missing), put the following

[newRouting]
REGEX= . DEST_KEY=_TCP_ROUTING FORMAT=newGroup

 Then, in outputs.conf under the same local directory (copy your main outputs.conf here and APPEND the following in it)

[tcpout:newGroup]

server=<ip of your indexers, where you want to send the data>:<port number>

 

If you aren't sure or don't use different sourcetypes, let me know and I'll suggest a different solution for it.

Hope this helps.

Thank you,

S

***If this helped, please accept it as a solution. It helps others to find the solution for similar issues quickly.***

 

View solution in original post

mibrahim8
Explorer

Hello, 

Here is a link [ https://mk-datalab.blogspot.com/2021/09/splunk-hf-advanced-data-routing-cloning.html ] of an Article that reference Splunk Documentation and emphasize on the above way in more details and more data routing & cloning scenarios. 

Please check ! and feedback me !

 

Thanks,

Mohamed Khalil

Tags (3)
0 Karma

shivanshu1593
Builder

Hello @redgoat ,

It can be done like this (Assuming that each S3 has it's own separate input and host or source or sourcetype, preferably sourcetype)

Identify the input, which you want to send to a separate set of Indexers, obtain it's sourcetype and then On your HF, go to props.conf or create one in the local directory of the AWS add on and put the following:

[host/source/sourcetype_name_here]
TRANSFORMS-routing=newRouting

 

Now, under transforms.conf under the same directory (create one if its missing), put the following

[newRouting]
REGEX= . DEST_KEY=_TCP_ROUTING FORMAT=newGroup

 Then, in outputs.conf under the same local directory (copy your main outputs.conf here and APPEND the following in it)

[tcpout:newGroup]

server=<ip of your indexers, where you want to send the data>:<port number>

 

If you aren't sure or don't use different sourcetypes, let me know and I'll suggest a different solution for it.

Hope this helps.

Thank you,

S

***If this helped, please accept it as a solution. It helps others to find the solution for similar issues quickly.***

 

View solution in original post

Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!