Deployment Architecture

Filter data forwarded from splunk instance 1 to splunk instance 2 using "Configure forwarding"

efika
Communicator

Hello,

I'm setting a new splunk instance that is supposed to replace an old one.
For the sake of validating that all is working correctly I used the "Forwarding and receiving" option to send that data from splunk 1 to splunk 2 - and it is working correctly.

Now since I need in splunk 2 only the data that is sent to splunk 1 I want to filter it.
I've tried to use the nullQueue/indexQueue techniques in indexer 2 props/transforms (configurations like this I've already uses hundreds times in with heavy forwarders) but it is not working in this case.

Appreciate your help !

 

Labels (1)
0 Karma
1 Solution

efika
Communicator

Getting back to this question just to let you know that I've found the right solution.

There is away to tell the second indexer to take the events back to parsing queue.

What you need to do is to add the following line in the inputs.conf of the receiving side.


route=has_key:_utf8:parsingQueue;has_key:_linebreaker:parsingQueue;absent_key:_utf8:parsingQueue;absent_key:_linebreaker:parsingQueue;

Link to the blog post where I've first learned about it : 
https://splunkes.wordpress.com/2019/09/02/re-parsing-queue

 

On splunk docs (search for "Data distribution: ") https://docs.splunk.com/Documentation/Splunk/7.3.5/Admin/Inputsconf

 

 

 

View solution in original post

efika
Communicator

Getting back to this question just to let you know that I've found the right solution.

There is away to tell the second indexer to take the events back to parsing queue.

What you need to do is to add the following line in the inputs.conf of the receiving side.


route=has_key:_utf8:parsingQueue;has_key:_linebreaker:parsingQueue;absent_key:_utf8:parsingQueue;absent_key:_linebreaker:parsingQueue;

Link to the blog post where I've first learned about it : 
https://splunkes.wordpress.com/2019/09/02/re-parsing-queue

 

On splunk docs (search for "Data distribution: ") https://docs.splunk.com/Documentation/Splunk/7.3.5/Admin/Inputsconf

 

 

 

efika
Communicator

@isoutamo , so basically what you are saying that if the data was already indexed on the first instance it goes straight to the indexQ and I can't manipulate it any further ?

0 Karma

isoutamo
SplunkTrust
SplunkTrust

I haven’t try this by myself, but you could try to send data to secondary indexer as tcp not as splunktcp. Then the second instance see it as normal tcp traffic and you can do whatever you want to it.

r. Ismo

efika
Communicator

Good idea, Thanks !

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

when you are looking splunk ingestion pipeline you see that all parsing etc has done on first full Splunk Enterprise instance. The last instance can do only indexing part if you are using S2S connection between nodes.

r. Ismo

0 Karma
Get Updates on the Splunk Community!

Happy CX Day to our Community Superheroes!

Happy 10th Birthday CX Day!What is CX Day? It’s a global celebration recognizing innovation and success in the ...

Check out This Month’s Brand new Splunk Lantern Articles

Splunk Lantern is a customer success center providing advice from Splunk experts on valuable data insights, ...

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...