Getting Data In

Changing the index on forwarded data

brianm1002
New Member

I have one Splunk indexer that both indexes and forwards the data to a second Splunk indexer. The name of the index is not the same on both indexers. When the data is received by the second indexer, I get the error that it received an event for an unrecognized index.

How can I change the index when it's sent to the second indexer? I've tried adding an index=newindex to the inputs.conf on the second indexer, but it's still trying to use the name of the index from the first indexer.

Brian

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Why not just name the indexes the same? It's not really a good idea to configure Splunk to be able to make this change. Basically, you would need to put:

[splunktcp]
route=has_key:_linebreaker:parsingQueue;absent_key:_utf8:parsingQueue;absent_key:_linebreaker:parsingQueue

into inputs.conf on the second indexer, then send all the received data through a TRANSFORM to set the index key. This can have non-trivial consequences, so I strongly discourage it.

0 Karma

jrodman
Splunk Employee
Splunk Employee

Note that reparsing already parsed events means that they will be timestamped and possibly merged a second time, with possibly worse results than the first time.

There are also some performance costs with processing all data again even if it is already processed.

0 Karma

brianm1002
New Member

The first indexer is managed by a different department and I have no control over it. The second indexer is one that I maintain, and I'm currently using multiple indexes to segregate different sets of data based on disk space and retention requirements.

What are some of the consequences of doing as you suggest?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...