Getting Data In

Heavy Forwarder batch stanza bug

cwacha
Path Finder

Actual Situation:

A Heavy Forwarder with the [batch://] stanza configured using default values is reading files from a directory and forwards them to a second indexer.
Mark that there is no index= mapping defined in the inputs.conf. A call to

splunk cmd btool inputs list --debug

shows that the index for this input is set to default. On the indexer side all log records from this source will end up in index=default (as we would expect). Unfortunately no matter what you configure on the indexers transforms.conf they always end up in the default index. It is not possible to re-direct the events to another index.

Expected Situation:

A proper transforms.conf on the indexer should make it possible to redirect the records to an index of choice.

Additional Findings:

If we add the exact statement

index=test1

to the batch stanza on the Heavy Forwarder (so that the records would go to index test1) the same settings in transforms.conf on the indexer suddenly begin to work!
It seems that redirecting the data on the indexer to an index of choice is only possible if any (even non-existent) index is configured on the Heavy Forwarder side.

I consider this a bug. Please fix. 🙂

0 Karma

Drainy
Champion

If I am understanding this correct. You have an indexer that is also forwarding its results onto another indexer?
If so, once it reads them in once it will assign an index to them, when they are forwarded on they will also head into the same index. In that case you need to define the initial index as you did in your additional findings section which would result in them landing in the correct index.

Some other parts incase I am off the mark;
What version of Splunk are you running? (Indexer and Heavy Forwarder).

Do you definitely require the use of a heavy forwarder or could you swap it out to an universal forwarder? This is more lightweight and if nothing is defined it should happily forward onto an indexer and into the specified index on the receiving side.

0 Karma