Getting Data In

Index override on heavy forwarder data

yu94
New Member

Hi,

There is situation where we have installed DB connect on HF and then the HF sends that data to 2 sets of different indexers and now we need to override the index name at one set of indexers .

We have tried to override the index which is coming from UF is working fine.

When we tried to override the index which is coming from HF (DB Connect), it was not working may be due to the metadata already set by this HF.

Can you help me to fix this issue?

Regards,
Thippesh

0 Karma

gjanders
SplunkTrust
SplunkTrust

Since this is a heavy forwarder the data is cooked on this server, therefore the index setting is done here.

Is it possible to get both sets of indexers to use the same index for this data?
If not the transforms.conf might work but you would need to do the transform based on which output was chosen, and I'm unsure how to do that part, also DB Connect can set an index, but it will only set one per input.

Perhaps you can change your indexers to have the same index name?

0 Karma

gcusello
Legend

Hi yu94,
the easiest way is to override the index name in the set of indexers:
on transforms.conf

[overrideindex]
DEST_KEY =_MetaData:Index
REGEX = .
FORMAT = my_new_index

on props.conf

[mysourcetype]
TRANSFORMS-index = overrideindex

Bye.
Giuseppe

0 Karma

yu94
New Member

Hi Giuseppe,

We have tried this. This is not working because the data which generates is HF and we tried to override at the indexer, so on the HF it is passed the parsing queue and set the metadata file so will not be able to override at the indexer.

Any other ways of doing it?

Regards,
Thippesh

0 Karma

gcusello
Legend

Hi yu94,
Did you tried to override index on indexers? you said that you want to override index only on a set of indexers, so you can:

  • take logs using DB Connect on HF setting index1,
  • then on indexers (only the ones of the selected set), use the index override configuration.

In this way you can ingest logs on HF with the original index and then modify it on the indexers.

Bye.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Happy CX Day to our Community Superheroes!

Happy 10th Birthday CX Day!What is CX Day? It’s a global celebration recognizing innovation and success in the ...

Check out This Month’s Brand new Splunk Lantern Articles

Splunk Lantern is a customer success center providing advice from Splunk experts on valuable data insights, ...

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...