Getting Data In

Index override on heavy forwarder data

yu94
New Member

Hi,

There is situation where we have installed DB connect on HF and then the HF sends that data to 2 sets of different indexers and now we need to override the index name at one set of indexers .

We have tried to override the index which is coming from UF is working fine.

When we tried to override the index which is coming from HF (DB Connect), it was not working may be due to the metadata already set by this HF.

Can you help me to fix this issue?

Regards,
Thippesh

0 Karma

gjanders
SplunkTrust
SplunkTrust

Since this is a heavy forwarder the data is cooked on this server, therefore the index setting is done here.

Is it possible to get both sets of indexers to use the same index for this data?
If not the transforms.conf might work but you would need to do the transform based on which output was chosen, and I'm unsure how to do that part, also DB Connect can set an index, but it will only set one per input.

Perhaps you can change your indexers to have the same index name?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi yu94,
the easiest way is to override the index name in the set of indexers:
on transforms.conf

[overrideindex]
DEST_KEY =_MetaData:Index
REGEX = .
FORMAT = my_new_index

on props.conf

[mysourcetype]
TRANSFORMS-index = overrideindex

Bye.
Giuseppe

0 Karma

yu94
New Member

Hi Giuseppe,

We have tried this. This is not working because the data which generates is HF and we tried to override at the indexer, so on the HF it is passed the parsing queue and set the metadata file so will not be able to override at the indexer.

Any other ways of doing it?

Regards,
Thippesh

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi yu94,
Did you tried to override index on indexers? you said that you want to override index only on a set of indexers, so you can:

  • take logs using DB Connect on HF setting index1,
  • then on indexers (only the ones of the selected set), use the index override configuration.

In this way you can ingest logs on HF with the original index and then modify it on the indexers.

Bye.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Get ready to show some Splunk Certification swagger at .conf24!

Dive into the deep end of data by earning a Splunk Certification at .conf24. We're enticing you again this ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Now On-Demand Join us to learn more about how you can leverage Service Level Objectives (SLOs) and the new ...

Database Performance Sidebar Panel Now on APM Database Query Performance & Service ...

We’ve streamlined the troubleshooting experience for database-related service issues by adding a database ...