Getting Data In

Why is renaming an index via transforms.conf and props.conf failing?

areeter
Explorer

Hello.

I really hope someone on here will be able to help me out. Long story short: I am having some difficulties renaming an index on some cooked data that is hitting my indexer with transforms.conf and props.conf. I am trying to rename it from bottles to newindex.

On the indexer, I have the following:

$SPLUNK_HOME/etc/system/local/transforms.conf:

[changeindex]
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = newindex

$SPLUNK_HOME/etc/system/local/props.conf:

[host::splunk-uf]
TRANSFORMS-index = changeindex

(For what it is worth) $SPLUNK_HOME/etc/system/local/inputs.conf:

[default]
host = splunk-indexer

[splunktcp:9997]
connection_host=none
index = newindex
compressed=true
listenOnIPv6=no

The error Splunk Web on the indexer is giving me when I send logs:

Received event for unconfigured/disabled/deleted index=bottles with source="source::/var/log/messages" host="host::splunk-uf" sourcetype="sourcetype::syslog". So far received events from 1 missing index(es). 

I have been sure to restart Splunk!

Any help would be great appreciated. Thanks!

EDIT: Some more info:

Basically, I need to be able to send data from a Universal Forwarder (UF), via a Heavy Forwarder (HWF) to two indexers. The data needs to be indexed under different indexes on each indexer. I have UF that forwards data to a HWF. The HWF forwarder does some transforms on the data to anonymize some components of it. It then forwards data to Indexer1 and Indexer2 using TCP ROUTING. I have been asked to send data to the bottles index on Indexer1 and to newindex on Indexer2. I have no control over Indexer1 hence why I have set the index to be bottles on the UF and need the HWF to do the anonymizing of data as I don't have control over Indexer1. Thus, I am trying to transform the data hitting Indexer2 to change the index name to newindex.

gcusello
Esteemed Legend

Hi areeter,
instead of [host::splunk-uf], can you use a sourcetype?
I frequently encountered this problem and solved it using sourcetype instead of host!

[your_sourcetype]
 TRANSFORMS-index = changeindex

Bye.
Giuseppe

0 Karma

areeter
Explorer

I have tried this and could not get it to work 😕 Thanks anyway!

0 Karma

somesoni2
Revered Legend

Assuming your Heavy Forwarder is Splunk Enterprise instance, these metadata override setting should be configured on Heavy Forwarder instead of Indexers (should be set on first full Splunk Enterprise instance which is capable of parsing events). So configure the same in HWF and restart it.

0 Karma

areeter
Explorer

Thanks for the response. I have updated the question (see the EDIT section at the bottom) with some more info that now has become relevant that I think makes this answer no long applicable. Any ideas?

0 Karma

somesoni2
Revered Legend

The data once cooked at HF won't be processed again at Indexer, hence the metadata name can't be changed after it's left HF. You may want to check the option suggested in following post.

https://answers.splunk.com/answers/61433/have-forwarder-duplicating-data-to-2-indexes.html

0 Karma
Get Updates on the Splunk Community!

Mission Control | Explore the latest release of Splunk Mission Control (2.3)

We’re happy to announce the release of Mission Control 2.3 which includes several new and exciting features ...

Cloud Platform | Migrating your Splunk Cloud deployment to Python 3.7

Python 2.7, the last release of Python 2, reached End of Life back on January 1, 2020. As part of our larger ...

Splunk Observability Cloud | Enhancing Your Onboarding Experience with the ...

We understand that your initial experience with getting data into Splunk Observability Cloud is crucial as it ...