Getting Data In

Why is renaming an index via transforms.conf and props.conf failing?

Explorer

Hello.

I really hope someone on here will be able to help me out. Long story short: I am having some difficulties renaming an index on some cooked data that is hitting my indexer with transforms.conf and props.conf. I am trying to rename it from bottles to newindex.

On the indexer, I have the following:

$SPLUNK_HOME/etc/system/local/transforms.conf:

[changeindex]
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = newindex

$SPLUNK_HOME/etc/system/local/props.conf:

[host::splunk-uf]
TRANSFORMS-index = changeindex

(For what it is worth) $SPLUNK_HOME/etc/system/local/inputs.conf:

[default]
host = splunk-indexer

[splunktcp:9997]
connection_host=none
index = newindex
compressed=true
listenOnIPv6=no

The error Splunk Web on the indexer is giving me when I send logs:

Received event for unconfigured/disabled/deleted index=bottles with source="source::/var/log/messages" host="host::splunk-uf" sourcetype="sourcetype::syslog". So far received events from 1 missing index(es). 

I have been sure to restart Splunk!

Any help would be great appreciated. Thanks!

EDIT: Some more info:

Basically, I need to be able to send data from a Universal Forwarder (UF), via a Heavy Forwarder (HWF) to two indexers. The data needs to be indexed under different indexes on each indexer. I have UF that forwards data to a HWF. The HWF forwarder does some transforms on the data to anonymize some components of it. It then forwards data to Indexer1 and Indexer2 using TCP ROUTING. I have been asked to send data to the bottles index on Indexer1 and to newindex on Indexer2. I have no control over Indexer1 hence why I have set the index to be bottles on the UF and need the HWF to do the anonymizing of data as I don't have control over Indexer1. Thus, I am trying to transform the data hitting Indexer2 to change the index name to newindex.

Legend

Hi areeter,
instead of [host::splunk-uf], can you use a sourcetype?
I frequently encountered this problem and solved it using sourcetype instead of host!

[your_sourcetype]
 TRANSFORMS-index = changeindex

Bye.
Giuseppe

0 Karma

Explorer

I have tried this and could not get it to work 😕 Thanks anyway!

0 Karma

SplunkTrust
SplunkTrust

Assuming your Heavy Forwarder is Splunk Enterprise instance, these metadata override setting should be configured on Heavy Forwarder instead of Indexers (should be set on first full Splunk Enterprise instance which is capable of parsing events). So configure the same in HWF and restart it.

0 Karma

Explorer

Thanks for the response. I have updated the question (see the EDIT section at the bottom) with some more info that now has become relevant that I think makes this answer no long applicable. Any ideas?

0 Karma

SplunkTrust
SplunkTrust

The data once cooked at HF won't be processed again at Indexer, hence the metadata name can't be changed after it's left HF. You may want to check the option suggested in following post.

https://answers.splunk.com/answers/61433/have-forwarder-duplicating-data-to-2-indexes.html

0 Karma