Splunk Search

Using a transform to set index from event field. How to fall back to a certain index?


I have events with a field that contains a desired destination index (see index=* below).

[timestamp] index=layer1 message="123456"
[timestamp] index=layer2 message="123456"
[timestamp] index=layer3 message="123456"
[timestamp] index=jumbled message="123456"
[timestamp] index=layer1 message="123456"
[timestamp] index=layer1 message="123456"
[timestamp] index=layer2 message="123456"

I'm currently using a transform like the following, and it works quite well to get events into their correct index.

REGEX = .*index="(.*?)"
DEST_KEY = _MetaData:Index

At least most of the time. Occasionally an event arrives whose index field names an index which doesn't exist (either because the name is wrong or I forgot to create the index). In this case the above transform appears to simply generate an error in the internal index and otherwise drop the data.

Is there another way to do this that would drop messages which failed match an existing index into some catch all index of my choosing?

Something like FORMAT if ($1).exists $1 else myDefaultCatchAllIndex

Tags (3)