Splunk Search

index time transform covering multiple destination indexes

juniormint
Communicator

I would like filter certain known data events into three different indexes (possibly more in the future).

Events have an embedded field called "AppName". The possible values happen to be the names of my indexes.

Is this the right approach? Does it look correct?
in transforms.conf
[RedirectToAppIndex]
REGEX = .*AppName="(?i)(app1|app2|app3)"
DEST_KEY = _MetaData:Index
FORMAT = $1

0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

This looks ok.
make sure that the indexes app1 app2 and app3 exists with the proper case.

View solution in original post

yannK
Splunk Employee
Splunk Employee

This looks ok.
make sure that the indexes app1 app2 and app3 exists with the proper case.

yannK
Splunk Employee
Splunk Employee

yes, each event has to be compared to the regex and routed, it adds load to the indexing.

0 Karma

juniormint
Communicator

By that you just mean...index time transforms take time and resources...add complexity, etc. Right?

0 Karma

yannK
Splunk Employee
Splunk Employee

But the best solution is still to specify the index at the input level.
If you have distinct inputs per sources of course.

0 Karma

juniormint
Communicator

Thanks for calling out that they need to exist. This transform is actually part of my strategy for dealing with indexes that logically should exist but haven't yet been created for whatever reason.

All my events from a group of machines have an initial default index defined in the index meta data field (think catchall). The log events can include a preference for destination index by specifying AppName, but if it doesn't exist the log events fall back on the default index.

0 Karma

juniormint
Communicator

My goal on the matching side was to have the "(?i)" make the regex case-insensitive.

On the index name side, my experience has suggested that the ingestion does not care about case (all my indexes show up in the manager as lower case and events coming in as App1 or aPP1 all make it into app1).

Does that not line up with your experience/understanding?

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...