Getting Data In

Dynamically set index based on source file path

mahesh_ravji1
Explorer

Hi All,

I have log files in directory structure like this:

/var/log/data/index-a/logfile1.log

/var/log/data/index-b/logfile1.log

/var/log/data/index-c/logfile1.log

I want to dynamically set the index to the 4th element of the source path (i.e. index-a, index-b or index-c).

I have configured the following settings:

$SPLUNK_HOME/etc/system/local/inputs.conf

[monitor:///var/log]

disabled = false

followTail = 0

_blacklist = .(gz|zip|bkz|arch|trc)$

$SPLUNK_HOME/etc/system/local/props.conf

[source::.../var/log/data/.*]

TRANSFORMS-index = override-index

$SPLUNK_HOME/etc/system/local/transforms.conf

[override-index]

SOURCE_KEY = MetaData:Source

REGEX = /var/log/data/([^/]+)

FORMAT = $1

DEST_KEY = _MetaData:Index

However all log files are directed default main index.

Tags (1)
0 Karma

kheli
Path Finder

have you tried to change the regex in the source stanza to something like this [source::.../var/log/data/...]

mahesh_ravji1
Explorer

That seems to have resolved by issue. Thanks every much.

0 Karma
Get Updates on the Splunk Community!

The OpenTelemetry Certified Associate (OTCA) Exam

What’s this OTCA exam? The Linux Foundation offers the OpenTelemetry Certified Associate (OTCA) credential to ...

From Manual to Agentic: Level Up Your SOC at Cisco Live

Welcome to the Era of the Agentic SOC   Are you tired of being a manual alert responder? The security ...

Splunk Classroom Chronicles: Training Tales and Testimonials (Episode 4)

Welcome back to Splunk Classroom Chronicles, our ongoing series where we shine a light on what really happens ...