Getting Data In

Dynamically set index based on source file path

mahesh_ravji1
Explorer

Hi All,

I have log files in directory structure like this:

/var/log/data/index-a/logfile1.log

/var/log/data/index-b/logfile1.log

/var/log/data/index-c/logfile1.log

I want to dynamically set the index to the 4th element of the source path (i.e. index-a, index-b or index-c).

I have configured the following settings:

$SPLUNK_HOME/etc/system/local/inputs.conf

[monitor:///var/log]

disabled = false

followTail = 0

_blacklist = .(gz|zip|bkz|arch|trc)$

$SPLUNK_HOME/etc/system/local/props.conf

[source::.../var/log/data/.*]

TRANSFORMS-index = override-index

$SPLUNK_HOME/etc/system/local/transforms.conf

[override-index]

SOURCE_KEY = MetaData:Source

REGEX = /var/log/data/([^/]+)

FORMAT = $1

DEST_KEY = _MetaData:Index

However all log files are directed default main index.

Tags (1)
0 Karma

kheli
Path Finder

have you tried to change the regex in the source stanza to something like this [source::.../var/log/data/...]

mahesh_ravji1
Explorer

That seems to have resolved by issue. Thanks every much.

0 Karma
Get Updates on the Splunk Community!

.conf25 Registration is OPEN!

Ready. Set. Splunk! Your favorite Splunk user event is back and better than ever. Get ready for more technical ...

Detecting Cross-Channel Fraud with Splunk

This article is the final installment in our three-part series exploring fraud detection techniques using ...

Splunk at Cisco Live 2025: Learning, Innovation, and a Little Bit of Mr. Brightside

Pack your bags (and maybe your dancing shoes)—Cisco Live is heading to San Diego, June 8–12, 2025, and Splunk ...