Getting Data In

Dynamically set index based on source file path

mahesh_ravji1
Explorer

Hi All,

I have log files in directory structure like this:

/var/log/data/index-a/logfile1.log

/var/log/data/index-b/logfile1.log

/var/log/data/index-c/logfile1.log

I want to dynamically set the index to the 4th element of the source path (i.e. index-a, index-b or index-c).

I have configured the following settings:

$SPLUNK_HOME/etc/system/local/inputs.conf

[monitor:///var/log]

disabled = false

followTail = 0

_blacklist = .(gz|zip|bkz|arch|trc)$

$SPLUNK_HOME/etc/system/local/props.conf

[source::.../var/log/data/.*]

TRANSFORMS-index = override-index

$SPLUNK_HOME/etc/system/local/transforms.conf

[override-index]

SOURCE_KEY = MetaData:Source

REGEX = /var/log/data/([^/]+)

FORMAT = $1

DEST_KEY = _MetaData:Index

However all log files are directed default main index.

Tags (1)
0 Karma

kheli
Path Finder

have you tried to change the regex in the source stanza to something like this [source::.../var/log/data/...]

mahesh_ravji1
Explorer

That seems to have resolved by issue. Thanks every much.

0 Karma
Get Updates on the Splunk Community!

From Alert to Resolution: How Splunk Observability Helps SREs Navigate Critical ...

It's 3:17 AM, and your phone buzzes with an urgent alert. Wire transfer processing times have spiked, and ...

ATTENTION!! We’re MOVING (not really)

Hey, all! In an effort to keep this Slack workspace secure and also to make our new members' experience easy, ...

Splunk Admins: Build a Smarter Stack with These Must-See .conf25 Sessions

  Whether you're running a complex Splunk deployment or just getting your bearings as a new admin, .conf25 ...