Getting Data In

Dynamically set index based on source file path

mahesh_ravji1
Explorer

Hi All,

I have log files in directory structure like this:

/var/log/data/index-a/logfile1.log

/var/log/data/index-b/logfile1.log

/var/log/data/index-c/logfile1.log

I want to dynamically set the index to the 4th element of the source path (i.e. index-a, index-b or index-c).

I have configured the following settings:

$SPLUNK_HOME/etc/system/local/inputs.conf

[monitor:///var/log]

disabled = false

followTail = 0

_blacklist = .(gz|zip|bkz|arch|trc)$

$SPLUNK_HOME/etc/system/local/props.conf

[source::.../var/log/data/.*]

TRANSFORMS-index = override-index

$SPLUNK_HOME/etc/system/local/transforms.conf

[override-index]

SOURCE_KEY = MetaData:Source

REGEX = /var/log/data/([^/]+)

FORMAT = $1

DEST_KEY = _MetaData:Index

However all log files are directed default main index.

Tags (1)
0 Karma

kheli
Path Finder

have you tried to change the regex in the source stanza to something like this [source::.../var/log/data/...]

mahesh_ravji1
Explorer

That seems to have resolved by issue. Thanks every much.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Classic British Rock Bands

This puzzle series (first published here) is based on extracting information from statements about well-known ...

New to the Community? Join Us for an Exclusive Post .conf26 Welcome Session (Plus a ...

Keep the .conf26 Energy Going: Welcome to the Splunk Community! Whether we scanned your badge at the Community ...

Accelerate Splunk App Development with AI

  Learn how to automate the repetitive parts of your build cycle.   Register to Attend   Join our Tech Talk ...