Getting Data In

Dynamically set index based on source file path

mahesh_ravji1
Explorer

Hi All,

I have log files in directory structure like this:

/var/log/data/index-a/logfile1.log

/var/log/data/index-b/logfile1.log

/var/log/data/index-c/logfile1.log

I want to dynamically set the index to the 4th element of the source path (i.e. index-a, index-b or index-c).

I have configured the following settings:

$SPLUNK_HOME/etc/system/local/inputs.conf

[monitor:///var/log]

disabled = false

followTail = 0

_blacklist = .(gz|zip|bkz|arch|trc)$

$SPLUNK_HOME/etc/system/local/props.conf

[source::.../var/log/data/.*]

TRANSFORMS-index = override-index

$SPLUNK_HOME/etc/system/local/transforms.conf

[override-index]

SOURCE_KEY = MetaData:Source

REGEX = /var/log/data/([^/]+)

FORMAT = $1

DEST_KEY = _MetaData:Index

However all log files are directed default main index.

Tags (1)
0 Karma

kheli
Path Finder

have you tried to change the regex in the source stanza to something like this [source::.../var/log/data/...]

mahesh_ravji1
Explorer

That seems to have resolved by issue. Thanks every much.

0 Karma
Get Updates on the Splunk Community!

Updated Data Type Articles, Anniversary Celebrations, and More on Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

A Prelude to .conf25: Your Guide to Splunk University

Heading to Boston this September for .conf25? Get a jumpstart by arriving a few days early for Splunk ...

4 Ways the Splunk Community Helps You Prepare for .conf25

.conf25 is right around the corner, and whether you’re a first-time attendee or a seasoned Splunker, the ...