Getting Data In

How to modify the source field to match normal splunk file location

coleman07
Path Finder

We had to put the log files in the /san/splunk/var/log/splunk directory vs the /opt/splunk/var/log/splunk directory. Of course, I can modify apps that have hard coded the path to files in that directory. Is there a way to use props or transform on the indexer with the awkward path so the source will show up as /opt/splunk/var/log/splunk on the search head? That way, I don't have to modify the search in apps needing to find files in the /san/splunk/var/log/splunk directory.

0 Karma

somesoni2
Revered Legend

Try this in your Indexers and Heavy Forwarders and Search Heads (wherever you're collecting internal logs with this different path

props.conf 

[source::/san/splunk/var/log/splunk*]
TRANSFORMS-setsource = correct_source


transforms.conf

[correct_source]
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Source
REGEX = \/san\/splunk(\/var\/log\/splunk.*)
FORMAT = source::/opt/splunk$1

landen99
Motivator

I think that this regex would only match sources that began with the parent directory /san/ . Perhaps the regex could be relaxed to only match on the capture group:

 [correct_source]
 SOURCE_KEY = MetaData:Source
 DEST_KEY = MetaData:Source
 REGEX = (\/var\/log\/splunk.*)
 FORMAT = source::$1
0 Karma

landen99
Motivator

You could hard code the source in your inputs.conf on your indexer or forwarder, http://docs.splunk.com/Documentation/Splunk/6.3.3/Admin/inputsconf

source= /opt/splunk/var/log/splunk/filename
0 Karma

jkat54
SplunkTrust
SplunkTrust

Change your search to look for */splunk/var/log...

Or use sedcmd in props.conf on your sourcetype

 sedcmd-santoopt = "s/\/san/\/opt/g"

You can also use sed with the rex command in search:

 ... | rex mode=sed "s/\/san/\/opt/g" | ...
0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Rounding off the Splunk Dashboard Contest

What does a contest-winning Splunk dashboard look like? In this case, it isn't in a browser tab at all. It ...

A Four Part Event Series: AI + Observability: AI Agents, LLMs, Apps, & Infrastructure

AI + Observability: AI Agents, LLMs, Apps, & Infrastructure The rapid evolution of artificial intelligence ...