All Apps and Add-ons

Log Segregation ?


Hi ,

I am having a log file , lets say Splunk.log which contains two type of logs written in it consider "internal" and "external"
Is there any way to segregate this in inputs.conf while motoring the log file.
The purpose is that i need to assign different sourcetype to each of them.

Also, is this possible to be done at in rsyslog which is acting as my centralized syslog server.
I have done the dynamic configuration of seperating it by hostname but is there any way to seperate by any message in the log incoming?


0 Karma

Ultra Champion
0 Karma

Splunk Employee
Splunk Employee

Splunk doesnt have a default capability to read logs and assign different sourcetypes based on inputs.conf. What you could do is use props / transforms to Split the Logs to different sourcetypes at the parsing level.

At a high level, you ingest the file as "SourcetypeA". Then based on regex rules defined, you can re-categorize the files as "Sourcetype1" or "Sourcetype2".

You do need to be aware of where in your Splunk environment this needs to happen and the ramifications of doing this when you have high volumes of data and indexers that are resource quenched..

See this article :

It outlines how to do this quite accurately on how to rewrite the sourcetype, again, based on a regex.

You need to also be aware that this focused more on single line events with a specific pattern occurring at the beginning of the line. If your events are multiline, this becomes much more difficult and compute intensive. Preprocessing is recommended in that case..

Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...