All Apps and Add-ons

Log Segregation ?


Hi ,

I am having a log file , lets say Splunk.log which contains two type of logs written in it consider "internal" and "external"
Is there any way to segregate this in inputs.conf while motoring the log file.
The purpose is that i need to assign different sourcetype to each of them.

Also, is this possible to be done at in rsyslog which is acting as my centralized syslog server.
I have done the dynamic configuration of seperating it by hostname but is there any way to seperate by any message in the log incoming?


0 Karma

Ultra Champion
0 Karma

Splunk Employee
Splunk Employee

Splunk doesnt have a default capability to read logs and assign different sourcetypes based on inputs.conf. What you could do is use props / transforms to Split the Logs to different sourcetypes at the parsing level.

At a high level, you ingest the file as "SourcetypeA". Then based on regex rules defined, you can re-categorize the files as "Sourcetype1" or "Sourcetype2".

You do need to be aware of where in your Splunk environment this needs to happen and the ramifications of doing this when you have high volumes of data and indexers that are resource quenched..

See this article :

It outlines how to do this quite accurately on how to rewrite the sourcetype, again, based on a regex.

You need to also be aware that this focused more on single line events with a specific pattern occurring at the beginning of the line. If your events are multiline, this becomes much more difficult and compute intensive. Preprocessing is recommended in that case..

Get Updates on the Splunk Community!

How I Instrumented a Rust Application Without Knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...