Log Segregation ?

[himapate]

Hi ,

I am having a log file , lets say Splunk.log which contains two type of logs written in it consider "internal" and "external"
Is there any way to segregate this in inputs.conf while motoring the log file.
The purpose is that i need to assign different sourcetype to each of them.

Also, is this possible to be done at in rsyslog which is acting as my centralized syslog server.
I have done the dynamic configuration of seperating it by hostname but is there any way to seperate by any message in the log incoming?

Thankyou

[ddrillic]

It's covered in Override source types on a per-event basis

[esix_splunk]

Splunk doesnt have a default capability to read logs and assign different sourcetypes based on inputs.conf. What you could do is use props / transforms to Split the Logs to different sourcetypes at the parsing level.

At a high level, you ingest the file as "SourcetypeA". Then based on regex rules defined, you can re-categorize the files as "Sourcetype1" or "Sourcetype2".

You do need to be aware of where in your Splunk environment this needs to happen and the ramifications of doing this when you have high volumes of data and indexers that are resource quenched..

See this article : https://answers.splunk.com/answers/221294/how-to-split-a-log-into-multiple-sourcetypes-on-a.html

It outlines how to do this quite accurately on how to rewrite the sourcetype, again, based on a regex.

You need to also be aware that this focused more on single line events with a specific pattern occurring at the beginning of the line. If your events are multiline, this becomes much more difficult and compute intensive. Preprocessing is recommended in that case..