How to seperate different Sourcetype logs from single syslog IP source based on Regex

[NightShark]

Greetings,

I am trying to get different log types such as security and audit logs for example from a single IP source from my HF instance, how exactly should I be settings my settings in Inputs, Transforms and Props conf in my HF to accomplish this?

Thanks,

[gcusello · Accepted solution]

Hi @NightShark,

you have to override sourcetypes on event basis.

You can find a description of this job at https://docs.splunk.com/Documentation/Splunk/latest/Data/Advancedsourcetypeoverrides#:~:text=You%20c....

In few words, you have to identify the regexes for each destination sourcetype and then put in your Indexers or (if present) in your Heavy Forwarders:

props.conf

[your_sourcetype]
TRANSFORMS-override_sourcetype = override_sourcetype_1, override_sourcetype_2

transforms.conf

[override_sourcetype_1]
REGEX = <your_regex_1>
FORMAT = sourcetype::<your_custom_sourcetype_value>
DEST_KEY = MetaData:Sourcetype

[override_sourcetype_2]
REGEX = <your_regex_2>
FORMAT = sourcetype::<your_custom_sourcetype_value>
DEST_KEY = MetaData:Sourcetype

 In this example I use two destination sourcetypes.

Ciao.

Giuseppe

[youngsuh]

@gcusello 

Could you review mind?  Here is my inputs.conf

[monitor:///var/log/$mask_host1$/syslog]
disabled = false
#initCrcLength = 800
crcSalt = <SOURCE>
index = test
sourcetype = uag:syslog

[monitor:///var/log/$mask_host2$/syslog]
disabled = false
#initCrcLength = 800
crcSalt = <SOURCE>
index = test
sourcetype = uag:syslog

[monitor:///var/log/$mask_host3/syslog]
disabled = false
#initCrcLength = 800
crcSalt = <SOURCE>
index = test
sourcetype = uag:syslog

Here is props.conf on the HF.

[uag:syslog]
NO_BINARY_CHECK = true
category = Custom
pulldown_type = 1
TRANSFORMS-sourcetype = vmware:uag:admin, vmware:uag:audit, vmware:uag:esmanager

Here is my transform.conf on the HF.

[uag:syslog]
REGEX = :\d\d\s+\w{5}\w{4}\suag-admin\:
FORMAT = sourcetype::vmware:uag:admin
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Sourcetype

[uag:syslog]
REGEX = :\d\d\s+\w{5}\w{4}\suag-audit\:
FORMAT = sourcetype::vmware:uag:admin
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Sourcetype

[uag:syslog]
REGEX = :\d\d\s+\w{5}\w{4}\suag-esmanager\:
FORMAT = sourcetype::vmware:uag:esmanager
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Sourcetype

Is there anything I am doing wrong in the configuration?

[gcusello]

Hi @youngsuh,

please don't attach a new question to an answered one: creating a new one you'll have faster and probably better answers!

Anyway, what do you want to match with your regex: the source or the log content?

from your props.conf it seems that you want to match with the source field, but in this case the regex is wrong.

If you want to match with the content of the log delete the "SOURCE_KEY" row.

Ciao.

Giuseppe

[youngsuh]

Will create a new post next time.  I want to match log content.  So, I delete 

SOURCE_KEY = MetaData:Source

you meant transform.conf right?  not props.conf

The props.conf should be change to match sourcetype right?

TRANSFORMS-uag:syslog = vmware:uag:admin, vmware:uag:audit, vmware:uag:esmanager

[NightShark]

Greetings,

Thank you very much gcusello,