topic Re: Different sourcetype with one hostname in Getting Data In

Different sourcetype with one hostname

khanlarloo — Mon, 11 May 2020 10:52:30 GMT

We have a series of logs from different devices such as (Firewall .waf. antivirus,...) that come from syslog server to Splunk with the same host name. I want to separate the logs based on sourcetype. All logs have the same Hostname and source.
is it possible to define different sourcetype?

Re: Different sourcetype with one hostname

richgalloway — Mon, 11 May 2020 14:11:18 GMT

All data coming from the same source makes it difficult for syslog to segregate it. Can you have the data come in on different ports? That will give syslog something to work with. Otherwise, syslog will have to parse each event to try to determine the sourcetype, which will be slow.

Re: Different sourcetype with one hostname

khanlarloo — Tue, 12 May 2020 02:53:26 GMT

no they don't come from different port.(hostname,Source and sourcetype) are the same.

Re: Different sourcetype with one hostname

richgalloway — Tue, 12 May 2020 12:43:43 GMT

I understand they are not coming from different ports today, but can you change them to do so? Have some services enter port 514, others in port 1514, etc., and have syslog sort them by incoming port.

Re: Different sourcetype with one hostname

khanlarloo — Wed, 13 May 2020 04:10:51 GMT

No. i can't change the port number.I have no way to change the port. is there a way i can configure the transforms.conf or props.conf files to receive logs in different sourcetype?

Re: Different sourcetype with one hostname

richgalloway — Wed, 13 May 2020 12:55:41 GMT

The problem with changing sourcetypes in props/transforms is the new sourcetype will not be processed. That is, the event will have the sourcetype=foo, but none of the field extractions, etc, associated with foo will be performed. That is because the application of props and transforms is a one-pass process.
I think the better option is to have syslog sort events based on their content into separate files or directories. Then the Splunk UF can apply the appropriate sourcetype based on the file or directory name.

Re: Different sourcetype with one hostname

khanlarloo — Sat, 16 May 2020 03:12:31 GMT

thank you for your help.

Re: Different sourcetype with one hostname

gcusello — Sun, 17 May 2020 09:44:41 GMT

Hi @khanlarloo,
you can override host and/or sourcetype reading the content of the syslogs:
usually hostname is in the beginning of each row, so you can use the process described at https://docs.splunk.com/Documentation/SplunkCloud/8.0.2003/Data/Overridedefaulthostassignments
for sourcetype, you can use https://www.splunk.com/en_us/blog/tips-and-tricks/overriding-default-syslog-host-extraction.html

Only one question: do you want a different sourcetype to identify the flow or for a different reason?
remember that knowledge object are usually related to sourcetype, so if you have more sourcetypes, you must create all the knowledge objects definition for each seorcetype.
If instead you have different definitions for each kind of log the sourcetype's override is a good idea.

Ciao.
Giuseppe

Re: Different sourcetype with one hostname

khanlarloo — Tue, 19 May 2020 08:04:51 GMT

no my reason is just for identifying the flow.
my setting in transforms.conf :
[fw-sourcetype]
REGEX = FG1092
DEST_KEY = MetaData:sourcetype
FORMAT = fortigate

in props.conf
[host::192.168.x.x]
TRANSFORMS-sourcetype-fg = fw-sourcetype

Re: Different sourcetype with one hostname

gcusello — Tue, 19 May 2020 12:25:57 GMT

Hi @khanlarloo,
if it's only to identify the flow, you could override host using the link I honted, so you can maintainal the knowledge objects related to the same sourcetype.
If you anyway want to override sourcetype follow the instructions in the above url:

[fw-sourcetype]
REGEX = FG1092
DEST_KEY = MetaData:sourcetype
FORMAT = sourcetype::fortigate

Ciao.
Giuseppe