Getting Data In

Different sourcetype with one hostname

khanlarloo
Explorer

We have a series of logs from different devices such as (Firewall .waf. antivirus,...) that come from syslog server to Splunk with the same host name. I want to separate the logs based on sourcetype. All logs have the same Hostname and source.
is it possible to define different sourcetype?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @khanlarloo,
you can override host and/or sourcetype reading the content of the syslogs:
usually hostname is in the beginning of each row, so you can use the process described at https://docs.splunk.com/Documentation/SplunkCloud/8.0.2003/Data/Overridedefaulthostassignments
for sourcetype, you can use https://www.splunk.com/en_us/blog/tips-and-tricks/overriding-default-syslog-host-extraction.html

Only one question: do you want a different sourcetype to identify the flow or for a different reason?
remember that knowledge object are usually related to sourcetype, so if you have more sourcetypes, you must create all the knowledge objects definition for each seorcetype.
If instead you have different definitions for each kind of log the sourcetype's override is a good idea.

Ciao.
Giuseppe

0 Karma

khanlarloo
Explorer

no my reason is just for identifying the flow.
my setting in transforms.conf :
[fw-sourcetype]
REGEX = FG1092
DEST_KEY = MetaData:sourcetype
FORMAT = fortigate

in props.conf
[host::192.168.x.x]
TRANSFORMS-sourcetype-fg = fw-sourcetype

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @khanlarloo,
if it's only to identify the flow, you could override host using the link I honted, so you can maintainal the knowledge objects related to the same sourcetype.
If you anyway want to override sourcetype follow the instructions in the above url:

[fw-sourcetype]
REGEX = FG1092
DEST_KEY = MetaData:sourcetype
FORMAT = sourcetype::fortigate

Ciao.
Giuseppe

0 Karma

richgalloway
SplunkTrust
SplunkTrust

All data coming from the same source makes it difficult for syslog to segregate it. Can you have the data come in on different ports? That will give syslog something to work with. Otherwise, syslog will have to parse each event to try to determine the sourcetype, which will be slow.

---
If this reply helps you, Karma would be appreciated.
0 Karma

khanlarloo
Explorer

no they don't come from different port.(hostname,Source and sourcetype) are the same.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

I understand they are not coming from different ports today, but can you change them to do so? Have some services enter port 514, others in port 1514, etc., and have syslog sort them by incoming port.

---
If this reply helps you, Karma would be appreciated.
0 Karma

khanlarloo
Explorer

No. i can't change the port number.I have no way to change the port. is there a way i can configure the transforms.conf or props.conf files to receive logs in different sourcetype?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The problem with changing sourcetypes in props/transforms is the new sourcetype will not be processed. That is, the event will have the sourcetype=foo, but none of the field extractions, etc, associated with foo will be performed. That is because the application of props and transforms is a one-pass process.
I think the better option is to have syslog sort events based on their content into separate files or directories. Then the Splunk UF can apply the appropriate sourcetype based on the file or directory name.

---
If this reply helps you, Karma would be appreciated.
0 Karma

khanlarloo
Explorer

thank you for your help.

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...