We have a series of logs from different devices such as (Firewall .waf. antivirus,...) that come from syslog server to Splunk with the same host name. I want to separate the logs based on sourcetype. All logs have the same Hostname and source.
is it possible to define different sourcetype?
All data coming from the same source makes it difficult for syslog to segregate it. Can you have the data come in on different ports? That will give syslog something to work with. Otherwise, syslog will have to parse each event to try to determine the sourcetype, which will be slow.
I understand they are not coming from different ports today, but can you change them to do so? Have some services enter port 514, others in port 1514, etc., and have syslog sort them by incoming port.
No. i can't change the port number.I have no way to change the port. is there a way i can configure the transforms.conf or props.conf files to receive logs in different sourcetype?
The problem with changing sourcetypes in props/transforms is the new sourcetype will not be processed. That is, the event will have the sourcetype=foo, but none of the field extractions, etc, associated with foo will be performed. That is because the application of props and transforms is a one-pass process.
I think the better option is to have syslog sort events based on their content into separate files or directories. Then the Splunk UF can apply the appropriate sourcetype based on the file or directory name.
you can override host and/or sourcetype reading the content of the syslogs:
usually hostname is in the beginning of each row, so you can use the process described at https://docs.splunk.com/Documentation/SplunkCloud/8.0.2003/Data/Overridedefaulthostassignments
for sourcetype, you can use https://www.splunk.com/en_us/blog/tips-and-tricks/overriding-default-syslog-host-extraction.html
Only one question: do you want a different sourcetype to identify the flow or for a different reason?
remember that knowledge object are usually related to sourcetype, so if you have more sourcetypes, you must create all the knowledge objects definition for each seorcetype.
If instead you have different definitions for each kind of log the sourcetype's override is a good idea.
no my reason is just for identifying the flow.
my setting in transforms.conf :
REGEX = FG1092
DEST_KEY = MetaData:sourcetype
FORMAT = fortigate
TRANSFORMS-sourcetype-fg = fw-sourcetype
if it's only to identify the flow, you could override host using the link I honted, so you can maintainal the knowledge objects related to the same sourcetype.
If you anyway want to override sourcetype follow the instructions in the above url:
[fw-sourcetype] REGEX = FG1092 DEST_KEY = MetaData:sourcetype FORMAT = sourcetype::fortigate