Our Splunk server receives data through syslog, and all data is tagged with 'sourcetype=syslog'.
I am interested in using the Splunk App for Web Intelligence to parse data from several dozen different Apache webservers. My weblogs are sent to Splunk, but they are sent via syslog and thus Splunk automatically assigns the 'syslog' sourcetype to this data. The Splunk App for Web Intelligence doesn't work with 'sourcetype=syslog'. I need to assign a new kind of sourcetype to this data.
Can data belong to multiple kinds of sourcetypes? If I define the Sourcetype like 'access_combined' for my Apache HTTP logs, will it still belong to the 'syslog' Sourcetype?
You can perform a sourcetype transform on the data coming in to your syslog input.
So you can identify particular messages by host or a regex pattern on the message contents etc.. and dynamically update the sourcetype field and any other meta fields.
inputs.conf
[udp://514]
connection_host = dns
sourcetype = syslog
disabled = false
props.conf
[syslog]
TRANSFORMS-web = set_web_sourcetype
transforms.conf
[set_web_sourcetype]
REGEX = (.+Some Regex Pattern.+)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::my_new_sourcetype
Alternatively ,you could just set up a dedicated syslog input for your webserver data.
inputs.conf
[udp://5678]
connection_host = dns
sourcetype = my_custom_sourcetype
disabled = false
index = my_index
Data may only have a single sourcetype. However, if you define a new sourcetype, you can set it to perform multiple transforms and field extractions that would apply to, say, both syslog and to access_combined, but looking at and duplicating the configurations of each in props.conf.
You can perform a sourcetype transform on the data coming in to your syslog input.
So you can identify particular messages by host or a regex pattern on the message contents etc.. and dynamically update the sourcetype field and any other meta fields.
inputs.conf
[udp://514]
connection_host = dns
sourcetype = syslog
disabled = false
props.conf
[syslog]
TRANSFORMS-web = set_web_sourcetype
transforms.conf
[set_web_sourcetype]
REGEX = (.+Some Regex Pattern.+)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::my_new_sourcetype
Alternatively ,you could just set up a dedicated syslog input for your webserver data.
inputs.conf
[udp://5678]
connection_host = dns
sourcetype = my_custom_sourcetype
disabled = false
index = my_index