Getting Data In

Can data have more then one source type?

stefanlasiewski
Contributor

Our Splunk server receives data through syslog, and all data is tagged with 'sourcetype=syslog'.

I am interested in using the Splunk App for Web Intelligence to parse data from several dozen different Apache webservers. My weblogs are sent to Splunk, but they are sent via syslog and thus Splunk automatically assigns the 'syslog' sourcetype to this data. The Splunk App for Web Intelligence doesn't work with 'sourcetype=syslog'. I need to assign a new kind of sourcetype to this data.

Can data belong to multiple kinds of sourcetypes? If I define the Sourcetype like 'access_combined' for my Apache HTTP logs, will it still belong to the 'syslog' Sourcetype?

Tags (2)
0 Karma
1 Solution

Damien_Dallimor
Ultra Champion

You can perform a sourcetype transform on the data coming in to your syslog input.

So you can identify particular messages by host or a regex pattern on the message contents etc.. and dynamically update the sourcetype field and any other meta fields.

inputs.conf

[udp://514]
connection_host = dns
sourcetype = syslog
disabled = false

props.conf

[syslog]
TRANSFORMS-web = set_web_sourcetype

transforms.conf

[set_web_sourcetype]
REGEX = (.+Some Regex Pattern.+)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::my_new_sourcetype

Alternatively ,you could just set up a dedicated syslog input for your webserver data.

inputs.conf

[udp://5678]
connection_host = dns
sourcetype = my_custom_sourcetype
disabled = false
index = my_index

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

Data may only have a single sourcetype. However, if you define a new sourcetype, you can set it to perform multiple transforms and field extractions that would apply to, say, both syslog and to access_combined, but looking at and duplicating the configurations of each in props.conf.

Damien_Dallimor
Ultra Champion

You can perform a sourcetype transform on the data coming in to your syslog input.

So you can identify particular messages by host or a regex pattern on the message contents etc.. and dynamically update the sourcetype field and any other meta fields.

inputs.conf

[udp://514]
connection_host = dns
sourcetype = syslog
disabled = false

props.conf

[syslog]
TRANSFORMS-web = set_web_sourcetype

transforms.conf

[set_web_sourcetype]
REGEX = (.+Some Regex Pattern.+)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::my_new_sourcetype

Alternatively ,you could just set up a dedicated syslog input for your webserver data.

inputs.conf

[udp://5678]
connection_host = dns
sourcetype = my_custom_sourcetype
disabled = false
index = my_index
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>