Getting Data In

Can data have more then one source type?

stefanlasiewski
Contributor

Our Splunk server receives data through syslog, and all data is tagged with 'sourcetype=syslog'.

I am interested in using the Splunk App for Web Intelligence to parse data from several dozen different Apache webservers. My weblogs are sent to Splunk, but they are sent via syslog and thus Splunk automatically assigns the 'syslog' sourcetype to this data. The Splunk App for Web Intelligence doesn't work with 'sourcetype=syslog'. I need to assign a new kind of sourcetype to this data.

Can data belong to multiple kinds of sourcetypes? If I define the Sourcetype like 'access_combined' for my Apache HTTP logs, will it still belong to the 'syslog' Sourcetype?

Tags (2)
0 Karma
1 Solution

Damien_Dallimor
Ultra Champion

You can perform a sourcetype transform on the data coming in to your syslog input.

So you can identify particular messages by host or a regex pattern on the message contents etc.. and dynamically update the sourcetype field and any other meta fields.

inputs.conf

[udp://514]
connection_host = dns
sourcetype = syslog
disabled = false

props.conf

[syslog]
TRANSFORMS-web = set_web_sourcetype

transforms.conf

[set_web_sourcetype]
REGEX = (.+Some Regex Pattern.+)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::my_new_sourcetype

Alternatively ,you could just set up a dedicated syslog input for your webserver data.

inputs.conf

[udp://5678]
connection_host = dns
sourcetype = my_custom_sourcetype
disabled = false
index = my_index

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

Data may only have a single sourcetype. However, if you define a new sourcetype, you can set it to perform multiple transforms and field extractions that would apply to, say, both syslog and to access_combined, but looking at and duplicating the configurations of each in props.conf.

Damien_Dallimor
Ultra Champion

You can perform a sourcetype transform on the data coming in to your syslog input.

So you can identify particular messages by host or a regex pattern on the message contents etc.. and dynamically update the sourcetype field and any other meta fields.

inputs.conf

[udp://514]
connection_host = dns
sourcetype = syslog
disabled = false

props.conf

[syslog]
TRANSFORMS-web = set_web_sourcetype

transforms.conf

[set_web_sourcetype]
REGEX = (.+Some Regex Pattern.+)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::my_new_sourcetype

Alternatively ,you could just set up a dedicated syslog input for your webserver data.

inputs.conf

[udp://5678]
connection_host = dns
sourcetype = my_custom_sourcetype
disabled = false
index = my_index
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...