topic Re: Alternative ways to assigning sourcetype? in Getting Data In

Alternative ways to assigning sourcetype?

dleung — Thu, 09 Sep 2010 01:06:34 GMT

I am checking out a sample application where an eventtype's search contains "sourcetype=..." . I having difficulty determining where this particular sourcetype gets assigned.

I would typically look for an entry in inputs.conf that may explicitly set the sourcetype for a given input, however, there is no inputs.conf

I do notice within props.conf there's a stanza for the sourcetype's field extractions. Does a stanza within props.conf implicitly declare and define the sourcetype?

For the following example, does the applicability to the REPORT clause associate the event to the sourcetype?

This is a working solution and I am interested in trying to understand how this works rather than alternative modifications.

(I've included sample as well as actual declarations)

=====================
eventtypes.conf
=====================
[eventtype_foo]
search = sourcetype=bar

[asa-authentication-failure]
search = sourcetype=cisco_asa "Message-Type=Authen failed"


=====================
props.conf
=====================
[bar]
REPORT-bar = bar-eventinfo

[cisco_asa]
REPORT-asa = ciscosyslog-eventinfo

=====================
transforms.conf
=====================
[bar-eventinfo]
REGEX = ^foobar-(\w+)-$
FORMAT = foobar_type::$1

[ciscosyslog-eventinfo]
REGEX = [^%]+%(\w+)-(\d)-(\d+):\s+.*
FORMAT = dvc_type::$1 log_level::$2 signature_id::$3

Thanks, Danny

Re: Alternative ways to assigning sourcetype?

gkanapathy — Thu, 09 Sep 2010 01:24:12 GMT

Sourcetypes may be set at index time via:

setting it in inputs.conf (on the machine where the input is configured)
setting it in a matching stanza in props.conf (only on the machine where the input is configred)
setting it via an index-time TRANSFORM in props.conf and transforms.conf (on the machine where the parse queue executes, which is either the heavy forwarder if one is used, or the indexer)
auto-generation if it's unspecified otherwise or if CHECK_FOR_HEADER tells it to use CSV headers and assign a new generated sourcetype.

A sourcetype can also be overridden in search-time configurations with the rename setting in props.conf, or with REPORT/EXTRACT extractions (all on the search head).

Re: Alternative ways to assigning sourcetype?

Lowell — Thu, 09 Sep 2010 01:26:26 GMT

Do you have a cisco app installed?

In general, a sourcetype can be determined by setting up a source pattern that sets the sourcetype. in props.conf. Or it can be set explicitly by inputs.conf. There are a few other methods, like [rule::...] and [delayedrule::...] and if all else fails then splunk will assign a new sourcetype which will often be the some portion of the source name; in which case you will find entries about it in your "learned" app in the sourcetypes.conf file.

Re: Alternative ways to assigning sourcetype?

hulahoop — Sat, 11 Sep 2010 12:33:16 GMT

Hi Danny,

To address some of your questions directly...

"Does a stanza within props.conf implicitly declare and define the sourcetype?"
--> The answer is No. Just because a sourcetype is referenced by a stanza in props.conf, this does not automatically create the sourcetype and associate it with any events.

"For the following example, does the applicability to the REPORT clause associate the event to the sourcetype?"
--> The answer is also No. REPORT signifies a search-time operation, and also does not create or associate anything to the sourcetype simply because it is referenced.

Are you looking at the Splunk for Cisco Security App? The cisco_asa sourcetype seems to be referenced in a number of places in the conf files, but I don't see that any events are ever assigned to this sourcetype. There are some rules that reference it, but I don't believe any of the rules ever take effect since no events actually get sourcetyped as cisco_asa. Even the sample cisco_asa.log gets sourcetypes as cisco_firewall. Admittedly, this is confusing.

The answer provided by gkanapathy covers all the cases for setting and manipulating sourcetype.

Re: Alternative ways to assigning sourcetype?

dleung — Wed, 15 Sep 2010 02:54:04 GMT

hulahoop,

The information shown is actually from the SKB-Cisco module included in ESS. The extractions are very similar to the cisco firewall addon module. I did a little further digging and found there were some sourcetypes set via an index-time TRANSFORM in props.conf and transforms.conf. Thanks to gkanapathy for pointing that out 🙂 Additionally, I followed-up with the developer and found that there would also be some manual setting of sourcetypes at the configuration of the data inputs. Thanks for the detailed help and explanation.