topic Re: source:: rule in props.conf ignored? in Getting Data In

source:: rule in props.conf ignored?

sowings — Mon, 17 Dec 2012 18:43:49 GMT

I have an inputs.conf that looks like this:

[monitor:///syslog/.../*.log]
host_segment = 4
sourcetype = syslog
ignoreOlderThan = 5d
blacklist = \.gz$

I use transforms to remap a lot of the events from the 'syslog' sourcetype into other types, as appropriate. There are a couple of hosts (with logs in a host-specific subdirectory) which emit a bunch of different event types, so a single transform rule didn't make sense. I wanted to do a source-based rule, triggering on the host IP in the directory name, to capture everything from this host in a sourcetype.

My rule looks like this:

[source::.../192.168.11.175/*.log]
sourcetype = other_log

I've tried a number of possible stanza definitions, guided in part by this answer: http://splunk-base.splunk.com/answers/57527/forwarder-propsconf-source-stanza

I can't get the source rule to trigger; I never have any events in the 'other_log' sourcetype, they always remain as 'syslog'. What can I do to triage this? What settings would I tweak in the log to show what Splunk is trying to do? Am I missing something obvious?

Re: source:: rule in props.conf ignored?

Mick — Mon, 17 Dec 2012 19:15:11 GMT

The instructions in the docs are for specifically resetting auto-sourcetyped data, but you have already set a manual sourcetype in inputs.conf, so it's never going to get overwritten again, unless you specifically use a props/transforms entry to re-write it completely, an example is posted here - http://docs.splunk.com/Documentation/Splunk/5.0/Data/Advancedsourcetypeoverrides#Example:_Assign_a_source_type_to_events_from_a_single_input_but_different_hosts

Another alternative would be to remove 'sourcetype = syslog' from inputs.conf and rely on a combination of auto-sourcetyping and other props.conf stanzas to set the sourcetypes on the non-syslog data.

Re: source:: rule in props.conf ignored?

sowings — Mon, 17 Dec 2012 19:36:42 GMT

Thanks.

I had mistakenly believed that [source:: ] rules had higher priority than [sourcetype] stanzas within props.conf, so that I could treat [source:: ] entries as exceptions and [sourcetype]s as the rule....

I'll find another approach.

Re: source:: rule in props.conf ignored?

gkanapathy — Mon, 17 Dec 2012 20:58:15 GMT

You can do this a couple of ways:

Remove the sourcetype from inputs.conf, and specify source:: rules in props.conf, making sure to cover all possible files from the inputs.conf; or
Remove the props.conf entry and simply use an overlapping inputs.conf entry with a whitelist that for your desired filename pattern, and specify the sourcetype there.

Re: source:: rule in props.conf ignored?

sowings — Wed, 10 Apr 2013 14:36:32 GMT

Can such an overlapping inputs.conf entry be used with Splunk 4.2.x?

Re: source:: rule in props.conf ignored?

gkanapathy — Wed, 10 Apr 2013 15:33:44 GMT

Yes, overlapping inputs.conf entries work from 4.2 on.