topic Re: Why is the default taking precedence over the sourcetype I've set in inputs.conf or props.conf? in Getting Data In

Why is the default taking precedence over the sourcetype I've set in inputs.conf or props.conf?

alange — Tue, 29 Sep 2020 12:10:30 GMT

I have set the sourcetype for access logs in inputs.conf + props.conf before, but on one host it is not recognizing the explicit sourcetype I set on the local host (running the Splunk forwarder). inputs.conf and props.conf in their own subdirectories because different applications sometimes reuse directory and file names with different formats, and having separate directories makes it easy to put the specific Splunk config files I need on each host depending on the applications it runs.

$SPLUNK/etc/apps/myapp/local/inputs.conf:
[monitor:///app/logs/webserver]
index = myindex
whitelist = access.log|error.log

$SPLUNK/etc/apps/myapp/local/props.conf:
[source::.../access\.log]
sourcetype=access_myapp

[source::.../error*]
sourcetype=error_nginx

metrics.log (and the Splunk index) show that the sourcetype being assigned is access_combined_wcookie, which is NOT correct, since it is a custom log (several extra fields in addition to access_combined).

Again, I have set sourcetypes this way for many other hosts - including a number which have custom webserver access logs, both Apache and Nginx.

Re: Why is the default taking precedence over the sourcetype I've set in inputs.conf or props.conf?

somesoni2 — Fri, 16 Dec 2016 21:11:09 GMT

Are you using Heavy Forwarder OR Universal Forwarder? If Universal Forwarder, then props.conf should be on Indexers. Also, why not just create two monitor entries in inputs.conf for each file type and specify sourcetype in inputs.conf itself.

Re: Why is the default taking precedence over the sourcetype I've set in inputs.conf or props.conf?

alange — Fri, 16 Dec 2016 22:57:56 GMT

Two separate files (inputs.conf and props.conf), since I often have multiple webserver instances on a single host (each writing to a separate subdirectory under /app/logs/webserver/). This way I don't have to make any changes to the Splunk configuration when adding another webserver instance - it will automatically get indexed. The instances can be separated when searching based on the source.

I don't recall whether my site is using Heavy or Universal forwarder - but the sourcetype is being set by the forwarder (clear in the metrics.log on the application host). With several hundred hosts running tens of different sets of applications, all data being proxied to a set of indexers, it would not be practical to manage all the different inputs and props on the indexers - so the forwarders are where I believe the sourcetype should be set.

Re: Why is the default taking precedence over the sourcetype I've set in inputs.conf or props.conf?

alange — Fri, 16 Dec 2016 23:03:32 GMT

Update - one of four hosts with the same webserver instance and Splunk configuration is assigning the correct sourcetype. I believe it was because the Splunk forwarder on that host had been stopped when I added the new inputs.conf and props.conf, and it used the desired values when I started it up.

In the past I have made changes to inputs.conf and props.conf, and the Splunk forwarder would honor the new values after being restarted (for any new entries) - this is the first time I've had it persist with old values for hours after the change.