topic If there are multiple outputs.conf files (apps vs system), how do they merge, or which one takes precedence? in Getting Data In

If there are multiple outputs.conf files (apps vs system), how do they merge, or which one takes precedence?

jlemoine — Wed, 12 Oct 2016 21:44:23 GMT

I've inherited an older Splunk instance (6.2.2) that appears to have multiple /local/outputs.conf files. While I'm familiar with the precedence inside of the outputs.conf file that I'm used to working with inside of /system/local/ (Global, Target, Single,) I don't know who wins the fight when there are multiple /local/outputs.conf files. Can anyone shed some light on the order of precedence for the following:

[splunk@superawesomeserver etc]$ locate outputs.conf
/data/splunkforwarder/etc/system/local/outputs.conf
/opt/splunkforwarder/etc/apps/uni_splunk_forwarders/local/outputs.conf
/opt/splunkforwarder/etc/system/local/outputs.conf

Obviously, there appears to be two $SPLUNK_HOME's, which is another issue altogether, but when it comes down to the apps vs system outputs.conf, how do they merge?

Thanks in advance!

Re: If there are multiple outputs.conf files (apps vs system), how do they merge, or which one takes precedence?

dmaislin_splunk — Wed, 12 Oct 2016 22:13:25 GMT

All covered in depth here: https://docs.splunk.com/Documentation/Splunk/6.5.0/Admin/Wheretofindtheconfigurationfiles

Basically:
etc/system/default loads first at the bottom of the totem pole
etc/apps/appname/default
etc/apps/appname/local
etc/system/local is highest

All the config files with the same name are merged and same settings under same name stanzas are overwritten based on the precedence docs I listed above.

Re: If there are multiple outputs.conf files (apps vs system), how do they merge, or which one takes precedence?

jlemoine — Thu, 13 Oct 2016 13:18:03 GMT

Sweet and simple, thank you!

Re: If there are multiple outputs.conf files (apps vs system), how do they merge, or which one takes precedence?

jlemoine — Thu, 13 Oct 2016 14:20:18 GMT

Will an etc/system/default/outputs.conf override an etc/apps/appname/local/outputs.conf?

Re: If there are multiple outputs.conf files (apps vs system), how do they merge, or which one takes precedence?

dmaislin_splunk — Fri, 14 Oct 2016 00:29:59 GMT

No, you shouldn't be messing with anything in the system/defaults folder anyway. The app takes precedence.

Re: If there are multiple outputs.conf files (apps vs system), how do they merge, or which one takes precedence?

minx66 — Wed, 29 Mar 2017 05:23:13 GMT

So if there are multipsle input files under different app folders but there are some of the same log types in them, which apps will get preference?

Re: If there are multiple outputs.conf files (apps vs system), how do they merge, or which one takes precedence?

earlhelms — Wed, 29 Mar 2017 13:07:49 GMT

btool is your friend - https://docs.splunk.com/Documentation/Splunk/6.5.2/Troubleshooting/Usebtooltotroubleshootconfigurations
IMO, it is the best way to validate what values are set to.