topic Re: Splunk precedence issue in Getting Data In

Splunk precedence issue

rameshlpatel — Thu, 29 May 2014 11:23:50 GMT

Hi,

I have outputs.conf file under etc/system/local folder with following conf.

[tcpout-server://10.248.180.196:9997]
[tcpout:default-autolb-group]
server = 10.248.180.196:9997

In addition, I deployed app with outputs.conf (with following conf) from deployment server to etc/app dir.

[tcpout-server://alpputl018:9997]

[tcpout:default-autolb-group]
server = alpputl018:9997

Ideally app folder outputs.conf should override system/local outputs.conf. means ideally logs should be forward to alpputl018, but in my scenario its still pointing to old indexer i.e. 10.248.180.196.

In addition. forwarder logs are forwarding to new indexer but not application log.

This issue is really strength to me and not working as per splunk precedence theory.

Please help me out to understand this issue.

Re: Splunk precedence issue

kheli — Thu, 29 May 2014 12:21:13 GMT

indexing is global context so config in /etc/system/local will take precendence.

You can also use btool command to find all outputs.conf value in a splunk instance.

http://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/Usebtooltotroubleshootconfigurations

If you cannot see application logs are being indexed, make sure the index for the application log is created in the indexer and data input has been configured properly in forwarder.

Re: Splunk precedence issue

rameshlpatel — Thu, 29 May 2014 12:24:13 GMT

I ran btool and its showing old one. Now problem is how I override this configuration with new from deployment server ?

Re: Splunk precedence issue

rameshlpatel — Thu, 29 May 2014 12:37:03 GMT

index has been created in new indexer and monitoring path is also properly configured in forwarders.

Re: Splunk precedence issue

sowings — Thu, 29 May 2014 13:30:42 GMT

$SPLUNK_HOME/etc/system/local takes precedence over any app config (whether local OR default) in $SPLUNK_HOME/etc/apps. If you are using the deployment server, you are best served by not placing any local (site-specific) configs in $SPLUNK_HOME/etc/system/local, since these cannot be overridden by apps sent by the deployment server.

Because of the precedence rules set out in $SPLUNK_HOME/etc/system/default/conf.conf, the behavior that [~rameshlpatel] is observing is correct, even if it's not what's intended.

Re: Splunk precedence issue

rameshlpatel — Thu, 29 May 2014 16:25:53 GMT

Thanks for clearing my doubts.

Re: Splunk precedence issue

sowings — Fri, 29 Aug 2014 20:39:06 GMT

btw, "cd $SPLUNK_HOME/etc/system/default ; grep conf conf.conf | grep -v confdb". The apps provided from a cluster master (placed in the slave-apps folder on the clustered indexer) override even system/local!