Why am I unable to update splunk cloud universal forwarder settings on Linux?

[hifimarko]

I ran into a problem while putting together an Ansible playbook for deploying forwarder config. The initial deployment works just fine but if I try and update the forwarders with the new outputs.conf it's as if the new configuration doesn't get picked up. I've restarted the service.

In order to have better control over splunk service restarts, I am not using splunk install app to install the forwarder. I'm placing splunkclouduf configuration files into the apps directory and restarting splunk service only if there are configuration changes.

Is there a location where splunk caches forwarder settings?

[renjith_nair · Accepted solution]

@hifimarko,

It could be due to the configuration file precedence. Your configuration files in the app might be overriden by a local directory parameter. Use btool to list and see the configuration sources

Precedence order within global context:
When the context is global (that is, where there's no app/user context), directory priority descends in this order:

1. System local directory -- highest priority
2. App local directories
3. App default directories
4. System default directory -- lowest priority

Precedence order within app or user context
When there's an app/user context, directory priority descends from user to app to system:

1. User directories for current user -- highest priority
2. App directories for currently running app (local, followed by default)
3. App directories for all other apps (local, followed by default) -- for exported settings only
4. System directories (local, followed by default) -- lowest priority

Reference : http://docs.splunk.com/Documentation/Splunk/7.1.2/Admin/Wheretofindtheconfigurationfiles