topic Re: How to properly configure a deployment app to deploy configuration files in Deployment Architecture

How to properly configure a deployment app to deploy configuration files

avalle — Tue, 29 Sep 2020 12:11:50 GMT

Hello,
I have searched the answer but can't seem to find out how to properly set up my apps in the deployment server.
I already have over 4k devices communicating to the deployment server, they are all in their correct server class and I have a directory for each app in the $SPLUNK_HOME/etc/deployment-apps. Let's say I have App1 and i want to use App1 to send my configuration files to a server class. I create $SPLUNK_HOME/etc/deployment-apps/App1/local/outputs.conf with the correct settings and I deploy the app but for some reason it does not update the outpts.conf on the clients. What am I doing wrong?

Re: How to properly configure a deployment app to deploy configuration files

somesoni2 — Fri, 23 Dec 2016 19:15:12 GMT

You need to assign your App1 to the servers who should receive. From your description, you've already completed step "create deployment app" https://docs.splunk.com/Documentation/Splunk/6.5.1/Updating/Createdeploymentapps. If you're going to use Deployment Server Web UI -> Forwarder Management dashboard to configure clients, you need to reload your deployment server now, to make the App1 visible in UI. Now use steps from following link to add/update serverclass, add clients and App1. https://docs.splunk.com/Documentation/Splunk/6.5.1/Updating/Useforwardermanagement

Same from serverclass.conf file directly:- http://docs.splunk.com/Documentation/Splunk/6.5.1/Updating/Useserverclass.conf

Re: How to properly configure a deployment app to deploy configuration files

ddrillic — Fri, 23 Dec 2016 20:10:11 GMT

You mentioned $SPLUNK_HOME/etc/deployment-apps/App1/local/outputs.conf. What about its twin? inputs.conf?

Re: How to properly configure a deployment app to deploy configuration files

Richfez — Sat, 24 Dec 2016 03:32:14 GMT

It depends by what you mean by "does not update the outputs.conf on the clients?"

If you mean the App1/local/outputs.conf doesn't get updated (physically checking the file on clients that show as having checked in recently don't show the update?) then are you sure you ran a splunk reload deploy-server as per the documentation on deploying apps?

If instead the files themselves update (are physically changed) but the settings don't "take effect", then perhaps you've hit a file precedence issue. Apps can't override settings made in $SPLUNK_HOME/etc/system/local/. In that case, confirm your app's files are in place and *should work, then delete the local copy out of the system/local version and confirm that works for a few test cases, then deploy whatever it is you did to delete those files more widely once you are happy..

Re: How to properly configure a deployment app to deploy configuration files

avalle — Tue, 27 Dec 2016 21:27:01 GMT

Right now I am working on updating one conf file at a time it does not matter which one as long as i get one working.

Re: How to properly configure a deployment app to deploy configuration files

avalle — Tue, 27 Dec 2016 21:31:00 GMT

Thank you so much for your answer! I did reload the deployment server and the configuration file I had in $SPLUNK_HOME/etc/deployment-apps/App1/local disappeared and the Forwarders did not get the updated configuration files therefore I am not ready to delete the conf files in the local directory until I get thus update to happen in the UF.

Re: How to properly configure a deployment app to deploy configuration files

avalle — Tue, 27 Dec 2016 21:32:00 GMT

I reloaded the deployment server and the configuration files under the $SPLUNK_HOME/etc/deployment-apps/App1/local directory disappeared and the UF did not get the updated configuration files.

Re: How to properly configure a deployment app to deploy configuration files

avalle — Tue, 27 Dec 2016 22:11:59 GMT

Apparently I was looking in the wrong UF directory.
The configuration files in the UF have been updated on $SPLUNK_HOME\etc\apps\security\local but for some reason the new entries are all in one line. Do I have to remove the conf files in the etc\system\local directory to test if these new files work?

Re: How to properly configure a deployment app to deploy configuration files

Richfez — Fri, 06 Jan 2017 21:07:42 GMT

The configuration file precedence docs page explains how this works, and I believe this link will take you right to the precedence section.

But, basically, yes. That last link above has a paragraph that's quite clear - if you have a setting set in the etc/system/local/blah.conf file, that setting takes precedence over all others.

As a concrete example, when I deploy my UFs manually (which I do for one-off installs or whatever) I supply a deployment server IP address and port for the UF to contact for its configuration. That writes a etc/system/local/deploymentclient.conf file on the UF system. Splunk starts, contacts the DS named in that file and gets its configuration, right? So far fine.

One of the files in one of the apps I have deployed is etc/apps/mycompany_deploymentclient/local/deploymentclient.conf. THIS deploymentclient.conf is my official one, and if I have to move or change my Deployment Server, I can set up the new DS and then change this file to point to the new server. The UFs will contact the old server, get a copy of the new configuration, restart and start talking to the NEW DS, right?

But NOT if they still have a etc/system/local/deploymentclient.conf. If that latter file exists, it's settings will override anything I could send in an app. So, basically for the one-off installs I use this method for I install the UF with a DS setup, let it start and get its "real" config (which includes the deploymentclient.conf file it really needs) and then delete the system/local/ version and restart it again.

Now that's an example of file-level precedence, but settings inside files work the same way. If in an etc/system/local/X.conf file you have setting Z, then NO OTHER setting for Z elsewhere will ever override the way it's set in etc/system/local/X.conf.

Indeed, this is a great feature once it's understood. For instance, all the default settings are in etc/system/default/blah.conf <- the LOWEST precedence files. That means that all other settings will take effect (according to their own respective precedences) and only if no other file has a particular setting that's set in default will that setting take the default.

Make sense?

Re: How to properly configure a deployment app to deploy configuration files

woodcock — Tue, 07 Feb 2017 20:13:18 GMT

The replacement of apps on the forwarder is atomic and this confuses many people. The process is like this:

DC(forwarder) -> DC: "I am host X"
DS            -> DC: "Send me checksums for these apps: A, B, C, D"
DC            -> DS: "A=123, B=234, C=345, D=<null>"
DS            -> DC: "Install this 'D' and replace your 'A' with this"
DC                 : cd $SPLUNK_HOME/etc/apps; mv A /tmp/, tar xvf A.tgz; merge /tmp/A/local with ./A/local

It may not be exactly like this but for the purposes of this question, it proves this point: If you are sitting in the A directory on the forwarder and watching for files to change, you will never see the changes because you are in the old directory that has been trashed. You need to cd out of the old/moved/trashed directory into the new/replacement/merged directory by again doing cd $SPLUNK_HOME/etc/apps/A/ after a few minutes to make sure that you are in the right place to see the changes.

Re: How to properly configure a deployment app to deploy configuration files

aaraneta_splunk — Sat, 11 Mar 2017 17:26:14 GMT

@avalle - Did one of the answers below help provide a solution your question? If yes, please click “Accept” below the best answer to resolve this post and upvote anything that was helpful. If no, please leave a comment with more feedback. Thanks.

Re: How to properly configure a deployment app to deploy configuration files

woodcock — Sat, 11 Mar 2017 17:58:03 GMT

If the problem is that you are deploying an outputs.conf from the DS and some (or all) of the forwarders do not begin to send differently (to the new indexers or whatever) then the problem is 1 of 2 things:

1: The splunkd instance has not been restarted on the forwarder. You need to set the Restart Splunk setting for the App that contains outputs.conf inside of your Deployment Server.

2: There is another outputs.conf file on the forwarder that has a higher precedence that the one that you are deploying. This probably means $SPLUNK_HOME/etc/system/local (or $SPLUNK_HOME/etc/system/default, which is just WRONG, but check anyway). Anything in system/local is GOD and cannot be overridden with one exception: slave-apps in an Indexer Cluster.