- Splunk Answers
- :
- Splunk Administration
- :
- Deployment Architecture
- :
- Tiered Deployment Servers, is it possible?
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi, We are working on planning for a large splunk deployment, I would like to know if it is all possible to set up a deployment server, which pushes out configurations / apps to another deployment server, which in turn pushes these out to lightfowarders & indexers on the other side of a firewall (from the original deployment server).
Also is it possible for an instance to be both a light forwarder and a deployment server?
Cheers
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Actually, the behavior on the deployment client side to the use of targetRepositoryLocation appears to be buggy: if you use it, the second tier deployment server (also a deployment client of the first tier deployment server) will forever try, and fail, to "install" the apps pushed to the alternate targetRepositoryLocation (say, $SPLUNK_HOME/etc/deployment-apps), but fail to do so. If you've also told it, in deploymentclient.conf, to restart deployment server when new apps are installed (which you almost definitely need to do), then that'll happen every pollingInterval, and your final deployment clients will keep getting told to redo their handshakes. The repeated attempts and failures to install the apps on the second-tier deployment server take place regardless of whether the server class on the first-tier deployment server has "stateOnClient" set to enabled or disabled.
I just tested all of this with 4.2.2 on RHEL Linux 5.6 systems, and the work-around I'm using for multi-tier deployment is to ignore targetRepositoryLocation and, instead, tell serverclass.conf on the second-tier deployment server that its repository is $SPLUNK_HOME/etc/apps, having set the apps I want that server to deploy (but not run) to be disabled. The details on how that works follow.
On the primary deployment server, deploy-svr-1, I've got two classes for my slave deployment server, deploy-svr-2, in serverclass.conf. They look like this:
[serverClass:Tier2-DeploymentServer]
whitelist.0=deploy-svr-2
[serverClass:Tier2-DeploymentServer:app:Tier2-DeploymentServer]
[serverClass:Tier2-DeploymentApps]
whitelist.0=deploy-svr-2
stateOnClient = disabled
[serverClass:Tier2-DeploymentApps:app:Tier2-Inputs]
[serverClass:Tier2-DeploymentApps:app:Tier2-Outputs]
[serverClass:Tier2-DeploymentApps:app:DeploymentClient]
[... so forth; anything that goes on the final deployment clients]
Then we tell deploy-svr-2 to be a deployment client of deploy-svr-1, reloading its own deployment server configuration after it's installed new apps (so that it will then push any of those out to the lower tier that it needs to):
[deployment-client]
reloadDSOnAppInstall = true
[target-broker:deploymentServer]
targetUri = deploy-svr-1:8089
The Tier2-DeploymentServer app on deploy-svr-1 contains only its own serverclass.conf for the final deployment clients, which looks like this:
[global]
continueMatching = true
restartSplunkd = true
repositoryLocation = $SPLUNK_HOME/etc/apps
[serverClass:Tier2-Forwarders]
whitelist.0=*
stateOnClient = enabled
[serverClass:Tier2-Forwarders:app:Tier2-Inputs]
[serverClass:Tier2-Forwarders:app:Tier2-Outputs]
[serverClass:Tier2-Forwarders:app:DeploymentClient]
As you can see, I'm telling deploy-svr-2's deployment server to look for apps to deploy right in $SPLUNK_HOME/etc/apps, where deploy-svr-1 will have pushed them, and set them to be disabled. (Setting stateOnClient explicitly in the Tier2-Forwarders server class on deploy-svr-2 is actually redundant--that's the default--but I've got it in there for clarity.)
Finally, the deploymentclient.conf for final tier deployment clients looks like this:
[deployment-client]
[target-broker:deploymentServer]
targetUri = deploy-svr-2:8089
The one caveat here is that we have to avoid having any of the apps that deploy-svr-2 will deploy also need to run on deploy-svr-2 itself, which will typically be inputs.conf (to pick up local log files and potentially run scripted inputs) and outputs.conf (pointing back to the main Splunk indexing cluster). In my case, that's fine, because deploy-svr-2 is also intended to be a forwarder for logs from systems within a separate network, so I need to have a different configuration for deployment clients in that environment (sending their events to deploy-svr-2) than I will for deploy-svr-2 itself (sending events to my main indexing cluster), but it's important to remember that if you want an app to run on deploy-svr-2, you can't include it in deploy-svr-1's Tier2-DeploymentApps server class, because that will get it disabled on deploy-svr-2.
Incidentally, the deployment server class documentation that Genti pointed to above has since moved to here.
.conf24 | Registration Open!
ICYMI - Check out the latest releases of Splunk Edge Processor
Introducing the 2024 SplunkTrust!
