Interconnect two deployment servers managed by two different departments

[tomasmoser]

Hi Experts,

due to politics I have a huge Deployment server problem. Please help me out. What is the viable and feasible technical solution? Let me explain.

Team A
- Large company has a large Splunk deployment managed by team A.
- has large Deployment Server (DS) infrastructure all over the world and they are in charge of deploying apps to UFs/HFs.
- needs only some data from source hosts

Team B
- decided later to deploy their own much smaller Splunk (just Indexer and Search Head).
- deployment of apps to IDX/SH is manual
- deployment of apps to UFs/HFs is via requests to Team A (slow, misunderstandings, issues, etc.) - does not work at all
- needs its independence to push apps via their future independent DS to UFs/HFs (shared with Team A)
- does not have even physical access to source servers with HFs/UFs 🙂
- does not have direct access (ssh, web) to team A's DS infrastructure (and will NOT get it for security reasons).
- needs its DS but cannot push apps to UFs/HFs from their potentially future DS because forwarders cannot have two DS servers

My questions - What are Team B's options? How can they technically interconnect with Team's A Deployment Server infrastructure? I am sure it's possible if there is a political will. Otherwise I presume team's B project will get stuck.

[nickhills]

In order for a DS to be useful (for managing your own apps etc), you really need access to the CLI on the DS.

If the app is already uploaded (and you don't need to change its config etc) you can get away with managing which UF clients get the apps from the DS GUI, but be aware that CLI managed serverclasses do not always play nicely with GUI managed serverclasses - expect conflicts!

You are correct in saying that a UF/DeploymentClient can not be a member of two Deployment Servers, so unless Team B gets access to Team A's DS, I think you are, as you say, Stuck.

One option is to have a process external from Splunk managed Team B's Apps on Team A's DS.
You could use git/jenkins/scripts (whatever) to sync Team B's apps to Team A's DS - then Team B could use the GUI to assign those apps to Team B's endpoints. This could technically work, however nothing would stop Team B publishing an evil app, and then pushing it to a Team A endpoint - not suggesting they would, but it is a risk.

It actually sounds like the process in place is 'high level correct' - Team A manages the DS and the clients, but you just need to work on the process so that Team B can request changes in a timely and accurate manor, and that Team A can correctly implement the requested changes.

[tomasmoser]

Would you recommend Team B to install their own set of UFs to a set of servers (of course with changing ports etc.). These UFs could be managed independently by Team B's DS. Sounds horrible but ... 🙂

[MuS]

It does sound horrible indeed 😉