Splunk Orchestration, aka Manage Distributed Deployment from A to Z

[theunf]

We´re researching for solutions to offer Splunk as an internal cloud solution thru all different and disperse areas of a client where an internal client could connect to a dashboard and create apps, users, indexes and start the input of multiple data sources.

This global managing solution would distribute the tasks among each role, so apps and users goes to the search head pooling/cluster, indexes goes to master cluster node or individually indexers (besides the best splunk practice suggests to create each index on all roles) and the data inputs to deployment server or directly to heavy/universal forwarders.

Does anyone experiencing Orchestration tools as from IBM or Cisco to manage the hole picture : servers, storage and all Splunk roles ?

[Ricapar]

I don't have a answer with specific products for this, but I do have some general ideas. I am also currently trying to tackle this problem.

• Find an orchestration tool that is capable of calling REST APIs and ingesting XML or JSON. Use this against Splunk's REST APIs, which are fairly well documented.
• Use source control - store all your configs in git in a central repository, and have automated scripts to pull down the appropriate configs to the respective clients.

Right now I'm using a hacked-up shell script to dynamically create apps/roles/indexes when onboarding new users using the REST API. This can be heavily improved with the use of an actual self-service type catalog that calls a workflow engine.