How do you automate the "Apply Change" to register new servers on the Distributed Management Console?

[pdaigle_splunk]

I have a customer who asked:

We need to automatically register new servers on the Distributed Management Console. For this we use the Splunk CLI add search-peer. However, after this, we still need to manually press “Apply Changes” in the UI to complete the setup. Are you familiar with a way to allows us to automate this step?

http://docs.splunk.com/Documentation/Splunk/6.2.8/Admin/ConfiguretheMonitoringConsole

---

Any help here is greatly appreciated!

Thank you!
PD2

[nibinabr]

I ended up tracking the HTTP requests and automated Apply Changes hitting the following endpoints.

/servicesNS/nobody/splunk_monitoring_console/configs/conf-splunk_monitoring_console_assets/settings -d configuredPeers=<comma separated lists of peers> -d output_mode=json -d disabled=0 -d eai:appName=splunk_monitoring_console -d eai:userName=nobody

/servicesNS/nobody/splunk_monitoring_console/saved/searches/DMC+Asset+-+Build+Full/dispatch -d output_mode=json -d trigger_actions=true -d dispatch.auto_cancel=30 -d dispatch.buckets=300 -d dispatch.enablePreview=true

/servicesNS/nobody/system/apps/local/splunk_monitoring_console -d output_mode=json -d author=Splunk -d check_for_updates=1 -d configured=1 -d description=The+Splunk+Monitoring+Console+application+gives+you+insight+into+your+Splunk+deployment. -d label=Monitoring+Console -d version=7.0.1 -d visible=1

First endpoint will update the splunk_monitoring_console_assets.conf. Second endpoint will run the DMC Asset search which is what I believe is building the assets and the last one to update the app.conf.

UPDATE: There were few more updates that I had to make to get this fully automated. I put this gist together.
https://gist.github.com/nmattam/bcfbc8a4ebd9a520c2ac50ab0137e58f

[Aftend1971]

Do you know what to do for Apply Changes click in Distributed Monitoring Console?
Peers are showing in State: New, instead of State: Configured. And I have no Dasboards on Overview page, unitl I click on Apply Changes

[nibinabr]

I updated my comment above with a gist of everything that I had to do to get this working. Try giving it a shot.

[Aftend1971]

It seems that last step from https://gist.github.com/nmattam/bcfbc8a4ebd9a520c2ac50ab0137e58f is actually /opt/splunk/etc/apps/splunk_monitoring_console/default/data/ui/nav/default.distributed.xml ... so if you rename it and copy it like: splunk cmd /bin/mkdir -p /opt/splunk/etc/apps/splunk_monitoring_console/local/data/ui/nav; /bin/cp /opt/splunk/etc/apps/splunk_monitoring_console/default/data/ui/nav/default.distributed.xml /opt/splunk/etc/apps/splunk_monitoring_console/local/data/ui/nav/default.xml. It should work, but it does not....

[nibinabr]

I was able to get it to work after renaming the xml and then restarting splunk.

[altinp]

To configure or update the DMC in automated fashion, it seems like you have to fully populate all of these files correctly (for Distributed mode):

splunk_monitoring_console/local/app.conf
splunk_monitoring_console/local/assets.csv
splunk_monitoring_console/local/splunk_monitoring_console_assets.conf
splunk_monitoring_console/local/savedsearches.conf
etc/system/local/distsearch.conf

You can copy aside the files after your config management tool (e.g. SaltStack) has generated them and then diff with the version Splunk has modified once you "Apply Changes". Any major changes will tell you what you still need to do 'manually'. Once you have it all, clicking "Apply Changes" (or enabling Distributed mode or 'Enable Monitoring') is no longer needed.

[lianwan]

tried this trick, but does not work. The contents of those files before and after click "Apply Changes" are now identical, except the order in etc/system/local/distsearch.conf may vary. Some how clicking that button results some randomness in order

[richgalloway]

Just an idea, but perhaps you could use curl to send the same form the Apply Changes button sends to Splunk.

[javiergn]

I don't know the answer for your question but what I've done in the past when I wanted to automate something that wasn't clearly documented was to look at the HTTP requests using a proxy or something similar.
For instance, the Live HTTP Headers add-on for Chrome and Firefox can do that very easily without needing to install or configure anything else.

Once you have the HTTP requests identified you can use the same POST/GET call from CURL or via the SDK and implement this yourself.

Hope that makes sense.

Thanks,
Javier