Deployment Architecture

How to force shcluster member to send config from local dir (e.g. savedsearches.conf) to captain from cli or via rest

Path Finder

I have reason to change *.conf in local directory of app on the searchhead in shcluster, and i need to tell to this searchhead to send changes to captain for spreading via whole cluster.
How i can do it via cli or REST?

Thank you

0 Karma

SplunkTrust
SplunkTrust

May I suggest you change this concept completely?

Instead of working against the clustering and updating configuration files on the individual search heads, work with the clustering by updating saved searches via the REST API
You could an updated search via REST and that will work with the clustering...

https://localhost:8000/en-US/debug/refresh?entity=saved/searches

Might also work, I'm unsure if that is officially supported or not, note that you would have to update all search heads at the same time and then trigger something like that, and I'm not sure it would work as expected...but it's something you could try

0 Karma

SplunkTrust
SplunkTrust

That's just against the concept of a master or - in this case - the captain. The captain always has the working version. If you'd update a config file on one search head and thus destroy your app on the whole cluster, this would be a big problem.

The concept is good, because even if you mess up your configuration on a node, you can simply resync/push out the last working configuration files from your captain and everything is fine. I would not recommend using any version control system on Search Heads. If you do development, do it on a standalone Splunk instance and deploy the app to the cluster via the captain.

0 Karma

Super Champion

welcome to the pain of Splunk SH clustering. We never change anything via directly into the SH member, but always makes changes into deployer and pushes it out. Unfortunately it goes into "default" of the members

0 Karma

Path Finder

Thank you.
Splunk shcluster is like wedding - no one says why it's bad, but everyone talk why it is great... and you have no way back.

0 Karma

SplunkTrust
SplunkTrust

how do you make that change? if you save a search or report or anything of that sort, change should replicate across cluster members regardless to where you made the change

0 Karma

Path Finder

i working directly with savedsearches.conf with vi-editor.

0 Karma

SplunkTrust
SplunkTrust

and when you save the file, the saved file does not replicate to other members?

0 Karma

Path Finder

Yes, because splunkd nothing know about changed savedsearches.conf.
In standalone searchhead i can go to https://splunk/debug/refresh and splunk will reload configs, but in shcluster i need not only reload configs, but also send to captain information about changed configs so old method is not suitable.

0 Karma

SplunkTrust
SplunkTrust

what is the use case for editing savedsearches.conf using VI directly on a search head?
if its part of an app, why not modify on deployer and push to members?
on the other hand, maybe use case is worthwhile to bring to splunk attention in a form of ER.

0 Karma

Path Finder

🙂 it is long, sad and strange story with some bugs in splunk which was not fixed in current version, but have fixed in some build of previous...
Reload from cli can make my life more comfortable and the hair silky.

0 Karma

SplunkTrust
SplunkTrust

sounds like General Hospital, but if you are already so deep in #@$%, why not just scp to the other search heads and get it over with?

0 Karma

Path Finder

I already doing it via github for versions control.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!