Deployment Architecture

Error trying to push app from deployer

Communicator

I'm getting this error when I try to push an app to my search head cluster:
Error when getting master uri from target to do a rolling-restart Service Unavailable

I've used the command:
sudo ./splunk apply shcluster-bundle -target https://[SOME SEARCH HEAD]:8089 -auth admin:'[THE PASSWORD]'

It stopped working since we've fixed the IP addresses of the Splunk nodes. Anyone an idea how to fix/debug this?

1 Solution

SplunkTrust
SplunkTrust

You need to correct anything wrong with your SHC first.

Run this command from your search head(s) and let us know the output:

 splunk show shcluster-status -auth <username>:<password>

Also it is best practice to create your own splunk user & group, and run splunk as that user.

#If splunk is running as sudo or root, stop it first.
groupadd splunk
useradd splunk -g splunk
chown -Rf splunk. /opt/splunk  #or wherever you have splunk installed

You'll probably have to fix/rebuild the SHC using some of these commands from each search head:

 splunk init shcluster-config -auth <username>:<password> -mgmt_uri <URI>:<management_port> -replication_port <replication_port> -replication_factor <n> -conf_deploy_fetch_url <URL>:<management_port> -secret <security_key>

 splunk restart

http://docs.splunk.com/Documentation/Splunk/6.2.0/DistSearch/ViewSHCstatus
http://docs.splunk.com/Documentation/Splunk/6.2.0/DistSearch/SHCdeploymentoverview

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

You need to correct anything wrong with your SHC first.

Run this command from your search head(s) and let us know the output:

 splunk show shcluster-status -auth <username>:<password>

Also it is best practice to create your own splunk user & group, and run splunk as that user.

#If splunk is running as sudo or root, stop it first.
groupadd splunk
useradd splunk -g splunk
chown -Rf splunk. /opt/splunk  #or wherever you have splunk installed

You'll probably have to fix/rebuild the SHC using some of these commands from each search head:

 splunk init shcluster-config -auth <username>:<password> -mgmt_uri <URI>:<management_port> -replication_port <replication_port> -replication_factor <n> -conf_deploy_fetch_url <URL>:<management_port> -secret <security_key>

 splunk restart

http://docs.splunk.com/Documentation/Splunk/6.2.0/DistSearch/ViewSHCstatus
http://docs.splunk.com/Documentation/Splunk/6.2.0/DistSearch/SHCdeploymentoverview

View solution in original post

0 Karma

Communicator

We've rebuild the SHC (splunk init shcluster-config ... ) and started the election for captain again. Then things started working again. Thanks guys!

0 Karma

Champion

I don't know if this causes your problem, but generally, I'd advise against sudoing splunk. It causes all kind of undesired behavior with folder/file ownership and permissions if some splunk actions are performed as a different user.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!