In search head clustering you use a deployer instance to push apps to the search head cluster members using the command:
$SPLUNK_HOME/bin:
./splunk apply shcluster-bundle -target https://10.10.10.1:8089 -preserve-lookups true -auth admin:pwd
where <10.10.10.1> is one member of the search head cluster.
The configuration bundle push fails and produces an error message like this:
Error while creating deployable apps: Error copying src="/opt/splunk/etc/shcluster" to tmp_staging_area="/opt/splunk/var/run/splunk/deploy.c6055628e3976924.tmp": No such file or directory
What does this mean?
environment is Linux
This error message can be generated if splunk (or the user splunk is running as) does not have read permissions to any file in the app you are trying to push. One approach to identifying what file splunk cannot read is to restart splunk
$SPLUNK_HOME/bin
./splunk restart
Upon restart there should be a display of the file(s) which the splunk user does not have permissions for.
example:
Splunk> Map. Reduce. Recycle.
Checking prerequisites...
Checking http port [8000]: open
Checking mgmt port [8089]: open
Checking appserver port [127.0.0.1:8065]: open
Checking kvstore port [8191]: open
Checking configuration... Error while parsing '/opt/splunk/etc/shcluster/apps/RobApp/local/data/ui/views/rob.xml':
[Errno 13] Permission denied: '/opt/splunk/etc/shcluster/apps/RobApp/local/data/ui/views/rob.xml'
To reslove the issue, update the permissions of the file and apply the shcluster-bundle again
This error message can be generated if splunk (or the user splunk is running as) does not have read permissions to any file in the app you are trying to push. One approach to identifying what file splunk cannot read is to restart splunk
$SPLUNK_HOME/bin
./splunk restart
Upon restart there should be a display of the file(s) which the splunk user does not have permissions for.
example:
Splunk> Map. Reduce. Recycle.
Checking prerequisites...
Checking http port [8000]: open
Checking mgmt port [8089]: open
Checking appserver port [127.0.0.1:8065]: open
Checking kvstore port [8191]: open
Checking configuration... Error while parsing '/opt/splunk/etc/shcluster/apps/RobApp/local/data/ui/views/rob.xml':
[Errno 13] Permission denied: '/opt/splunk/etc/shcluster/apps/RobApp/local/data/ui/views/rob.xml'
To reslove the issue, update the permissions of the file and apply the shcluster-bundle again
I ran into this same issue, but the Splunk startup messages on the deployer were not showing me any errors. I had to run strace on the main splunkd process to find the EACESS errors.
I downvoted this post because this answer does not answer the original issue.
I have a thread with a little more details regarding the bundle created on the local deployer instance that Splunk cannot find for some reason.
https://answers.splunk.com/answers/331300/why-am-i-getting-error-while-creating-deployable-a.html
I have support case 293707 open.
Note: the permission denied message generated at startup will not appear if splunk has read access to the file. In the case above the splunk user did not have R/W access to the file. Running a test where splunk ONLY had read access to the file resulted in a sucessful push of the configuration bundle.