Observed the following error on executing the following command on Indexer Cluster
splunk apply cluster-bundle. Need to remediate these issue in order to push the cluster to avoid --skip-validation
splunk@ulvlfimw01:~/bin> /opt/splunk/splunk/bin/splunk apply cluster-bundle
Warning: Under some circumstances, this command will initiate a rolling restart of all peers. This depends on the contents of the configuration bundle. For details, refer to the documentation. Do you wish to continue? [y/n]: y
Encountered some errors while applying the bundle.
In handler 'clustermastercontrol': The Master could not push the latest configuration bundle because it contains an invalid configuration. Fix any errors and push the bundle again. Alternatively, you can skip the validation process like this: "splunk apply cluster-bundle --skip-validation". Use this option carefully, as it can cause the master to push an invalid configuration to the peers. The following errors were encountered: No spec file for: /opt/splunk/splunk/etc/master-apps/PA-nmon/default/inputs_forWindows.conf ;No spec file for: /opt/splunk/splunk/etc/master-apps/PA-nmon/default/nmon.conf ;No spec file for: /opt/splunk/splunk/etc/master-apps/PA-nmon/default/props_forWindows.conf ;No spec file for: /opt/splunk/splunk/etc/master-apps/Splunk_TA_nix/default/eventgen.conf ; Value in stanza [app=/network/ntp:default] in /opt/splunk/splunk/etc/master-apps/Splunk_TA_nix/default/tags.conf, line 783 not URI encoded: app = /network/ntp:default ; Value in stanza [shell=/bin/bash] in /opt/splunk/splunk/etc/master-apps/Splunk_TA_nix/default/tags.conf, line 835 not URI encoded: shell = /bin/bash ; Value in stanza [shell=/bin/sh] in /opt/splunk/splunk/etc/master-apps/Splunk_TA_nix/default/tags.conf, line 838 not URI encoded: shell = /bin/sh ; Value in stanza [shell=/usr/bin/bash] in /opt/splunk/splunk/etc/master-apps/Splunk_TA_nix/default/tags.conf, line 841 not URI encoded: shell = /usr/bin/bash ; Value in stanza [shell=/usr/bin/pfksh] in /opt/splunk/splunk/etc/master-apps/Splunk_TA_nix/default/tags.conf, line 844 not URI encoded: shell = /usr/bin/pfksh ; Value in stanza [shell=/usr/bin/pfsh] in /opt/splunk/splunk/etc/master-apps/Splunk_TA_nix/default/tags.conf, line 847 not URI encoded: shell = /usr/bin/pfsh ;No spec file for: /opt/splunk/splunk/etc/master-apps/TemplateForXenApp/default/eventgen.conf
I am not able to figure out where to find the spec files for PA-nmon app as suggested in
Please suggest how to create these spec files?
The spec files are not available in the latest version of the app as well.
Yes I am referring to the same app.I could not find spec files as listed below:
No spec file for: /opt/splunk/splunk/etc/master-apps/PA-nmon/default/inputsforWindows.conf
;No spec file for: /opt/splunk/splunk/etc/master-apps/PA-nmon/default/nmon.conf
;No spec file for: /opt/splunk/splunk/etc/master-apps/PA-nmon/default/propsforWindows.conf
Also when I downloaded the app and checked for these spec files in README ,I was not able to find them.
I am the author of the Nmon Perf application.
To answer about the PA-nmon app, it is part of the Core Application (available in the resources directory), it is the addon version that must be deployed on indexers.
Currently, there are in deed no spec files, i am working on including spec files for next release.
Therefore, i am quite surprised as i have never encountered the error in indexer cluster deployment, i have read the other link (https://answers.splunk.com/answers/286887/cluster-master-cannot-push-configuration-bundle-du.html).
Have you applied any custom setting or command that activates this verification step ?
As far as i can see, there is lot of messages concerning others Apps.
You can rename:
/opt/splunk/splunk/etc/master-apps/PA-nmon/default/inputsforWindows.conf to /opt/splunk/splunk/etc/master-apps/PA-nmon/default/inputsforWindows.conf.spec
/opt/splunk/splunk/etc/master-apps/PA-nmon/default/propsforWindows.conf to /opt/splunk/splunk/etc/master-apps/PA-nmon/default/propsforWindows.conf.spec
Must kept in place, it used to manage default nmon options at startup.
Does creating an "nmon.conf.spec" (empty or copy the origin nmon.conf file) is enough for the message to disappear ?
Creating nmon.conf.spec did not help the issue.We still received the same errors.
Issues you are encountering while applying the cluster bundle songs quite strange to me.
Not only advertising messages concerning the Nmon App, but also all messages you show.
I could never reproduce this, neither my current client having multiple clusters running the PA-nmon, i have never seen this message.
Wouldn't have you increased splunkd verbosity ? Like changing log level messages ? Or any specific parameter that would increase these validation steps verbosity ?
Hey All, I just thought I would add some value to this discussion. I am wondering the version of Splunk you are running as I have come to find some issues with bundle replication in general in the original cluster releases.
The splunk version used currently is 6.2.3.