Getting Data In

What is the best practice to address the "..is neither in the bundle downloaded from master nor managed by local deployment client..." validation failure?

sloshburch
Splunk Employee
Splunk Employee

Two indexes are failing bundle validation checks on my cluster master with this error message:

[Critical] App='system' with replicated index='_introspection' is neither in the bundle downloaded from master nor managed by local deployment client. Either define this index at the master or specify repFactor=0 on peer to skip replication.

[Critical] App='system' with replicated index='_metrics' is neither in the bundle downloaded from master nor managed by local deployment client. Either define this index at the master or specify repFactor=0 on peer to skip replication.

1 Solution

sloshburch
Splunk Employee
Splunk Employee

The message is sent to the cluster master from the search peers. The message means that there are indexes on the peer nodes that are set for replication, but are not defined in the cluster master's configuration bundle.

The indexes are flagged for replication because the repFactor = auto. This setting is not set in system/default out of the box so it's likely set in other apps, perhaps as part of a default stanza.

You can find out for sure by running the btool command $SPLUNK_HOME/bin/splunk btool indexes list _metrics --debug | grep repFactor where _metrics can also be _introspection. The result will show only the configuration file where repFactor is defined.

Two potential work-arounds are:

  • define a stanza for those indexes within the cluster master bundle for deployment to the peer nodes
  • apply the cluster bundle on the command line by skipping validation

Add a stanza to the bundle

To get the validation to pass, add a definition of the indexes that are throwing the error to the cluster master configuration bundle. For example, add these stanzas to $SPLUNK_HOME/master-apps/_cluster/local/indexes.conf on the master node and push the bundle again:

[_metrics]
repFactor = 0

[_introspection]
repFactor = 0

Skip validation

The cluster bundle can be forced to skip validated on the command line interface. The entire validation functionality protects against breaking indexers or losing data with faulty configuration, therefore it is discouraged to circumvent this protection given the ease of the other workaround options available for this. Learn more about validation and how to skip it, as a last resort, at Update common peer configurations and apps in the Managing Indexers and Clusters of Indexers documentation.

Learn more

Learn more about the repFactor setting in Configure the peer indexes in an indexer cluster of the Managing Indexers and Clusters of Indexers documentation.

Learn more about using btool at Use btool to troubleshoot configurations within the Troubleshooting Manual.

View solution in original post

0 Karma

sloshburch
Splunk Employee
Splunk Employee

The message is sent to the cluster master from the search peers. The message means that there are indexes on the peer nodes that are set for replication, but are not defined in the cluster master's configuration bundle.

The indexes are flagged for replication because the repFactor = auto. This setting is not set in system/default out of the box so it's likely set in other apps, perhaps as part of a default stanza.

You can find out for sure by running the btool command $SPLUNK_HOME/bin/splunk btool indexes list _metrics --debug | grep repFactor where _metrics can also be _introspection. The result will show only the configuration file where repFactor is defined.

Two potential work-arounds are:

  • define a stanza for those indexes within the cluster master bundle for deployment to the peer nodes
  • apply the cluster bundle on the command line by skipping validation

Add a stanza to the bundle

To get the validation to pass, add a definition of the indexes that are throwing the error to the cluster master configuration bundle. For example, add these stanzas to $SPLUNK_HOME/master-apps/_cluster/local/indexes.conf on the master node and push the bundle again:

[_metrics]
repFactor = 0

[_introspection]
repFactor = 0

Skip validation

The cluster bundle can be forced to skip validated on the command line interface. The entire validation functionality protects against breaking indexers or losing data with faulty configuration, therefore it is discouraged to circumvent this protection given the ease of the other workaround options available for this. Learn more about validation and how to skip it, as a last resort, at Update common peer configurations and apps in the Managing Indexers and Clusters of Indexers documentation.

Learn more

Learn more about the repFactor setting in Configure the peer indexes in an indexer cluster of the Managing Indexers and Clusters of Indexers documentation.

Learn more about using btool at Use btool to troubleshoot configurations within the Troubleshooting Manual.

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...