Deployment Architecture

Bundle replication fails with: response_code=204

drrushi_splunk
Splunk Employee
Splunk Employee

The search-head fails to retrieve results from some/all search-peers and emits messages like so on the UI:

"Problem replicating config (bundle) to search peer 'peer_host:8089', got http response code 204 HTTP/1.1 204 No Content"

The search-head splunkd.log shows:

ERROR DistributedBundleReplicationManager - got non-200 response from peer.uri=https://peer_host:8089, reply="HTTP/1.1 204 No Content" response_code=204

1 Solution

drrushi_splunk
Splunk Employee
Splunk Employee

First check the peer's splunkd.log for any messages during the same time as the search-head's DistributedBundleReplicationManager error.

If you do find in the peer's splunkd.log messages such as:

ERROR DistBundleRestHandler - File users/xxx/yyy/local/props.conf in knowledge bundle is either not in white list or else excluded by black list. Bundle /opt/splunk/var/run/searchpeers/ will be removed

...then this means that there must be on the peer a rouge 'distsearch.conf' which does't not explicitly whitelist or blacklist any bundle files ... as a result by default the peer simply rejects the bundle.

To workaround this please remove any distsearch.conf (from system/local OR etc/apps/appname/local) on the peers and restart Splunk.

In version 6.1 a new functionality was added to the peer which allows peers to blacklist/whitelist bundle contents based on locally defined rules (via local distsearch.conf).

View solution in original post

drrushi_splunk
Splunk Employee
Splunk Employee

First check the peer's splunkd.log for any messages during the same time as the search-head's DistributedBundleReplicationManager error.

If you do find in the peer's splunkd.log messages such as:

ERROR DistBundleRestHandler - File users/xxx/yyy/local/props.conf in knowledge bundle is either not in white list or else excluded by black list. Bundle /opt/splunk/var/run/searchpeers/ will be removed

...then this means that there must be on the peer a rouge 'distsearch.conf' which does't not explicitly whitelist or blacklist any bundle files ... as a result by default the peer simply rejects the bundle.

To workaround this please remove any distsearch.conf (from system/local OR etc/apps/appname/local) on the peers and restart Splunk.

In version 6.1 a new functionality was added to the peer which allows peers to blacklist/whitelist bundle contents based on locally defined rules (via local distsearch.conf).

View solution in original post

Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.