Deployment Architecture

Bundle Replication: Problem replicating config (bundle) to search peer


I am very frequently getting below warning on UI:
"Bundle Replication: Problem replicating config (bundle) to search peer 'servername:8089', HTTP response code 413 (HTTP/1.1 413 Content-Length of 3174266880 too large (maximum is 3145728000)). Content-Length of 3174266880 too large (maximum is 3145728000) (Unknown write error)"

With a bit of finding and analysis, i have increased max_content_length under [http server] in $SPLUNK_HOME/etc/system/default/server.conf from 2GB to 8GB.
Also * files were getting coagulated at $SPLUNK_HOME/var/run/, because the size of bundle was 3.01GB and what i have set in distsearch.conf (maxbundlesize) is 3gb, so i have increased that size to 4gb.

But still my splunk directory is wholly consumed, 100% utilization.

Kindly suggest if you have any comments on this please.

Ultra Champion

What version of Splunk are you running? There have been some specific versions with bugs related to this.

Have you tried manually cleaning up that folder?

Have you tried shrinking your bundle? Do you have some very large lookups in there or so? Do you really need those on the indexers (you do if you want to use them in automatic lookups)? If not, blacklist them. Also, avoid including apps with big binary components in them, that are of no use on SH/IDX layers (e.g. scripts/binaries used for data input on a HF or so).

0 Karma


Hey @FrankVl thanks for dropping by,
This is Splunk Enterprise 6.6.3.

Our search head is Standalone, on-prem, and indexer is on cloud.

I am just wondering does max_content_length is set to 3145728000 on Indexer?

I am about to reduce the bundle size, but if i have set
max_content_lenght=8Gb in server.conf &
maxbundlesize = 4gb in distsearch.conf.

Then ideally these bundles should parse from Search Head to Indexer. And should not give error as maximum is 3145728000.

0 Karma

Ultra Champion

Not sure if I get your question correct, but yes, I do believe you need to set that max_content_lenght on the indexers. It is the indexers that are rejecting the oversized bundle coming from the Search Heads.

You might also want to double check using btool, that those settings are taken correctly.

0 Karma
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...