topic Re: Why am I unable to apply search head cluster bundle? in Deployment Architecture

Why am I unable to apply search head cluster bundle?

martaBenedetti — Tue, 24 Jan 2023 16:49:45 GMT

Hi community,

I've just performed an upgrade on my infrastructure (distributed environment) from Splunk 8.2.3 to Splunk 9.0.3.

All the instances seem to work fine, I have problems though in applying search head cluster bundle.

I use this command to upgrade Splunk Enterprise Security:

$SPLUNK_HOME/bin/splunk apply shcluster-bundle -preserve-lookups true -target https://instance1:8089

But it doesn't work and I receive this message:

Error while deploying apps to first member, aborting apps deployment to all members: Error while updating app=SplunkEnterpriseSecuritySuite on target=https://instance1:8089: Error in JSON response: Unexpected EOF

Do you have any idea of what could be the problem?

Thank you

Marta

Re: Why am I unable to apply search head cluster bundle?

gcusello — Tue, 24 Jan 2023 18:08:39 GMT

Hi @martaBenedetti,

I encountered a similar problem (not on ES) caused by too few disk space on the Deployer.

But anyway, immediately open a case to Splunk Support.

Ciao.

Giuseppe

Re: Why am I unable to apply search head cluster bundle?

kvm — Tue, 18 Apr 2023 01:50:05 GMT

Hi @gcusello

What was the root cause & solution? and which Splunk version were you using?

Re: Why am I unable to apply search head cluster bundle?

gcusello — Tue, 18 Apr 2023 06:52:59 GMT

Hi @kvm,

it's always better to open a new question instead append it to an existing one, event if with the same topic because because less people should answer to your question.

Anyway, the root cause were two:

not enough disk space on the Deployer,
there were many large lookups.

So when the Deployer prepared the bundle to deploy it didn't have enough space.

You can solte this problem in three ways:

remove old versions of the lookups from the apps to deploy,
add more disk to the Deployer,
use an option in deploy to avoid the lookups (http://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Limittheknowledgebundlesize?_ga=2.93053597.1009283784.1678368092-1538999818.1678368092&_gac=1.81124709.1680933931.Cj0KCQjw_r6hBhDdARIsAMIDhV_WP5lduipB0sHkvGUaKgofFI3rQa07rHSlghT5wuSchN0xXZRa9BYaAuW9EALw_wcB&_gl=1*vym8po*_ga*MTUzODk5OTgxOC4xNjc4MzY4MDky*_ga_5EPM2P39FV*MTY4MTc5OTkxNS4xODUuMS4xNjgxODAwNzE5LjYwLjAuMA..

Ciao.

Giuseppe

Re: Why am I unable to apply search head cluster bundle?

skrivis — Wed, 10 May 2023 14:53:48 GMT

I found that I was getting that same error about "Unexpected EOF," but there was plenty of disk space on the deployer and all cluster members.

I finally tried restarting splunk on all of the SH cluster members and after that I was able to successfully push the bundle.

Re: Why am I unable to apply search head cluster bundle?

gcusello — Thu, 11 May 2023 07:15:08 GMT

Hi @skrivis,

if one answer solves your need, please accept one answer for the other people of Community or tell us how we can help you.

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the Contributors;-)

Re: Why am I unable to apply search head cluster bundle?

computermathguy — Mon, 28 Aug 2023 16:27:12 GMT

We got the same error for all the members of the cluster. When it occurred, we had to restart Splunk on each member.