Why am I unable to apply search head cluster bundle?

[martaBenedetti]

Hi community,

I've just performed an upgrade on my infrastructure (distributed environment) from Splunk 8.2.3 to Splunk 9.0.3.

All the instances seem to work fine, I have problems though in applying search head cluster bundle.

I use this command to upgrade Splunk Enterprise Security:

$SPLUNK_HOME/bin/splunk apply shcluster-bundle -preserve-lookups true -target https://instance1:8089

But it doesn't work and I receive this message:

Error while deploying apps to first member, aborting apps deployment to all members: Error while updating app=SplunkEnterpriseSecuritySuite on target=https://instance1:8089: Error in JSON response: Unexpected EOF

Do you have any idea of what could be the problem?

Thank you

Marta

[gcusello · Accepted solution]

Hi @martaBenedetti,

I encountered a similar problem (not on ES) caused by too few disk space on the Deployer.

But anyway, immediately open a case to Splunk Support.

Ciao.

Giuseppe

[kvm]

Hi @gcusello 

What was the root cause & solution? and which Splunk version were you using?

[gcusello]

Hi @kvm,

it's always better to open a new question instead append it to an existing one, event if with the same topic because because less people should answer to your question.

Anyway, the root cause were two:

• not enough disk space on the Deployer,
• there were many large lookups.

So when the Deployer prepared the bundle to deploy it didn't have enough space.

You can solte this problem in three ways:

• remove old versions of the lookups from the apps to deploy,
• add more disk to the Deployer,
• use an option in deploy to avoid the lookups (http://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Limittheknowledgebundlesize?_ga=2.9305....

Ciao.

Giuseppe

[skrivis]

I found that I was getting that same error about "Unexpected EOF," but there was plenty of disk space on the deployer and all cluster members.

I finally tried restarting splunk on all of the SH cluster members and after that I was able to successfully push the bundle.

[computermathguy]

We got the same error for all the members of the cluster.  When it occurred, we had to restart Splunk on each member. 

[gcusello]

Hi @skrivis,

if one answer solves your need, please accept one answer for the other people of Community or tell us how we can help you.

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the Contributors;-)