splunk clustered index upgrade to version 6

[shervinfernando]

I upgraded my clustered index and after the upgrade /opt/splunk/bin/splunk show cluster-bundle-status shows all peer nodes with the status Detention.

Rolling restart of the peer nodes didn't fix this issue - can you let me know how to fix it

[Ellen]

Under Splunk 5 & 4, the default minimum free disk space is 2000MB.

http://docs.splunk.com/Documentation/Splunk/4.3.7/Admin/Setlimitsondiskusage
http://docs.splunk.com/Documentation/Splunk/5.0.5/Indexer/Setlimitsondiskusage

vs.

Splunk 6, The default minimum free disk space is 5000MB.

http://docs.splunk.com/Documentation/Splunk/6.0/Indexer/Setlimitsondiskusage

[shervinfernando]

Fixed it by adding following to server.conf file on the indexer reporting this error

[diskUsage]
minFreeSpace = 2000

[ChrisG]

This is also mentioned in the upgrade information: http://docs.splunk.com/Documentation/Splunk/latest/Installation/Aboutupgradingto6.0READTHISFIRST#We_...

[asetiawan · Accepted solution]

I experienced similar issue when my indexer is out of disk space.

Splunk 6 seems to require min 5GB free space by default. So, if you have less than 5GB space on version 5.x and upgraded to version 6, your indexer will stop receiving logs and see warning messages like:


skipped indexing of internal audit event will keep dropping events until indexer congestion is remedied. Check disk space and other issues that may cause indexer to block


You can still search indexed logs, but the indexer status on the cluster master's clustering dashboard will be "Detention"

[shervinfernando]

Fixed it by adding following to server.conf file on the indexer reporting this error

[diskUsage]
minFreeSpace = 2000