Deployment Architecture

splunk clustered index upgrade to version 6

shervinfernando
Explorer

I upgraded my clustered index and after the upgrade /opt/splunk/bin/splunk show cluster-bundle-status shows all peer nodes with the status Detention.

Rolling restart of the peer nodes didn't fix this issue - can you let me know how to fix it

1 Solution

asetiawan
Explorer

I experienced similar issue when my indexer is out of disk space.

Splunk 6 seems to require min 5GB free space by default. So, if you have less than 5GB space on version 5.x and upgraded to version 6, your indexer will stop receiving logs and see warning messages like:


skipped indexing of internal audit event will keep dropping events until indexer congestion is remedied. Check disk space and other issues that may cause indexer to block

You can still search indexed logs, but the indexer status on the cluster master's clustering dashboard will be "Detention"

View solution in original post

Ellen
Splunk Employee
Splunk Employee

shervinfernando
Explorer

Fixed it by adding following to server.conf file on the indexer reporting this error

[diskUsage]
minFreeSpace = 2000

0 Karma

ChrisG
Splunk Employee
Splunk Employee
0 Karma

asetiawan
Explorer

I experienced similar issue when my indexer is out of disk space.

Splunk 6 seems to require min 5GB free space by default. So, if you have less than 5GB space on version 5.x and upgraded to version 6, your indexer will stop receiving logs and see warning messages like:


skipped indexing of internal audit event will keep dropping events until indexer congestion is remedied. Check disk space and other issues that may cause indexer to block

You can still search indexed logs, but the indexer status on the cluster master's clustering dashboard will be "Detention"

shervinfernando
Explorer

Fixed it by adding following to server.conf file on the indexer reporting this error

[diskUsage]
minFreeSpace = 2000

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...