Why do we get kernel:BUG: soft lockup on the Search Head?

[ddrillic]

We get the following -

$ ./splunk status

Message from syslogd@<host> at Oct 11 06:02:24 ...
kernel:BUG: soft lockup - CPU#0 stuck for 90s! [setroubleshootd:6739]

Message from syslogd@<host>  at Oct 11 06:05:27 ...
kernel:BUG: soft lockup - CPU#0 stuck for 66s! [perfd:63790]

Message from syslogd@<host>  at Oct 11 06:05:27 ...
kernel:BUG: soft lockup - CPU#3 stuck for 68s! [kblockd/3:157]

Message from syslogd@<host>  at Oct 11 06:05:28 ...
kernel:BUG: soft lockup - CPU#8 stuck for 66s! [splunkd:2903]

[harsmarvania57]

Few questions

1. Are you running splunk on NFS ?
2. Are your splunk running on VMWare or Physical host?

[inventsekar · Accepted solution]

which linux version you are running..

googled for the kernel bug and found this -
https://access.redhat.com/solutions/2039183
it suggest the resolution as:
Update to the RHEL 6.7 kernel package (kernel-2.6.32-573.el6) or later

[ddrillic]

I see -

$ uname -a
Linux <host> 2.6.32-642.1.1.el6.x86_64 #1 SMP Fri May 6 14:54:05 EDT 2016 x86_64 x86_64 x86_64 GNU/Linux

So, we are at 2.6.32-642.1.1.el6.