Monitoring Splunk

Can splunk-optimize kill my box?

howyagoin
Contributor

Last night around 2345 a splunk-optimize process seems to have kicked off, and my server has essentially been unresponsive all day since. I will get remote hands on the box tomorrow at the soonest (grr, holiday weekends) but in the mean time I'm wondering if anyone else has seen this type of behaviour.

I have one working shell on the box at the moment, any further attempts to connect are met with failure, and other sessions hung after a few commands, so I'm extremely hesitant to do much till I can get to the console.

From ps:


root     64784 100.0  0.2  2595160  78012   ??  Rs   11:44PM 809:35.51 splunk-optimize -d /Volumes/foo/Applications/splunk/var/lib/splunk/zz/db/hot_v1_104

That %CPU varies, so whatever this process is seems to actually still be going. I tried (perhaps foolishly) to kill -9 it, but it made no difference.

I realise this is a longshot, but, just wondering if others have had issues with this process on particularly large indices.

For what it's worth, the OS is OS X 10.6.7.

Tags (1)
0 Karma
1 Solution

howyagoin
Contributor

I ended up rebooting. Killing the optimize didn't help, and even ps was getting wedged telling me about what was running. Oh well.

View solution in original post

0 Karma

howyagoin
Contributor

I ended up rebooting. Killing the optimize didn't help, and even ps was getting wedged telling me about what was running. Oh well.

View solution in original post

0 Karma

netwrkr
Communicator

killall -9 splunk ? It's probably chewing up all available disk io.

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!