Splunk-optimize Warning ...

[MikeyG]

Can't find a reference to the following error. What does it mean and how do I fix it?

Indexing Significant Warns:

WARN timeinvertedIndex - splunk-optimize failed to start for index /opt/splunk/var/lib/splunk/defaultdb/db/hot_quar_v1_17

[tpaulsen]

The splunk-optimize process can´t run on that subdirectory, since it doesn´t exist. Even if i create it manually, splunk-optimize won´t notice, except by creating another error:

05-11-2010 13:10:40.476 ERROR databasePartitionPolicy - Index is empty refusing to move. oldDirPath=/opt/splunk/splunk/var/lib/splunk/fishbucket/db/db-hot

The other message is still there:

05-11-2010 14:33:52.045 WARN  timeinvertedIndex - splunk-optimize failed to start for index /opt/splunk/var/lib/splunk/fishbucket/db/db-hot

[gkanapathy · Accepted solution]

There is nothing to fix if it's rare. It just means that Splunk was busy at the time it would otherwise have run an optimization on the indexed data. Optimization runs frequently to improve the way data is stored in the index as new data gets added.

If the warning occurs regularly, it is a sign that your system is overloaded. If the warning occurs more often than every few minutes, your indexed data may not be well optimized which will lead to slower searches over that data.

[Mick]

Sporadic failures are to be expected, as there are times when Splunk will be indexing heavily to a particular hot DB, and it won't always be the optimal time for splunk-optimize to run on that particular bucket.

If it's a consistent failure however, and splunk-optimize has never been able to run on that bucket, that may indicate a more serious problem with the data inside the bucket - a possible data corruption for example.

If it's a consistent message, you should file a case with the Splunk Support team and they will work with you to determine the root cause - http://www.splunk.com/page/submit_issue

[jrodman]

If it's expected, why is it a failure?
What does it mean that it isn't an optimal time, is this a locking issue?