"Indexer was started dirty"

[krussell101]

I have no clue what this error means.

The entire error in the splunkd.log is:

Indexer was started dirty, searches may not be accurate. Consider restarting Splunk and accepting the recovery request.

When I stop and restart splunk I'm not offered a recovery option.

I am getting these on virtually every server where I'm running splunk. Heavy forwarders and the indexer itself. The only exception are the two servers where I am running universal forwarders.

What does it mean and how do I clear it?

Thanks!!!

[Drainy · Accepted solution]

Have a read of;
http://docs.splunk.com/Documentation/Splunk/latest/admin/HowSplunkstoresindexes#Troubleshoot_your_bu...

It sounds like you need to do a complete fsck of your buckets, this can take a few hours though depending on how big they are so set aside some time for it. It sounds like Splunk isn't being shut down cleanly or the servers are crashing out.

[krussell101]

perfect! The page you reference suggests splunk fsck with the rebuild option.

I ran it with --repair --all on each server and that did the trick.

On several of the servers, there were no errors when splunk was started (before running fsck). So the only evidence of a problem was the log entry.

Interesting.

At any rate. Thanks very much for taking the time to help.

Much appreciated.