Deployment Architecture

ERROR DispatchReaper - Failed to reap

scheckenbachb
Explorer

We had to restore the Splunk indexer from a backup, loosing the 'colddb'. I now see the folowing ERROR in splunk.log, anything I need to worry about?

10-11-2018 11:22:50.108 +0200 ERROR DispatchReaper - Failed to reap D:\Program Files\Splunk\var\run\splunk\dispatch\remote_pvpsvgs6c01_scheduler_nobody_RUNCLUF1ZGl0RMD524612f09816f458b_at_1536120900_323 because of The directory is not empty.
10-11-2018 11:22:50.117 +0200 ERROR DispatchReaper - Failed to reap D:\Program Files\Splunk\var\run\splunk\dispatch\remote_PVPSVHC2C01_SummaryDirector_1536121209.679 because of The directory is not empty.
10-11-2018 11:23:20.074 +0200 ERROR DispatchReaper - Failed to reap D:\Program Files\Splunk\var\run\splunk\dispatch\remote_pvpsvgs6c01_scheduler
nobody_RUNCLUF1ZGl0_RMD524612f09816f458b_at_1536120900_323 because of The directory is not empty.
10-11-2018 11:23:20.091 +0200 ERROR DispatchReaper - Failed to reap D:\Program Files\Splunk\var\run\splunk\dispatch\remote_PVPSVHC2C01_SummaryDirector_1536121209.679 because of The directory is not empty.
10-11-2018 11:23:21.473 +0200 WARN DatabaseDirectoryManager - Directory='D:\Program Files\Splunk\var\lib\splunk_internaldb\colddb\911_8E7523CC-5821-453D-AE7E-7E46FB5189E5' does not exist. The directory representing this bucket might have just rolled.

Tags (1)
0 Karma

behlkush
Path Finder

Have you checked the ownership of directories: under $SPLUNK_HOME/var/run/dispatch. it might be the case that ownership is root and you are running splunk as splunk user.

 

This can happen when a user runs splunk as root by mistake and then later on runs as splunk. This is what happened in our case.

0 Karma

scheckenbachb
Explorer

It looks like we have a courrupted and unreadable *_323 directory. May I simple delete this?

0 Karma
Get Updates on the Splunk Community!

Building Reliable Asset and Identity Frameworks in Splunk ES

 Accurate asset and identity resolution is the backbone of security operations. Without it, alerts are ...

Cloud Monitoring Console - Unlocking Greater Visibility in SVC Usage Reporting

For Splunk Cloud customers, understanding and optimizing Splunk Virtual Compute (SVC) usage and resource ...

Automatic Discovery Part 3: Practical Use Cases

If you’ve enabled Automatic Discovery in your install of the Splunk Distribution of the OpenTelemetry ...