Why is batch processing not removing files after indexing them in Splunk 6.2.1?

[jizzmaster]

I have an app that is not removing/deleting the files after consuming them. They are indexed appropriately, but just not deleted afterwards.

inputs.conf

[batch:///opt/splunk/etc/apps/my-special-app/pickup/*.json]
index = test
sourcetype = nessus_json
move_policy = sinkhole

I have tested this on a second Splunk box and the exact same app will correctly remove the files after indexing them. I can't tell where the issue may be on this main Splunk box, however. Any suggestions?

On Splunk v6.2.1. This worked a month or so ago. I'd rather figure out the cause before moving to upgrade the Splunk instance.

[micahkemp]

I forced a permissions issue with a file such that the Splunk user had read, not write permissions to a file that was configured as a batch input. It resulted in this line in splunkd.log:

11-24-2017 22:49:10.062 +0000 ERROR TailReader - Unable to remove sinkhole file: path=/tmp/batch_del_fail.log, errno=Operation not permitted

Can you look for a similar message to verify if it is a simple permissions issue?

[samhays]

Sounds like the permissions on the filesystem got screwed up. Check to make sure the account splunk is running as can actually delete the files.