Batch file Input not removing files.

[kenoski]

I am attempting to use Splunk to remove the Oracle WebLogic files that are filling up our harddrive.

I have been able to remove other files with a different filename format using the batch command.

But... the following stanza is not working

[batch://C:\Oracle\config\domains\csel\servers\...\DefaultAuditRecorder.*.log]

The filename format is: DefaultAuditRecorder.############.log

where # is a number

Any suggestions?

[PickleRick]

Read the inputs.spec carefuly 😉

* This stanza must include the 'move_policy = sinkhole' setting.
* This input reads and indexes the files, then DELETES THEM IMMEDIATELY.

[kenoski]

I use the move_policy. I have tried the following, and it acts the same way.

To monitor for log files I have this in inputs.conf

[monitor://C:\Oracle\config\domains\csel\servers\...\logs\*.logs]

I have tried both of the following to batch the archived files.

1st try:

[batch://C:\Oracle\config\domains\csel\servers\...\DefaultAuditRecorder\[0-9]*.log]
move_policy = sinkhole
crcSalt = <SOURCE>

2nd try:

[batch://C:\Oracle\config\domains\csel\servers\...\]
whitelist = /DefaultAuditRecorder\.[0-9]+\.log$
move_policy = sinkhole
crcSalt = <SOURCE>

I even tried to blacklist the monitor stanza for the files I whitelist in the batch

[monitor://C:\Oracle\config\domains\csel\servers\...\]
blacklist = /DefaultAuditRecorder\.[0-9]+\.log$

Splunk still seems to try and monitor these files, and not batch them.

[PickleRick]

1. Did you check

splunk list monitor

and

splunk list inputstatus

2. This might not be related but batch input does not have crcSalt parameter (it makes no sense in batch input context at all).

3. Ok, so you have two separate file inputs covering the same path? That might be the problem.

[richgalloway]

Does the account running Splunk have permission to delete the files?  Are there any messages in splunkd.log about the files?

[kenoski]

There is no problem with removing files from the directory. Other files are being removed using batch.

This appears to be a regular expression processing issue.

SplunkD log shows the watch being put on the path, and processes the stanzas that relate to the files in question.

The file I want to monitor has the filename of DefaultAuditRecorder.log.

The files I want to use batch on have the form of DefaultAduitRecorder.############.log

The automatic Splunk conversion to Regular expression can't differentiate between these tow filename formats, and defaults to monitor.

I have tried several attempts to working on a whitelist regular expression for the monitor and batch, but it still doesn't work.