Getting Data In

Batch file Input not removing files.

kenoski
Path Finder

I am attempting to use Splunk to remove the Oracle WebLogic files that are filling up our harddrive.

I have been able to remove other files with a different filename format using the batch command.

But... the following stanza is not working

 

[batch://C:\Oracle\config\domains\csel\servers\...\DefaultAuditRecorder.*.log]

 

The filename format is: DefaultAuditRecorder.############.log

where # is a number

 

Any suggestions?

Labels (2)
Tags (1)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Read the inputs.spec carefuly 😉

* This stanza must include the 'move_policy = sinkhole' setting.
* This input reads and indexes the files, then DELETES THEM IMMEDIATELY.
0 Karma

kenoski
Path Finder

I use the move_policy. I have tried the following, and it acts the same way.

To monitor for log files I have this in inputs.conf

[monitor://C:\Oracle\config\domains\csel\servers\...\logs\*.logs]

I have tried both of the following to batch the archived files.

1st try:

[batch://C:\Oracle\config\domains\csel\servers\...\DefaultAuditRecorder\[0-9]*.log]
move_policy = sinkhole
crcSalt = <SOURCE>

 

2nd try:

[batch://C:\Oracle\config\domains\csel\servers\...\]
whitelist = /DefaultAuditRecorder\.[0-9]+\.log$
move_policy = sinkhole
crcSalt = <SOURCE>

 

I even tried to blacklist the monitor stanza for the files I whitelist in the batch

[monitor://C:\Oracle\config\domains\csel\servers\...\]
blacklist = /DefaultAuditRecorder\.[0-9]+\.log$

Splunk still seems to try and monitor these files, and not batch them.

0 Karma

PickleRick
SplunkTrust
SplunkTrust

1. Did you check

splunk list monitor

and

splunk list inputstatus

2. This might not be related but batch input does not have crcSalt parameter (it makes no sense in batch input context at all).

3. Ok, so you have two separate file inputs covering the same path? That might be the problem.

richgalloway
SplunkTrust
SplunkTrust

Does the account running Splunk have permission to delete the files?  Are there any messages in splunkd.log about the files?

---
If this reply helps you, Karma would be appreciated.
0 Karma

kenoski
Path Finder

There is no problem with removing files from the directory. Other files are being removed using batch.

This appears to be a regular expression processing issue.

SplunkD log shows the watch being put on the path, and processes the stanzas that relate to the files in question.

The file I want to monitor has the filename of DefaultAuditRecorder.log.

The files I want to use batch on have the form of DefaultAduitRecorder.############.log

The automatic Splunk conversion to Regular expression can't differentiate between these tow filename formats, and defaults to monitor.

I have tried several attempts to working on a whitelist regular expression for the monitor and batch, but it still doesn't work.

 

0 Karma
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...