topic How can we automatically delete old files we're monitoring? in Monitoring Splunk

How can we automatically delete old files we're monitoring?

uagraw01 — Mon, 13 Mar 2023 13:18:56 GMT

Hello Splunkers!!

As mentioned below we have these two files which are carrying payload events which we monitored already. But on a daily basis with new timestamp we want to monitor new files and delete the existing monitored files from that path. Is there any mechanism to achieve this.

WPLAT_order_2023-03-07T14-35-21.669Z.json
WPLAT_order_2023-03-08T15-45-30.232Z.json

Suppose for example :

day 1 : under D:\\splunk folder we are monitoring below two files.

WPLAT_order_2023-03-07T14-35-21.669Z.json
WPLAT_order_2023-03-08T15-45-30.232Z.json

day 2: we need to delete day 1 files from folder D:\\ and monitor the new created files with new timestamps.

WPLAT_order_2023-03-09T11-35-21.669Z.json
WPLAT_order_2023-03-10T12-45-30.232Z.json

Re: monitoring

gcusello — Thu, 09 Mar 2023 07:43:23 GMT

Hi @uagraw01,

as you can read at https://docs.splunk.com/Documentation/Splunk/9.0.4/Admin/Inputsconf you can use the "batch" command instead "monitor", so, Splunk deletes the file after reading.

So you have to create an input like the following:

[batch:///<your_path>/WPLAT_order_*.json] disabled = 0 index = <your_index> sourcetype = <your_sourcetype>

Ciao.

Giuseppe

Re: monitoring

uagraw01 — Thu, 09 Mar 2023 10:33:19 GMT

@gcusello we need to mention

move_policy = sinkhole

Re: monitoring

gcusello — Thu, 09 Mar 2023 10:57:07 GMT

Hi @uagraw01,

ok, find you configuration so you learn how to do and add your additional configurations to your inputs.conf.

Ciao.

Giuseppe