Monitoring Splunk

How can we automatically delete old files we're monitoring?

uagraw01
Motivator

Hello Splunkers!!

As mentioned below we have these two files which are carrying payload events which we monitored already. But on a daily basis with new timestamp we want to monitor new files and delete the existing monitored files from that path. Is there any mechanism to achieve this.

WPLAT_order_2023-03-07T14-35-21.669Z.json
WPLAT_order_2023-03-08T15-45-30.232Z.json

Suppose for example :

day 1 :  under D:\\splunk folder we are monitoring below two files.

WPLAT_order_2023-03-07T14-35-21.669Z.json
WPLAT_order_2023-03-08T15-45-30.232Z.json

day 2: we need to delete day 1 files from folder D:\\ and monitor the new created files with new timestamps.

WPLAT_order_2023-03-09T11-35-21.669Z.json
WPLAT_order_2023-03-10T12-45-30.232Z.json

 

 

 

0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @uagraw01,

as you can read at https://docs.splunk.com/Documentation/Splunk/9.0.4/Admin/Inputsconf you can use the "batch" command instead "monitor", so, Splunk deletes the file after reading.

So you have to create an input like the following:

[batch:///<your_path>/WPLAT_order_*.json]
disabled = 0
index = <your_index>
sourcetype = <your_sourcetype>

Ciao.

Giuseppe

View solution in original post

gcusello
SplunkTrust
SplunkTrust

Hi @uagraw01,

as you can read at https://docs.splunk.com/Documentation/Splunk/9.0.4/Admin/Inputsconf you can use the "batch" command instead "monitor", so, Splunk deletes the file after reading.

So you have to create an input like the following:

[batch:///<your_path>/WPLAT_order_*.json]
disabled = 0
index = <your_index>
sourcetype = <your_sourcetype>

Ciao.

Giuseppe

uagraw01
Motivator

@gcusello we need to mention 

move_policy = sinkhole

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @uagraw01,

ok, find you configuration so you learn how to do and add your additional configurations to your inputs.conf.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Now Playing: Splunk Education Summer Learning Premieres

It’s premiere season, and Splunk Education is rolling out new releases you won’t want to miss. Whether you’re ...

The Visibility Gap: Hybrid Networks and IT Services

The most forward thinking enterprises among us see their network as much more than infrastructure – it's their ...

Get Operational Insights Quickly with Natural Language on the Splunk Platform

In today’s fast-paced digital world, turning data into actionable insights is essential for success. With ...