topic Re: How do I forward and delete logs? in Getting Data In

How do I forward and delete logs?

nick405060 — Thu, 29 Aug 2019 23:11:38 GMT

I would like to be able to forward logs and then delete them using a UF. How can I do this?

For the sake of the Splunk community, it would be nice if this question had a run-anywhere solution. However, I will also detail my use case specifically.

I am using Windows Event Forwarding (WEF) to collect 4800/4801 Windows security logs from 2000 of our workstations into a Windows Event Collector (WEC) that has a UF on it. I only spun up the WEC VM with an 80GB disk, as there is no reason to assign more disk space to merely a collection node, and storage is money. I can forward the logs from the WEC without a problem, but I need to be able to purge the logs after forwarding.

Re: How do I forward and delete logs?

nick405060 — Thu, 29 Aug 2019 23:33:06 GMT

Looks like the answer is a batch input

https://docs.splunk.com/Documentation/Splunk/7.3.1/Data/Monitorfilesanddirectorieswithinputs.conf#Batch

Re: How do I forward and delete logs?

mayurr98 — Thu, 29 Aug 2019 23:55:19 GMT

@nick405060 , Indeed it is

according to the description of "batch" in inputs.conf.spec , you should set move_policy = sinkhole .

move_policy = sinkhole
 * IMPORTANT: This attribute/value pair is required. You *must* include "move_policy = sinkhole" when defining batch inputs.
 * This loads the file destructively.  
 * Do not use the batch input type for files you do not want to consume destructively.
 * As long as this is set, Splunk won't keep track of indexed files. Without the "move_policy = sinkhole" setting,  it won't load the files destructively and will keep a track of them.