topic Re: How to use batch in inputs.conf file to upload files? in Getting Data In

How to use batch in inputs.conf file to upload files?

SplunkDash — Mon, 13 Jun 2022 20:31:32 GMT

Hello,

I have some use cases where we need to delete files right after those are read/push by UF. How I would do it. There are any ways we may let the UF to do this task using batch in inputs.conf file. Any recommendation would be highly appreciated, thank you!

Re: How to use batch in inputs.conf file to upload files?

m_pham — Tue, 14 Jun 2022 15:26:53 GMT

See the batch stanza configs below:

Use the 'batch' input for large archives of historic data. If you
want to continuously monitor a directory or index small archives, use 'monitor'
(see the MONITOR section). 'batch' reads in the file and indexes it, and then
deletes the file on disk.

[batch://<path>]
* A one-time, destructive input of files in <path>.
* This stanza must include the 'move_policy = sinkhole' setting.
* This input reads and indexes the files, then DELETES THEM IMMEDIATELY.
* For continuous, non-destructive inputs of files, use 'monitor' instead.

https://docs.splunk.com/Documentation/Splunk/latest/Admin/inputsconf#BATCH_.28.22Upload_a_file.22_in_Splunk_Web.29:

Re: How to use batch in inputs.conf file to upload files?

SplunkDash — Tue, 14 Jun 2022 17:46:00 GMT

Hello,

Thank you so much for your response and truly appreciate it.

Do you think following is the typical structure for inputs.conf file using batch and going to work?

[batch:///home/mydatafolder/*.log]

disabled = false

index = myindex

sourcetype = mysourcetpye

move_policy = sinkhole

Your recommendation will be highly appreciated, thank you so much again.

Re: How to use batch in inputs.conf file to upload files?

m_pham — Tue, 14 Jun 2022 17:50:36 GMT

It should work fine. Since the logs will be deleted, best you double check the file path and index/sourcetype names.