Getting Data In

Can you auto remove CSV files after indexing?

Builder

Is there a configuration in Splunk where it can remove/move a CSV file after it has been indexed? so it does not show as active in the configured data input folder for being scanned as a valid file..( since its already indexed )

John.

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Hi jiaqya,

you are looking for the [batch://...] stanza in inputs.conf, here are the docs http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#BATCH_.28.22Upload_a_file.22_in_...

[batch://<path>]
* A one-time, destructive input of files in <path>.
* For continuous, non-destructive inputs of files, use 'monitor' instead.

# Additional settings:

move_policy = sinkhole
* IMPORTANT: This setting is required. You *must* include
  "move_policy = sinkhole" when you define batch inputs.
* This setting causes the input to load the file destructively.

Hope this helps ...

cheers, MuS

View solution in original post

SplunkTrust
SplunkTrust

Hi jiaqya,

you are looking for the [batch://...] stanza in inputs.conf, here are the docs http://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#BATCH_.28.22Upload_a_file.22_in_...

[batch://<path>]
* A one-time, destructive input of files in <path>.
* For continuous, non-destructive inputs of files, use 'monitor' instead.

# Additional settings:

move_policy = sinkhole
* IMPORTANT: This setting is required. You *must* include
  "move_policy = sinkhole" when you define batch inputs.
* This setting causes the input to load the file destructively.

Hope this helps ...

cheers, MuS

View solution in original post

Motivator

Hey@MuS,

What permission does the file require for destructive input?

0 Karma

SplunkTrust
SplunkTrust

If on nix you need write (+w) permissions, and parent directory should be accessible (+x) to the user which is you want to have delete permission.

If on Windows ... ¯\_(ツ)_/¯ sorry cannot help here, but I'm sure you will find something asking google.

cheers, MuS

Builder

Mus, Thanks, thats mostly what i want. But , is there an option to delete only beyond 7 days or 'n' number of days , so at least i retain few recent files...

0 Karma