Getting Data In

How to set up a stanza batch to index same file?

arthurf
Explorer

Hello,

I set up my batch stanza to delete the file once indexed, but sometime, the file can't be due to permission error and the same file is indexed multiple time.

I used both of the following options:

initCrcLength = 2000
crcSalt = 

But the same file is still indexed again.
How can I set up the stanza not to read the same file?
Is that possible with batch stanza which deletes the files? (move_policy = sinkhole)

0 Karma

tiagofbmm
Influencer

The move_policy MUST be sinkhole for that to work as you want.

move_policy = sinkhole
* This setting is required. You must include "move_policy = sinkhole"
when you define batch inputs.

arthurf
Explorer

The move_policy is set.
Here my actual configuration :

[batch://FOLDER/Pattern]
sourcetype = fluxeditique:cabestan:dtr:in:planapurement3
index=cabestan
initCrcLength = 2000
move_policy = sinkhole

I replaced "crcSalt = " with "initCrcLength = 2000" but the result is the same.

0 Karma

tiagofbmm
Influencer

CRC settings belong to the monitor::// stanzas, not the batch ones.

It seems you need to take care of the deleting permissions of the files for Splunk user to be able to delete them.

arthurf
Explorer

Ok, so if get you correctly, it's not possible to prevent the indexation of the same file with batch stanza ?
And we need to ensure that the same log file is not generated twice, right ?

0 Karma

tiagofbmm
Influencer

Yes, with batch you don't get that possibility

arthurf
Explorer

Thanks for the confirmation.

Have a good day.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...