Issues with Rollup Events

[SplunkDash]

Hello,

I have a Roll Up events. One file created every month and new events added up every day within that file. How would I avoid duplicate ingestion (or avoid same events to be indexed twice) for the same events as SPLUNK is using the same file to read and ingest? Any help will be highly appreciated. Thank you.

[gcusello · Accepted solution]

Hi @SplunkDash ,

if you don't use crcSalt = <SOURCE> option, Splunk recognizes already indexed events and it doesn't index them twice. even if they come from files with different filenames.

The only situation where the same logs from different files are indexed is using the above option in inputs.conf.

Ciao.

Giuseppe

[SplunkDash]

Hello @gcusello 

I posted a new question here (in following link), would it be possible to have your recommendation when you get a chance, thank you so much.

Re: Field Extraction -Key/ Value Pairs with Specia... - Splunk Community

[SplunkDash]

Hello @gcusello,

Thank you so much for your respond, it's answered major part of my question. But, other part of my question, new records/events added everyday within the same file (like added at the end of the same file)), how SPLUNK will ingest/treat those new events/records as those new events will be within the same file?

[yeahnah]

Hi @SplunkDash 

I've read your question a couple of times but I still do not really understand what you are asking.  Please expand on the situation you describe and provide examples.

If it is just that a file is renamed each month in a monitored folder, then this is OK, as Splunk does not track files by only their filename.  It also check sums the first 256 bytes of the head of the file, so if it is renamed then it knows it has already ingested it. 

Anyway, best to describe your concern better so the correct answer can be provided. 

[SplunkDash]

Hello @yeahnah 

Thank you so much for your quick response. My question was, new records/events added everyday within the same file (like at the end of the same file)), how SPLUNK will ingest those new events/records?