Getting Data In

Will splunk reindex the same file with new data if the file was overwritten?

reggie_123
Explorer

Hi,

What will splunk behave like in the two following cases:
1) File A.log, having the lines:
1
2
3
Someone overwrites the file and places a new file with the same name A.log, having the lines:
1
2
3
4
5
Will lines 1 2 3 be indexed again? Will only the lines 4-5 be indexed after the overwriting takes place? Will Splunk even keep track of the

file after it was overwritten or lines 4-5 will not be indexed?
2) File A with rows as above, read and deleted using sinkhole policy by splunk. Afterwards new file A.log is created with rows like above. Will lines 1 2 3 be indexed again? Will the lines 4-5 be indexed after the same file reappears?

gjanders
SplunkTrust
SplunkTrust

Splunk checksums the file contents, as per the link ddrillic posted you can tweak the crcSalt, and the CHECK_METHOD, furthermore you can also tweak the initCrcLength in the inputs.conf

In the example your providing above, I would expect Splunk to re-index the entire file as the file was overwritten, however it should be straightforward to test it...

0 Karma

ddrillic
Ultra Champion
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.
Get Updates on the Splunk Community!

.conf25 Global Broadcast: Don’t Miss a Moment

Hello Splunkers, .conf25 is only a click away.  Not able to make it to .conf25 in person? No worries, you can ...

Observe and Secure All Apps with Splunk

 Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

What's New in Splunk Observability - August 2025

What's New We are excited to announce the latest enhancements to Splunk Observability Cloud as well as what is ...