https://community.splunk.com/t5/Getting-Data-In/I-m-testing-splunk-and-when-I-edit-my-logfiles-splunk-doesn-t/m-p/11683#M682 <P>By default, Splunk detects changes in files by first checking the modification time. </P> <UL> <LI>If that has changed, it checks the first 256 bytes of the file <UL> <LI>If that is different from the last time it saw the file, it will index the file from the beginning.</LI> <LI>If it is the same as the last time it saw the file, it will check the last 256 bytes of the file. <UL> <LI>If it has changed, Splunk will index new events from the point that previously read.</LI> </UL></LI> </UL></LI> </UL> <P>Thus, if you change the file in the middle, Splunk may detect the modification times, but will not see any change at the beginning or end of the file, and therefore will index any part of the file anew.</P> Sat, 17 Apr 2010 14:25:46 GMT gkanapathy 2010-04-17T14:25:46Z https://community.splunk.com/t5/Getting-Data-In/I-m-testing-splunk-and-when-I-edit-my-logfiles-splunk-doesn-t/m-p/11682#M681 <P>I have a test logfile I fed into Splunk:</P> <P>Apr 13 10:41:16 support05 kernel: [1815783.556088] usb 2-1: new full speed USB device using uhci_hcd and address 32 Apr 13 10:41:16 support05 kernel: [1815783.699049] usb 2-1: not running at top speed; connect to a high speed hub</P> <P>and so on.</P> <P>Splunk consumed the file just fine.</P> <P>Then I opened the file and overwrote some text in the middle of the file. Splunk ignored my changes. Why didn't splunk re-index those lines?</P> Sat, 17 Apr 2010 08:52:36 GMT https://community.splunk.com/t5/Getting-Data-In/I-m-testing-splunk-and-when-I-edit-my-logfiles-splunk-doesn-t/m-p/11682#M681 jrodman 2010-04-17T08:52:36Z https://community.splunk.com/t5/Getting-Data-In/I-m-testing-splunk-and-when-I-edit-my-logfiles-splunk-doesn-t/m-p/11683#M682 <P>By default, Splunk detects changes in files by first checking the modification time. </P> <UL> <LI>If that has changed, it checks the first 256 bytes of the file <UL> <LI>If that is different from the last time it saw the file, it will index the file from the beginning.</LI> <LI>If it is the same as the last time it saw the file, it will check the last 256 bytes of the file. <UL> <LI>If it has changed, Splunk will index new events from the point that previously read.</LI> </UL></LI> </UL></LI> </UL> <P>Thus, if you change the file in the middle, Splunk may detect the modification times, but will not see any change at the beginning or end of the file, and therefore will index any part of the file anew.</P> Sat, 17 Apr 2010 14:25:46 GMT https://community.splunk.com/t5/Getting-Data-In/I-m-testing-splunk-and-when-I-edit-my-logfiles-splunk-doesn-t/m-p/11683#M682 gkanapathy 2010-04-17T14:25:46Z https://community.splunk.com/t5/Getting-Data-In/I-m-testing-splunk-and-when-I-edit-my-logfiles-splunk-doesn-t/m-p/11684#M683 <P>Fwiw, in 4.1 it's changes in modification time or file size.</P> Sat, 17 Apr 2010 15:07:23 GMT https://community.splunk.com/t5/Getting-Data-In/I-m-testing-splunk-and-when-I-edit-my-logfiles-splunk-doesn-t/m-p/11684#M683 jrodman 2010-04-17T15:07:23Z https://community.splunk.com/t5/Getting-Data-In/I-m-testing-splunk-and-when-I-edit-my-logfiles-splunk-doesn-t/m-p/11685#M684 <P>Er, if the last place it was in the file changed, it reindex the whole file too. If they both match and there's new data, it starts from that offset. <span class="lia-unicode-emoji" title=":winking_face:">😉</span></P> Sat, 17 Apr 2010 15:08:55 GMT https://community.splunk.com/t5/Getting-Data-In/I-m-testing-splunk-and-when-I-edit-my-logfiles-splunk-doesn-t/m-p/11685#M684 jrodman 2010-04-17T15:08:55Z