How to get Splunk to replace events containing "abc" with "xyz" without duplicating other events?

[jagdish007]

While continually indexing data from a file or directory, when I made some changes in file for eg. modified a single line having "abc" to "xyz", at that time, the entire file is indexed and I get the duplicate events of the entire file. I want Splunk to replace events that contain "abc" with "xyz" without duplicating other events.

[chris · Accepted solution]

Splunk does not keep track of every line it has ever read. And you can't modify records in Splunk like you can in a "normal" DB. Splunk works with checksums on files by default the first 256 and the last 256 bytes are used for the checksum (plus a pointer up to where a file was read). If the checksums do not match Splunk thinks it's a file it has never seen before and it will read it again.

Why are you changeing "abc" to "xyz" in your file? Maybe someone can tell you how to solve the problem with Splunk. Or maybe Splunk is not the right tool for you.

Regards
Chris

[jagdish007]

Thanx...Your answer helps me...

[chris]

No it will not reprocess the entire file. Splunk will know up to which point the file has been read and it will just read what is new. It nows that the first part of the file has ben read beacause of the checksums.

[jagdish007]

If I am continuous monitoring one LogFile which is edited by other application and that application added one more log to that LogFile. So when new log added to that LogFile at that time also entire file get processed?