Splunk Search

How to get Splunk to replace events containing "abc" with "xyz" without duplicating other events?

Explorer

While continually indexing data from a file or directory, when I made some changes in file for eg. modified a single line having "abc" to "xyz", at that time, the entire file is indexed and I get the duplicate events of the entire file. I want Splunk to replace events that contain "abc" with "xyz" without duplicating other events.

Tags (3)
1 Solution

Motivator

Splunk does not keep track of every line it has ever read. And you can't modify records in Splunk like you can in a "normal" DB. Splunk works with checksums on files by default the first 256 and the last 256 bytes are used for the checksum (plus a pointer up to where a file was read). If the checksums do not match Splunk thinks it's a file it has never seen before and it will read it again.

Why are you changeing "abc" to "xyz" in your file? Maybe someone can tell you how to solve the problem with Splunk. Or maybe Splunk is not the right tool for you.

Regards
Chris

View solution in original post

Motivator

Splunk does not keep track of every line it has ever read. And you can't modify records in Splunk like you can in a "normal" DB. Splunk works with checksums on files by default the first 256 and the last 256 bytes are used for the checksum (plus a pointer up to where a file was read). If the checksums do not match Splunk thinks it's a file it has never seen before and it will read it again.

Why are you changeing "abc" to "xyz" in your file? Maybe someone can tell you how to solve the problem with Splunk. Or maybe Splunk is not the right tool for you.

Regards
Chris

View solution in original post

Explorer

Thanx...Your answer helps me...

0 Karma

Motivator

No it will not reprocess the entire file. Splunk will know up to which point the file has been read and it will just read what is new. It nows that the first part of the file has ben read beacause of the checksums.

0 Karma

Explorer

If I am continuous monitoring one LogFile which is edited by other application and that application added one more log to that LogFile. So when new log added to that LogFile at that time also entire file get processed?

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!