We are trying to index NetApp XML audit logs. The look like this
<Events xmlns=blah>
<Event>stuff</Event>
<Event>stuff</Event>
<Event>stuff</Event>
<Event>stuff</Event>
</Events>
Unfortunately, new events are INSERTED BEFORE the final tag.
So the next time it reads the file, the value of scrc will be different and Splunk reindexes the entire file. The error message is "Checksum for seekptr didn't match, will re-read entire file"
This is expected behaviour from Splunk, but I'm wondering if anyone has managed to work around it? One method that comes to mind is can the seekptr be told to ignore a regex? If it ignored the final element, then there would be no scrc mismatch.
Any ideas?
Hello! Have you fixed it?
Are you monitoring the files directly? How about exclude the "filerXYZ-last.xml" and getting only the one that was already rotated? That could fix your issue, right?
You could set it to rotate every 5 minutes (or something like that) and keep only the 10, 15 files (loglimit).
If you need the cDot commands, I can help you with this.
Did you ever fix this? I have tried (to no avail):
INPUTS (only monitoring last .xml file)
[ontap]
initCrcLength = 2048
multiline_event_extra_waittime = true
disabled = 0
sourcetype = ontap
index = ontap
PROPS:
[ontap]
SHOULD_LINEMERGE = false
KV_MODE = xml
LINE_BREAKER = ()
MUST_BREAK_AFTER = \
TRANSFORMS-t1 = remove_header_footer
TRANSFORMS:
[remove_header_footer]
REGEX=^<(\/|)Events(\s|>)
DEST_KEY = queue
FORMAT = nullQueue
Still getting
WatchedFile - Checksum for seekptr didn't match, will re-read entire file=(.xml file)
WatchedFile - Will begin reading at offset=0 for file= (.xml file)
CORRECTION:
LINE_BREAKER = (<Event>)
MUST_BREAK_AFTER = \</Event\>
We ended up reading the rotated log files instead of the live file, as there is no way to manipulate seekptr
Hello! Have you fixed it?
Are you monitoring the files directly? How about exclude the "filerXYZ-last.xml" and getting only the one that was already rotated? That could fix your issue, right?
You could set it to rotate every 5 minutes (or something like that) and keep only the 10, 15 files (loglimit).
If you need the cDot commands, I can help you with this.
Hi,
We did in fact end up reading the rotated file. Works fine, but we miss being able to get real time info.