topic Re: Duplicating events on .txt log file in Getting Data In

Duplicating events on .txt log file

julima — Tue, 03 Dec 2013 17:29:33 GMT

We have a Windows machine that writes events on a log with the .txt extension, monitored by the Splunk Universal Forwarder (monitor stanza). Every time the file changes, Splunk re-read it all and writes this to the splunkd.log:

12-03-2013 15:12:33.432 -0200 INFO  WatchedFile - File too small to check seekcrc, probably truncated.  Will re-read entire file='D:\path\to\file\file.txt'.
12-03-2013 15:12:33.432 -0200 INFO  WatchedFile - Will begin reading at offset=0 for file='D:\path\to\file\file.txt'.

We've noticed that this happens to all .txt files that we try to monitor with Splunk. It's like Splunk have a config to deal with .txt files on a different way.

We've tried to setup an props.conf with:

[source::D:\\path\\to\\file\\file.txt]
CHECK_METHOD = endpoint_md5

But Splunk still duplicates the events.

Have anyone seen something like? There is a way to config Splunk to not re-read .txt files on each update?

Thanks!
Julio

Re: Duplicating events on .txt log file

lukejadamec — Tue, 03 Dec 2013 17:41:45 GMT

Try adding a CRC Salt, see this post:

http://answers.splunk.com/answers/1568/windows-dhcp-log-files-too-small-to-match-seekptr-checksum

Re: Duplicating events on .txt log file

julima — Tue, 03 Dec 2013 18:13:59 GMT

Hi, lukejadamec.

We've already tried "crcSalt = " to the monitor stanza, but it didn't work =[

Now, our stanza uses only index and sourcetype attributes.

Re: Duplicating events on .txt log file

somesoni2 — Tue, 03 Dec 2013 19:52:02 GMT

just to be sure you added "crcSalt = " (sometime case makes difference) and restarted splunk instance?

Re: Duplicating events on .txt log file

julima — Wed, 04 Dec 2013 14:54:09 GMT

Actually we added "crcSalt = " (UPPERCASE). And yes, we restarted the Splunk instance.

Re: Duplicating events on .txt log file

lukejadamec — Wed, 04 Dec 2013 17:28:06 GMT

How large are these files, and there any changes other than at the end of the file?