topic Re: Duplicate IIS event logs | WatchedFile - Checksum for seekptr didn't match in Getting Data In

Duplicate IIS event logs | WatchedFile - Checksum for seekptr didn't match

Mike737 — Thu, 24 Oct 2013 04:25:20 GMT

I'm receiving duplicate events from IIS logs being sent through the universal forwarder.

The forwardeds 'splunkd.log' is showing:

10-24-2013 14:45:02.882 +1100 INFO  WatchedFile - Checksum for seekptr didn't match, will re-read entire file='C:\path\to\iis\logs\u_ex131024.log'.
10-24-2013 14:45:02.882 +1100 INFO  WatchedFile - Will begin reading at offset=0 for file='C:\path\to\iis\logs\u_ex131024.log'.
10-24-2013 14:45:02.882 +1100 INFO  WatchedFile - Resetting fd  to re-extract header.

Splunk versions are:

Splunk 6.0.182037
Splunk universal forwarder 6.0.182611

inputs.conf

[monitor://C:\path\to\iis\logs\*.log]     
disabled = false    
sourcetype = iis

props.conf (as per universal forwarder defaults)

[iis]
pulldown_type = true 
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
INDEXED_EXTRACTIONS = w3c
detect_trailing_nulls = auto

Any ideas where I am going wrong?

Re: Duplicate IIS event logs | WatchedFile - Checksum for seekptr didn't match

mParticle — Mon, 04 Nov 2013 20:16:17 GMT

+1... Splunk indexer and UF both on 6.0.182037

inputs.conf

[monitor://C:\inetpub\logs\LogFiles\W3SVC1] sourcetype=iis index=iis_logs

props config

[iis] pulldown_type = true MAX_TIMESTAMP_LOOKAHEAD = 32 SHOULD_LINEMERGE = False INDEXED_EXTRACTIONS = w3c detect_trailing_nulls = auto

I also tried adding

initCrcLength = 1024 crcSalt = <SOURCE>

(crcSalt first by itself, then together with initCrcLength), neither is helping.

Re: Duplicate IIS event logs | WatchedFile - Checksum for seekptr didn't match

Mike737 — Tue, 05 Nov 2013 01:32:57 GMT

Glad to know someone else is facing the same issue

Re: Duplicate IIS event logs | WatchedFile - Checksum for seekptr didn't match

mParticle — Thu, 07 Nov 2013 14:49:37 GMT

Splunk guys, any suggestions? Anyone?

Re: Duplicate IIS event logs | WatchedFile - Checksum for seekptr didn't match

arvidn — Fri, 08 Nov 2013 20:08:57 GMT

We had the same problem with our IIS logs.
Think I have tried anything with UF version 6.0-82037 & 6.0-82611, upgrades and fresh install with different configurations (input.conf).
Uninstalled UF version 6 and reinstalled version 5.0.5-179365.
So far it has been stable, and no checksum error.

Splunk 6.0.182037 (indexer and heavy forwarder) &
Splunk Universal Forwarder 5.0.5-179365(again)

Re: Duplicate IIS event logs | WatchedFile - Checksum for seekptr didn't match

mParticle — Mon, 11 Nov 2013 16:09:19 GMT

Thanks arvidn! I tried this and so far the UF doesn't seem to get thrown in a loop, however the indexer doesn't parse the logs properly/automatically as it did with the 6.0 UF, so I am guessing some transforms are in order. Would you mind sharing what other conf file changes you have made on the UF/Indexer side to get this to work?

Re: Duplicate IIS event logs | WatchedFile - Checksum for seekptr didn't match

arvidn — Mon, 28 Sep 2020 15:15:03 GMT

On Indexer,.
Create or edit " $SPLUNK_HOME\etc\system\local\props.conf"
[iis]
TZ = GMT
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = True
REPORT - iis2 = iis2

Add more stanzas if nessesary (sample)
[u_ex-too_small]
rename = iis
TZ = GMT
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = True
REPORT - iis2 = iis2

[u_ex-2]
rename = iis
TZ = GMT
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = True
REPORT - iis2 = iis2

Create or edit " $SPLUNK_HOME\etc\system\local\transforms.conf"
[iis2]
DELIMS = " "
FIELDS = date, time(GMT), s-ip, cs-method, cs-uri-stem, cs-uri-query, s-port, cs-username, c-ip, cs(User-Agent), sc-status, sc-substatus, sc-win32-status, time-taken

I think this is default fields from IIS, add or remove if more or less fields are chosen.
Restart splunkd service

Re: Duplicate IIS event logs | WatchedFile - Checksum for seekptr didn't match

arvidn — Mon, 11 Nov 2013 21:05:06 GMT

Hi mParticle. You will find my answer below. Couldn’t comment it here, too many characters…..

Re: Duplicate IIS event logs | WatchedFile - Checksum for seekptr didn't match

mParticle — Wed, 13 Nov 2013 22:13:50 GMT

Excellent, thank you! It works perfectly. Hopefully Splunk fixes this in the next release...

Sorry for the delayed comment - the automated SplunkBase email went to my Junk folder and I just saw it...

Re: Duplicate IIS event logs | WatchedFile - Checksum for seekptr didn't match

mParticle — Wed, 13 Nov 2013 22:21:55 GMT

Just one note - I added these to the two files you mentioned above, so that the IIS log comments get removed from the results:

To each stanza in the props.conf:
TRANSFORMS-removecomments = removecomments

To the transforms.conf:
[removecomments]
REGEX = ^#.*
DEST_KEY = queue
FORMAT = nullQueue

Thanks again!

Re: Duplicate IIS event logs | WatchedFile - Checksum for seekptr didn't match

jbsplunk — Tue, 17 Dec 2013 19:10:15 GMT

This is a known issue with 6.0, SPL-77048. It is tentatively scheduled to be fixed in the forthcoming maintenance release, which will be post 6.0.1.

Re: Duplicate IIS event logs | WatchedFile - Checksum for seekptr didn't match

ekost — Thu, 06 Mar 2014 01:41:09 GMT

FIxed in 6.0.2: http://docs.splunk.com/Documentation/Splunk/6.0.2/ReleaseNotes/6.0.2