I'm getting an error for some files I am monitoring about hitting EOF while computing CRC. What does this mean?

[the_wolverine]

I'm seeing the following errors in splunkd.log and my file isn't being monitored properly -- the events don't seem to be coming in anymore!

FileInputTracker - Stored seekPtr longer than file: /opt/IBM/WebSphere/AppServer/myServer/SystemOut.log.
FileInputTracker - Read a similar file to this that was longer. Perhaps you have two files that have the same content.
FileInputTracker - Hit EOF while computing CRC: 0/256.
FileInputTracker - Couldn't get CRC for file: /opt/IBM/WebSphere/AppServer/myServer/SystemOut.log

What could be the issue?

[shirolu]

CrcSalt = I set the parameters in the input.conf

I have the right information into the index

But the data

The emergence of many duplicate information

How do I remove the duplicate information to

CT

[jrodman]

The "simialr file to this that was longer" message typically means you've got files with headers, so more than one looks the same, and when the log rolls, splunk is a bit unsure what the story is.

The "hit EOF while computing CRC: 0/256" means that, as you say, we failed to grab a CRC from the file, which could be because it's too short (under 256 bytes), or sometimes it can indicate weird read errors that splunk didn't expect.

If you're seeing these within milliseconds of each other, you may want to take a closer look at what's going on with the files before taking action.

[the_wolverine · Accepted solution]

This means that Splunk cannot get a correct read on the file to calculate the CRC. Or, the file may be the same as another file. To workaround, add the following setting to the appropriate monitor stanza in $SPLUNK_HOME/etc/system/local/inputs.conf: crcSalt = <SOURCE>

For example:

[monitor:///opt/IBM/WebSphere/AppServer/myServer/SystemOut.log]
crcSalt = <SOURCE>

*NOTE: the example above is literal/verbatim. You will need to enter it exactly as specified in the example above including exact case/capitalization.