While debugging an issue where a forwarder would not send a specific log to our main splunk instance, i found this great post (among others):
https://community.splunk.com/t5/Deployment-Architecture/Unix-Forwarder-is-not-Sending-Logs/m-p/16734...
in the inputs.conf on the Universal Forwarder, the fix was adding initCrcLength = 2048 to the specific logs stanza:
( in: C:\Program Files\SplunkUniversalForwarder\etc\apps\SplunkUniversalForwarder\local\inputs.conf)
Apparently the default is initCrcLength = 256. This made me start thinking, how many other forwarder logs are we not getting/indexing that im not aware of !?
thus, this search below showed me that 4 of our Forwarder's (of ~22 forwarders in total) where showing this same error for various specific log files (thus those specific logs have not been getting indexed):
index=_internal source=*splunkd.log host=* "seekptr checksum"
(while this is very unfortunate), my question is:
Should we be manually setting something like initCrcLength = 2048 on every one of our Forwarders (and on future new forwarders)?
I assume the downside is increased RAM and CPU usage on the forwarders (but this is not an issue for us, as volume is not very high, and resources plentiful). Anything else im not considering as a downside?
question 2:
I assume we can "globally" set this on a forwarders inputs.conf by simply placing:
[default]
initCrcLength = 2048
and it will apply to all stanzas (unless a stanza overrides initCrcLength, of-course), Right?
thanks!
