topic unable to process binary log file in Splunk Search

unable to process binary log file

clearslide_cwon — Mon, 30 Nov 2015 17:05:00 GMT

i have splunkforwarder running but once a while we run into issue with the following error about file being binary -

11-30-2015 03:28:02.240 -0800 INFO  WatchedFile - File too small to check seekcrc, probably truncated.  Will re-read entire file='/var/log/tomcat/catalina.out'.
11-30-2015 03:28:07.418 -0800 WARN  FileClassifierManager - The file '/var/log/tomcat/catalina.out' is invalid. Reason: binary
11-30-2015 03:28:07.418 -0800 INFO  TailReader - Ignoring file '/var/log/tomcat/catalina.out' due to: binary

i have tried following the instruction here but didnt seem to work -
https://answers.splunk.com/answers/36739/how-can-we-monitor-binary-log-data-in-splunk-is-invalid-reason-binary.html?utm_source=typeahead&utm_medium=newquestion&utm_campaign=no_votes_sort_relev

not sure if this is the file to edit though -

/opt/splunkforwarder/etc/system/default/props.conf

[catalina]
# 20151124 cwong - added NO_BINARY_CHECK = 1
NO_BINARY_CHECK = 1

Re: unable to process binary log file

martin_mueller — Mon, 30 Nov 2015 17:41:03 GMT

Do NOT edit files in system/default - these will get overwritten on upgrades of Splunk.
Instead, add a props.conf to the app that contains your inputs.conf and make the change there. Use the local subdirectory if the app wasn't created by your team.
http://docs.splunk.com/Documentation/Splunk/6.3.1511/Admin/Configurationfiledirectories

Re: unable to process binary log file

clearslide_cwon — Tue, 29 Sep 2020 08:00:57 GMT

hey martin,

thanks for the quick reply.
i added props.conf to the local/ dir -

cat /opt/splunkforwarder/etc/system/local/props.conf

[catalina]

20151124 cwong - added NO_BINARY_CHECK = 1

NO_BINARY_CHECK = 1
BREAK_ONLY_BEFORE_DATE = true
SHOULD_LINEMERGE=true
MAX_TIMESTAMP_LOOKAHEAD=30
TIME_PREFIX = ^
pulldown_type = 1
category = Application
description = Output produced by Apache Tomcat Catalina (System.out and System.err)

i still see the "WARN FileClassifierManager - The file '/var/log/tomcat/catalina.out' is invalid. Reason: binary" in splunkd.log

Re: unable to process binary log file

martin_mueller — Mon, 30 Nov 2015 20:32:24 GMT

Don't forget to set this on the indexer, timestamp extraction and the like is not handled by a universal forwarder.
Also restart instances after these changes.

Re: unable to process binary log file

clearslide_cwon — Fri, 18 Dec 2015 22:15:11 GMT

thanks martin, sorry for the slow reply. having the following worked:

/opt/splunkforwarder/etc/system/local/inputs.conf

[monitor:///var/log/tomcat/catalina.out]
sourcetype=tomcat-catalina-out

/opt/splunkforwarder/etc/system/local/props.conf

[tomcat-catalina-out]
NO_BINARY_CHECK = true
BREAK_ONLY_BEFORE_DATE = true
SHOULD_LINEMERGE=true
MAX_TIMESTAMP_LOOKAHEAD=30
TIME_PREFIX = ^
pulldown_type = 1
category = Application
description = Output produced by Apache Tomcat Catalina (System.out and System.err)

Re: unable to process binary log file

yannK — Tue, 29 Sep 2020 08:11:57 GMT

just to put my 2 cents.

try to add NO_BINARY_CHECK = 1
to the sourcetype in .../local/props.conf on the forwarder and indexer.
and in the .../local/inputs.conf on the forwarder under the monitor stanza
then restart to apply