Splunk Search

unable to process binary log file

clearslide_cwon
New Member

i have splunkforwarder running but once a while we run into issue with the following error about file being binary -

11-30-2015 03:28:02.240 -0800 INFO  WatchedFile - File too small to check seekcrc, probably truncated.  Will re-read entire file='/var/log/tomcat/catalina.out'.
11-30-2015 03:28:07.418 -0800 WARN  FileClassifierManager - The file '/var/log/tomcat/catalina.out' is invalid. Reason: binary
11-30-2015 03:28:07.418 -0800 INFO  TailReader - Ignoring file '/var/log/tomcat/catalina.out' due to: binary

i have tried following the instruction here but didnt seem to work -
https://answers.splunk.com/answers/36739/how-can-we-monitor-binary-log-data-in-splunk-is-invalid-rea...

not sure if this is the file to edit though -

/opt/splunkforwarder/etc/system/default/props.conf

[catalina]
# 20151124 cwong - added NO_BINARY_CHECK = 1
NO_BINARY_CHECK = 1
Tags (1)
0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Do NOT edit files in system/default - these will get overwritten on upgrades of Splunk.
Instead, add a props.conf to the app that contains your inputs.conf and make the change there. Use the local subdirectory if the app wasn't created by your team.
http://docs.splunk.com/Documentation/Splunk/6.3.1511/Admin/Configurationfiledirectories

View solution in original post

yannK
Splunk Employee
Splunk Employee

just to put my 2 cents.

try to add NO_BINARY_CHECK = 1
to the sourcetype in .../local/props.conf on the forwarder and indexer.
and in the .../local/inputs.conf on the forwarder under the monitor stanza
then restart to apply

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Do NOT edit files in system/default - these will get overwritten on upgrades of Splunk.
Instead, add a props.conf to the app that contains your inputs.conf and make the change there. Use the local subdirectory if the app wasn't created by your team.
http://docs.splunk.com/Documentation/Splunk/6.3.1511/Admin/Configurationfiledirectories

clearslide_cwon
New Member

hey martin,

thanks for the quick reply.
i added props.conf to the local/ dir -

cat /opt/splunkforwarder/etc/system/local/props.conf

[catalina]

20151124 cwong - added NO_BINARY_CHECK = 1

NO_BINARY_CHECK = 1
BREAK_ONLY_BEFORE_DATE = true
SHOULD_LINEMERGE=true
MAX_TIMESTAMP_LOOKAHEAD=30
TIME_PREFIX = ^
pulldown_type = 1
category = Application
description = Output produced by Apache Tomcat Catalina (System.out and System.err)

i still see the "WARN FileClassifierManager - The file '/var/log/tomcat/catalina.out' is invalid. Reason: binary" in splunkd.log

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Don't forget to set this on the indexer, timestamp extraction and the like is not handled by a universal forwarder.
Also restart instances after these changes.

0 Karma

clearslide_cwon
New Member

thanks martin, sorry for the slow reply. having the following worked:

/opt/splunkforwarder/etc/system/local/inputs.conf

[monitor:///var/log/tomcat/catalina.out]
sourcetype=tomcat-catalina-out

/opt/splunkforwarder/etc/system/local/props.conf

[tomcat-catalina-out]
NO_BINARY_CHECK = true
BREAK_ONLY_BEFORE_DATE = true
SHOULD_LINEMERGE=true
MAX_TIMESTAMP_LOOKAHEAD=30
TIME_PREFIX = ^
pulldown_type = 1
category = Application
description = Output produced by Apache Tomcat Catalina (System.out and System.err)
0 Karma
Get Updates on the Splunk Community!

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...

Avoid Certificate Expiry Issues in Splunk Enterprise with Certificate Assist

This blog post is part 2 of 4 of a series on Splunk Assist. Click the links below to see the other ...