Splunk Search

unable to process binary log file

clearslide_cwon
New Member

i have splunkforwarder running but once a while we run into issue with the following error about file being binary -

11-30-2015 03:28:02.240 -0800 INFO  WatchedFile - File too small to check seekcrc, probably truncated.  Will re-read entire file='/var/log/tomcat/catalina.out'.
11-30-2015 03:28:07.418 -0800 WARN  FileClassifierManager - The file '/var/log/tomcat/catalina.out' is invalid. Reason: binary
11-30-2015 03:28:07.418 -0800 INFO  TailReader - Ignoring file '/var/log/tomcat/catalina.out' due to: binary

i have tried following the instruction here but didnt seem to work -
https://answers.splunk.com/answers/36739/how-can-we-monitor-binary-log-data-in-splunk-is-invalid-rea...

not sure if this is the file to edit though -

/opt/splunkforwarder/etc/system/default/props.conf

[catalina]
# 20151124 cwong - added NO_BINARY_CHECK = 1
NO_BINARY_CHECK = 1
Tags (1)
0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Do NOT edit files in system/default - these will get overwritten on upgrades of Splunk.
Instead, add a props.conf to the app that contains your inputs.conf and make the change there. Use the local subdirectory if the app wasn't created by your team.
http://docs.splunk.com/Documentation/Splunk/6.3.1511/Admin/Configurationfiledirectories

View solution in original post

yannK
Splunk Employee
Splunk Employee

just to put my 2 cents.

try to add NO_BINARY_CHECK = 1
to the sourcetype in .../local/props.conf on the forwarder and indexer.
and in the .../local/inputs.conf on the forwarder under the monitor stanza
then restart to apply

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Do NOT edit files in system/default - these will get overwritten on upgrades of Splunk.
Instead, add a props.conf to the app that contains your inputs.conf and make the change there. Use the local subdirectory if the app wasn't created by your team.
http://docs.splunk.com/Documentation/Splunk/6.3.1511/Admin/Configurationfiledirectories

clearslide_cwon
New Member

hey martin,

thanks for the quick reply.
i added props.conf to the local/ dir -

cat /opt/splunkforwarder/etc/system/local/props.conf

[catalina]

20151124 cwong - added NO_BINARY_CHECK = 1

NO_BINARY_CHECK = 1
BREAK_ONLY_BEFORE_DATE = true
SHOULD_LINEMERGE=true
MAX_TIMESTAMP_LOOKAHEAD=30
TIME_PREFIX = ^
pulldown_type = 1
category = Application
description = Output produced by Apache Tomcat Catalina (System.out and System.err)

i still see the "WARN FileClassifierManager - The file '/var/log/tomcat/catalina.out' is invalid. Reason: binary" in splunkd.log

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Don't forget to set this on the indexer, timestamp extraction and the like is not handled by a universal forwarder.
Also restart instances after these changes.

0 Karma

clearslide_cwon
New Member

thanks martin, sorry for the slow reply. having the following worked:

/opt/splunkforwarder/etc/system/local/inputs.conf

[monitor:///var/log/tomcat/catalina.out]
sourcetype=tomcat-catalina-out

/opt/splunkforwarder/etc/system/local/props.conf

[tomcat-catalina-out]
NO_BINARY_CHECK = true
BREAK_ONLY_BEFORE_DATE = true
SHOULD_LINEMERGE=true
MAX_TIMESTAMP_LOOKAHEAD=30
TIME_PREFIX = ^
pulldown_type = 1
category = Application
description = Output produced by Apache Tomcat Catalina (System.out and System.err)
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...