Splunk Search

How to use splunk for binary log file?

txing
New Member

Our application is generating a binary format log files, and we want to use splunk to collect and search through it, can splunk process binary logs? what else should we do?

Thanks

Tags (1)
0 Karma

ziegfried
Influencer

Another option is to use the unarchive command. There has been a blog entry that illustrates how to do that:

http://blogs.splunk.com/2011/07/19/the-naughty-bits-how-to-splunk-binary-logfiles/

It still involves scripting, though.

Ayn
Legend

While you could technically force Splunk to index a file even though it's in a binary format, the question is what you'd get out of it. If your application logs in some kind of binary format where special knowledge is required to make any sense out of the data, you need to somehow provide that logic so that the data can be converted to something that makes sense to (and in) Splunk. A common way of doing this is using scripted inputs, where you have a script that reads your binary logs, retrieves the data and outputs it as a plaintext format that's suitable to be consumed by Splunk.

ziegfried
Influencer

Creating such a scripted input is pretty simple. Everthing that the script sends to STDOUT will be indexed by Splunk.

0 Karma

txing
New Member

Thanks for the answers, but I'm not scripting guy, for the scripted input, is there a sample or we need to do it by ourselves?

Thanks

0 Karma

MarioM
Motivator

some people like to look at hexadecimal... 😜

0 Karma

MarioM
Motivator

you can allow processing of binary files in props.conf:

NO_BINARY_CHECK = [true|false]
* When set to true, Splunk processes binary files.
* Can only be used on the basis of [<sourcetype>], or [source::<source>], not [host::<host>].
* Defaults to false (binary files are ignored).
0 Karma
Get Updates on the Splunk Community!

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...