Getting Data In

How to index a binary data from UDP?

matthewgao
Engager

I am trying to receive a binary data from UDP in splunk, I have tested many method to achieve it, but it still doesn't work.
I already add a NO_BINARY_CHECK = true in props.conf.

Is there anyone have experience on this?

Tags (2)
0 Karma
1 Solution

Damien_Dallimor
Ultra Champion

You could write a simple python modular input, that allows users to open up any UDP port, captures the UDP datagram , programmatically decodes the binary data into some textual format, transforms this into events in a best practice logging semantic, and writes these events out to Splunk over STDOUT.

View solution in original post

lphirke
New Member

Splunk can not consume binary data, you have to convert it to a splunk readable format before forwarding it to splunk. are you trying to forward netflows to splunk?

0 Karma

lphirke
New Member
0 Karma

matthewgao
Engager

I am trying to forward ipfix to splunk

0 Karma

Damien_Dallimor
Ultra Champion

You could write a simple python modular input, that allows users to open up any UDP port, captures the UDP datagram , programmatically decodes the binary data into some textual format, transforms this into events in a best practice logging semantic, and writes these events out to Splunk over STDOUT.

Damien_Dallimor
Ultra Champion

Exactly 🙂

0 Karma

matthewgao
Engager

So you mean that I create a UDP listener by myself, Not use the splunkd to listen the UDP

0 Karma

Damien_Dallimor
Ultra Champion

You missed the point.

You program your own Modular Input listening on it's own UDP port.Think of it like a Splunk UDP proxy. So it can capture and preprocess any type of data(ie: raw bninary) into an appropriate format for Splunk because you are programming it.

0 Karma

matthewgao
Engager

Can it capture the RAW binary data? Splunk seems automatically ignore the binary data.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Splunk will consume any Human-Readable text, so Binary data is not going to work. You will most likely want to capture that data via UDP and write it to disk in ASCII, and then have Splunk consume the text.

alacercogitatus
SplunkTrust
SplunkTrust

Did this help you?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...