Getting Data In
Provide Splunk Cloud feedback in this confidential UX survey by June 17
for a chance to win a $200 Amazon gift card!

How to index a binary data from UDP?

matthewgao
Engager

I am trying to receive a binary data from UDP in splunk, I have tested many method to achieve it, but it still doesn't work.
I already add a NO_BINARY_CHECK = true in props.conf.

Is there anyone have experience on this?

Tags (2)
0 Karma
1 Solution

Damien_Dallimor
Ultra Champion

You could write a simple python modular input, that allows users to open up any UDP port, captures the UDP datagram , programmatically decodes the binary data into some textual format, transforms this into events in a best practice logging semantic, and writes these events out to Splunk over STDOUT.

View solution in original post

lphirke
New Member

Splunk can not consume binary data, you have to convert it to a splunk readable format before forwarding it to splunk. are you trying to forward netflows to splunk?

0 Karma

lphirke
New Member
0 Karma

matthewgao
Engager

I am trying to forward ipfix to splunk

0 Karma

Damien_Dallimor
Ultra Champion

You could write a simple python modular input, that allows users to open up any UDP port, captures the UDP datagram , programmatically decodes the binary data into some textual format, transforms this into events in a best practice logging semantic, and writes these events out to Splunk over STDOUT.

View solution in original post

Damien_Dallimor
Ultra Champion

Exactly 🙂

0 Karma

matthewgao
Engager

So you mean that I create a UDP listener by myself, Not use the splunkd to listen the UDP

0 Karma

Damien_Dallimor
Ultra Champion

You missed the point.

You program your own Modular Input listening on it's own UDP port.Think of it like a Splunk UDP proxy. So it can capture and preprocess any type of data(ie: raw bninary) into an appropriate format for Splunk because you are programming it.

0 Karma

matthewgao
Engager

Can it capture the RAW binary data? Splunk seems automatically ignore the binary data.

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Splunk will consume any Human-Readable text, so Binary data is not going to work. You will most likely want to capture that data via UDP and write it to disk in ASCII, and then have Splunk consume the text.

alacercogitatus
SplunkTrust
SplunkTrust

Did this help you?

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!