UDP Events truncated at 2048 bytes

[blurblebot]

Hello, Splunkmind -

I'm having an issue with a UDP data inputs. All of my events are being cutoff after 2048 bytes, despite that fact that the events that are being sent are actually around 5000 bytes. 2048 is too perfect a number for this truncation to feel like I'm totally lost, but I am. I've searched all of the input.confs and props.confs on my splunk install for "2048", but only come up with a MAX_EVENTS setting that is unrelated.

After failing to sort this out on a production box, I've set up a brand new splunk instance with almost no modifications. I'm performing my searches with the search app.

I have one data input set up to receive UDP:5141. I set this up through the manager GUI and the resultant inputs.conf is at /opt/splunk/etc/apps/launcher/local/inputs.conf

[udp://5141]
connection_host = none
sourcetype = NSP
_rcvbuf = 3000000
queueSize = 128MB
persistentQueueSize = 256 MB

As you may notice, I've carried over some fruitless finagles from when I thought the issue had to do with traffic volume.

I have tried the modifying the TRUNCATE from the default (which, at 10000 bytes shouldn't be the problem, anyway, but maybe the default was somehow busted) setting in several of the props.conf locations to add

[sourcetype::NSP]
TRUNCATE = 9999

..but no dice.

I'm at a total loss at this point and would sing praises to anyone with a clue. I hope you like bad singing.

Thanks

-Steve

[torbael]

Are you trying to send 5000 Bytes over UDP with syslog? If you use syslog I think that it could be a maximun Byte limit on the protocol.

[jkerai]

Steve, are you seeing the remaining data (5000-2048) show up at all? We do try to read 2048 bytes at a time and queue it up for processing. One of the solutions could be to make UDP input configurable on splunk such that it tries to read configured number of bytes, 5000 in your case.

[williamsweat]

Which version of Splunk is this affecting?

[Masa]

jkerai: I reproduced the issue. Splunk indexes only the first 2048 bytes and the rest is gone. The event contains more than 2048 characters in one line. I will file a bug. Let's talk about it soon.

[blurblebot]

The more views I get with no responses, the more stupid I fear my question to be. Anyhoozles, I'm working with Splunk Enterprise Support to fix this, so when they tell me what I've borked, I'll let y'all know.

[blurblebot]

I also wanted to add that I've confirmed receipt of the whole of the UDP events via tcpdump on the OS of the SPlunk instance.