Getting Data In

Size limit on for an event? Part 2

alextsui
Path Finder

Hi, I searched the Splunk>answers and saw someone had asked the question before. But my situation is a little different.

Some events receive from F5 ASM thru tcp port are been split into two events. The first part of the split events all have 2094 characters in length. And a monitored input, where events are longer than 2094 characters, aren't been split. It looks like Splunk is splitting the events only if they are longer than 2094 characters and are coming in from F5 ASM (or network port). I have tried increasing the TRUNCATE and MAX_EVENT in props.conf with no luck.

Any suggestion?

Splunk version 4.1.3 on CentOS 5.4 64-bit.

Thanks, alex.

Tags (2)

gkanapathy
Splunk Employee
Splunk Employee

It's likely that your F5 is splitting events at 2048 characters because the syslog protocol limits it to that. You would probably want to use SHOULD_LINEMERGE and a BREAK_ONLY_BEFORE or BREAK_ONLY_BEFORE_DATE to assemble your "lines" back into full events. You could also try setting LINE_BREAKER so get Splunk to ignore whatever character the device is using in the stream to split lines, and provide and alternate sequence.

alextsui
Path Finder

Tried first using the SHOULD_LINEMERGE and BREAK_ONLY_BEFORE, couldn't get the split events to merge. Then change the max event size to 60k on F5 web console. Most of large size events showed as a single event, but noticed events started getting split at random length.
After using SHOULD_LINEMERGE=false, and LINE_BREAKER=(<134>) (all events from F5 ASM starts with <134> in my case) still seeing one or two events every hour with <134> at the beginning of events in Splunk search results. Shouldn't all events indexed by Splunk now be without the <134> at the beginning?

0 Karma

Lowell
Super Champion

Are your events coming over the syslog protocol?

Because syslog imposes a 2k (2048 byte) limit on the size of its log events. If this is the case then I'm guessing that splunk is configured to add a timestamp or something that amounts to 46 characters adding up to your 2094 size observation.

I know is some syslog implementations can bump this up to 4k (4096), but I think that's the max.

I could be way off here, but seems like it's a possibility

alextsui
Path Finder

Thanks for the info on the syslog's limit on event size. After checking the related settings on F5 ASM web cosole, it does provide an option for selecting the size for events. It defaults to 2k, and has 10k and 60k for selection.

0 Karma

erydberg
Splunk Employee
Splunk Employee

Can you post your settings from props.conf?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...