Splunk Search
Highlighted

find broken events

Communicator

How to figure which events are broken or truncated by splunk.
I know that the default is 256 lines for multiline events, and 10000 char for a single line.

I want to optimize my sourcetypes props.conf, but I cannot identify which ones are being cut.

Highlighted

Re: find broken events

Splunk Employee
Splunk Employee

Hi Mata,

  • For one part you can use the internal logs of the indexers, look for events like
    WARN  LineBreakingProcessor - Truncating line because limit of 10000 has been exceeded: 10868
    WARN  AggregatorMiningProcessor - Breaking event because limit of 256 has been exceeded
    

Use this search to see the evolution of the number of errors.


index=_internal source=splunkd.log WARN "Truncating" OR "Breaking event" | timechart count by component

  • For the other part, you can search in the actual real events to find the cut ones.

Look for events cut at exactly 257 lines (or higher for certain sourcetype)


index=stormsplunk linecount>256 | stats count values(source) values(sourcetype) values(host) values(index) by linecount

Look for events that are cut a 10000 characters.

index=stormsplunk | eval eventlen=len(raw) | WHERE eventlen > 9999 | stats count values(source) values(sourcetype) values(host) values(index) by eventlen

[edit] value fixed

View solution in original post

Highlighted

Re: find broken events

Communicator

Your final example has one too many 9's in the conditional. Should be
index=stormsplunk | eval eventlen=len(raw) | WHERE eventlen > 9999 | stats count values(source) values(sourcetype) values(host) values(index) by event_len

Highlighted

Re: find broken events

Splunk Employee
Splunk Employee

thanks, number fixed

0 Karma