find broken events

[mataharry]

How to figure which events are broken or truncated by splunk.
I know that the default is 256 lines for multiline events, and 10000 char for a single line.

I want to optimize my sourcetypes props.conf, but I cannot identify which ones are being cut.

[yannK · Accepted solution]

Hi Mata,

• For one part you can use the internal logs of the indexers, look for events like

  WARN  LineBreakingProcessor - Truncating line because limit of 10000 has been exceeded: 10868
  WARN  AggregatorMiningProcessor - Breaking event because limit of 256 has been exceeded
  

Use this search to see the evolution of the number of errors.

index=_internal source=splunkd.log WARN "Truncating" OR "Breaking event" | timechart count by component

• For the other part, you can search in the actual real events to find the cut ones.

Look for events cut at exactly 257 lines (or higher for certain sourcetype)

index=storm_splunk linecount>256 | stats count values(source) values(sourcetype) values(host) values(index) by linecount 

Look for events that are cut a 10000 characters.

index=storm_splunk | eval event_len=len(_raw) | WHERE event_len > 9999 | stats count values(source) values(sourcetype) values(host) values(index) by event_len

[edit] value fixed

[yannK]

thanks, number fixed

[mslvrstn]

Your final example has one too many 9's in the conditional. Should be
index=storm_splunk | eval event_len=len(_raw) | WHERE event_len > 9999 | stats count values(source) values(sourcetype) values(host) values(index) by event_len