topic Re: Why does pretty-print JSON events get truncated? in Getting Data In

Why does pretty-print JSON events get truncated?

yuanliu — Wed, 18 Aug 2021 02:54:49 GMT

I have some events that exceeds the default 10000-byte TRUNCATE limit. This triggers "truncating line because limit of 10000 bytes has been exceeded." In Splunk documentation, this is characterized as "line-breaking issues".

Because the events are received in JSON, I thought reducing line length would solve this problem without tweaking TRUNCATE. But this doesn't. After pretty-print, large JSON documents still get truncated.

Does this mean that the "line breaking issue" is really "event breaking issue", that the indexer requires every event to be under TRUNCATE limit? The JSON documents have the correct syntax including the opening and closing brackets.

Re: Why does pretty-print JSON events get truncated?

scelikok — Wed, 18 Aug 2021 04:38:42 GMT

TRUNCATE applies to all events. You should set this value high enough for each sourcetype that exceeds the default 10000 limit.

Re: Why does pretty-print JSON events get truncated?

yuanliu — Thu, 19 Aug 2021 10:52:26 GMT

So, is it correct to say that TRUNCATE applies to total length of an event message, not about an exceedingly long line in a message? Specific to structured message types like JSON and CSV, each document/row must be within TRUNCATE limit, no matter whether and where newline is placed.