topic Re: JSON is truncated in All Apps and Add-ons

Why is JSON getting truncated?

tcoq — Fri, 11 Mar 2022 19:21:55 GMT

Hi together,

I am trying to get data via REST API input, but I'am getting this message in splunkd.log:

Truncating line because limit of 10000 has been exceeded with a line length >= 62248

It seems to be that my JSON file is to long/big.

When I take a look into splunk, I can see my JSON which is cut off in the middle. This is still a problem, because the JSON is no longer well-formed and cannot longer pre-processed.

Is there a better way to get long JSON data in? (without setting line length limit because I have got some other JSONs which a much longer than my test request)

Best regards
Steffen

Re: JSON is truncated

mloven_splunk — Tue, 10 Sep 2013 19:43:22 GMT

tcoq,

you can create a props.conf with this:

[yourjsonsourcetypehere]
TRUNCATE = 999999

This should work for any json logs you throw in.

However, I wonder... are your logs linebreaking properly? It's not often that I see json events that are that long.

Re: JSON is truncated

lpolo — Wed, 11 Sep 2013 18:35:05 GMT

You may try TRUNCATE = 0. Details:

TRUNCATE =
* Change the default maximum line length (in bytes).
* Although this is in bytes, line length is rounded down when this would
otherwise land mid-character for multi-byte characters.
* Set to 0 if you never want truncation (very long lines are, however, often a sign of
garbage data).
* Defaults to 10000 bytes.

Re: JSON is truncated

tcoq — Thu, 12 Sep 2013 12:47:45 GMT

Great! I switched to TRUNCATE=0 and it works!

Re: JSON is truncated

napomokoetle — Tue, 22 Nov 2016 08:37:28 GMT

I am experiencing this same problem. But the TRUNCATE = 0 setting doesn't seem to fix the issue for me.

Did you add this parameter on the Universal Forwarder, or on the Indexer?
Or does it not matter?

Re: JSON is truncated

davedoucette — Fri, 11 Mar 2022 18:36:43 GMT

Where do I put the props.conf file on a windows system?