All Apps and Add-ons

Why is JSON getting truncated?

tcoq
Path Finder

Hi together,

I am trying to get data via REST API input, but I'am getting this message in splunkd.log:

Truncating line because limit of 10000 has been exceeded with a line length >= 62248

It seems to be that my JSON file is to long/big.

When I take a look into splunk, I can see my JSON which is cut off in the middle. This is still a problem, because the JSON is no longer well-formed and cannot longer pre-processed.

Is there a better way to get long JSON data in? (without setting line length limit because I have got some other JSONs which a much longer than my test request)

Best regards
Steffen

Labels (2)
0 Karma
1 Solution

mloven_splunk
Splunk Employee
Splunk Employee

tcoq,

you can create a props.conf with this:

[yourjsonsourcetypehere]
TRUNCATE = 999999

This should work for any json logs you throw in.

However, I wonder... are your logs linebreaking properly? It's not often that I see json events that are that long.

View solution in original post

0 Karma

tcoq
Path Finder

Great! I switched to TRUNCATE=0 and it works!

0 Karma

mloven_splunk
Splunk Employee
Splunk Employee

tcoq,

you can create a props.conf with this:

[yourjsonsourcetypehere]
TRUNCATE = 999999

This should work for any json logs you throw in.

However, I wonder... are your logs linebreaking properly? It's not often that I see json events that are that long.

0 Karma

davedoucette
Loves-to-Learn

Where do I put the props.conf file on a windows system?

0 Karma

lpolo
Motivator

You may try TRUNCATE = 0. Details:

TRUNCATE =
* Change the default maximum line length (in bytes).
* Although this is in bytes, line length is rounded down when this would
otherwise land mid-character for multi-byte characters.
* Set to 0 if you never want truncation (very long lines are, however, often a sign of
garbage data).
* Defaults to 10000 bytes.

napomokoetle
Communicator

I am experiencing this same problem. But the TRUNCATE = 0 setting doesn't seem to fix the issue for me.

Did you add this parameter on the Universal Forwarder, or on the Indexer?
Or does it not matter?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Using the Splunk Threat Research Team’s Latest Security Content

REGISTER HERE Tech Talk | Security Edition Did you know the Splunk Threat Research Team regularly releases ...

SplunkTrust | 2024 SplunkTrust Application Period is Open!

It's that time again, folks! That's right, the application/nomination period for the 2024 SplunkTrust is ...