All Apps and Add-ons

Why is JSON getting truncated?

tcoq
Path Finder

Hi together,

I am trying to get data via REST API input, but I'am getting this message in splunkd.log:

Truncating line because limit of 10000 has been exceeded with a line length >= 62248

It seems to be that my JSON file is to long/big.

When I take a look into splunk, I can see my JSON which is cut off in the middle. This is still a problem, because the JSON is no longer well-formed and cannot longer pre-processed.

Is there a better way to get long JSON data in? (without setting line length limit because I have got some other JSONs which a much longer than my test request)

Best regards
Steffen

Labels (2)
0 Karma
1 Solution

mloven_splunk
Splunk Employee
Splunk Employee

tcoq,

you can create a props.conf with this:

[yourjsonsourcetypehere]
TRUNCATE = 999999

This should work for any json logs you throw in.

However, I wonder... are your logs linebreaking properly? It's not often that I see json events that are that long.

View solution in original post

0 Karma

tcoq
Path Finder

Great! I switched to TRUNCATE=0 and it works!

0 Karma

mloven_splunk
Splunk Employee
Splunk Employee

tcoq,

you can create a props.conf with this:

[yourjsonsourcetypehere]
TRUNCATE = 999999

This should work for any json logs you throw in.

However, I wonder... are your logs linebreaking properly? It's not often that I see json events that are that long.

0 Karma

davedoucette
Loves-to-Learn

Where do I put the props.conf file on a windows system?

0 Karma

lpolo
Motivator

You may try TRUNCATE = 0. Details:

TRUNCATE =
* Change the default maximum line length (in bytes).
* Although this is in bytes, line length is rounded down when this would
otherwise land mid-character for multi-byte characters.
* Set to 0 if you never want truncation (very long lines are, however, often a sign of
garbage data).
* Defaults to 10000 bytes.

napomokoetle
Communicator

I am experiencing this same problem. But the TRUNCATE = 0 setting doesn't seem to fix the issue for me.

Did you add this parameter on the Universal Forwarder, or on the Indexer?
Or does it not matter?

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...