All Apps and Add-ons

Why is JSON getting truncated?

tcoq
Path Finder

Hi together,

I am trying to get data via REST API input, but I'am getting this message in splunkd.log:

Truncating line because limit of 10000 has been exceeded with a line length >= 62248

It seems to be that my JSON file is to long/big.

When I take a look into splunk, I can see my JSON which is cut off in the middle. This is still a problem, because the JSON is no longer well-formed and cannot longer pre-processed.

Is there a better way to get long JSON data in? (without setting line length limit because I have got some other JSONs which a much longer than my test request)

Best regards
Steffen

Labels (2)
0 Karma
1 Solution

mloven_splunk
Splunk Employee
Splunk Employee

tcoq,

you can create a props.conf with this:

[yourjsonsourcetypehere]
TRUNCATE = 999999

This should work for any json logs you throw in.

However, I wonder... are your logs linebreaking properly? It's not often that I see json events that are that long.

View solution in original post

0 Karma

tcoq
Path Finder

Great! I switched to TRUNCATE=0 and it works!

0 Karma

mloven_splunk
Splunk Employee
Splunk Employee

tcoq,

you can create a props.conf with this:

[yourjsonsourcetypehere]
TRUNCATE = 999999

This should work for any json logs you throw in.

However, I wonder... are your logs linebreaking properly? It's not often that I see json events that are that long.

0 Karma

davedoucette
Loves-to-Learn

Where do I put the props.conf file on a windows system?

0 Karma

lpolo
Motivator

You may try TRUNCATE = 0. Details:

TRUNCATE =
* Change the default maximum line length (in bytes).
* Although this is in bytes, line length is rounded down when this would
otherwise land mid-character for multi-byte characters.
* Set to 0 if you never want truncation (very long lines are, however, often a sign of
garbage data).
* Defaults to 10000 bytes.

napomokoetle
Communicator

I am experiencing this same problem. But the TRUNCATE = 0 setting doesn't seem to fix the issue for me.

Did you add this parameter on the Universal Forwarder, or on the Indexer?
Or does it not matter?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...