topic Re: Raw json file getting truncated in Splunk Search

Raw json file getting truncated

jimbolya — Fri, 24 Mar 2017 17:17:30 GMT

Using splunk developer version. I've set TRUNCATE to 25000 and max_events to a large number as well, but the information keeps getting truncated in the event window. I'm basically trying to import large amounts of json data from a single file. I know I'm not the only one working with json data so I'm sure the fix is nice and easy.

Re: Raw json file getting truncated

briancronrath — Mon, 27 Mar 2017 22:14:17 GMT

Do you have an extraction_cutoff value set anywhere in your limits.conf files?

Re: Raw json file getting truncated

paulbannister — Fri, 07 Apr 2017 09:23:06 GMT

Try setting the TRUNCATE to 0 to remove the limit, we also had an issue when bringing in large amounts of json data via a REST API which was resolved by setting the "response handler" to "JSONArrayHandler", which specifically handles large amounts of json data... although I am unsure if this relates purely to the REST API

Re: Raw json file getting truncated

hardikJsheth — Fri, 07 Apr 2017 12:14:58 GMT

You can handle it via props configuration for your specific source type.
i.e I am using following stanza which is able to extract raw JSON file of > 8MB.

[cmxmap]
TRUNCATE = 0
KV_MODE = json
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false