topic spath truncation in Deployment Architecture

spath truncation

davidmills — Mon, 12 Nov 2018 06:31:38 GMT

We have a log format which contains a JSON payload. When we attempt to parse using spath, anything after a certain character limit is getting missed. We have tried setting:

[spath]
# number of characters to read from an XML or JSON event when auto extracting
extraction_cutoff = 10000
max_mem_usage_mb = 500

(up from 5000 and 200 in the defaults/limits.conf file respectively) but it made no difference. Is there some other setting that may be responsible?

Thanks, David.

Re: spath truncation

davidmills — Tue, 13 Nov 2018 02:47:51 GMT

We added the settings in $SPLUNK_HOME/etc/system/local/limits.conf and then restarted the 3 Search Head Cluster instances.
The command:

$SPLUNK_HOME/bin/splunk btool limits list

shows:

[spath]
extract_all = true
extraction_cutoff = 10000
max_mem_usage_mb = 500

suggesting that the new settings will have been picked up by the restarts.

Re: spath truncation

DalJeanis — Tue, 13 Nov 2018 22:31:01 GMT

1) Have you validated that the JSON is well formed, using any of the various web tools?

2) What is the number of characters it is cutting off at? Have you experimented to determine the precise cutoff?

3) Have you verified that the entire JSON has been ingested and is retained in your index?

Re: spath truncation

MikeElliott — Wed, 14 Nov 2018 05:24:06 GMT

I think that amending limits.conf will sort your issue. Have you indexed new data since making the amendment to limits.conf? The new spath threshold will not be applied retroactively.

We had a very similar issue recently where some user AD profiles were upwards to 15k characters due to global group memberships. Raising the limit to 20k solved the problem, but we couldn't validate until new data had been indexed (daily pull).

EDIT: Have you used a tool such as http://www.charactercountonline.com/ to confirm that you are raising the limits to the correct level?