topic Re: Why are our logs being truncated to 10,000 characters? in Deployment Architecture

Why are our logs being truncated to 10,000 characters?

davidmills — Mon, 12 Nov 2018 06:09:55 GMT

I have set...

[default]
TRUNCATE = 20000

...in $SPLUNK_HOME/etc/system/local/props.conf for our search heads (a cluster of 3), indexers (a cluster of 3) and Heavy forwarder. I have restarted all the search heads and indexers to pick up the change, but we are still getting just as many records where "meta::truncated" is getting set and the log entry is just 10,000 characters. This then plays havoc with our attempts with downstream searches to parse out JSON values using spath. Some of the information we need is just not present.

Do I need to restart the Heavy Forwarder? Is it where the truncation is occurring? Are there any data loss implications of stopping the single Heavy Forwarder?

Thanks,

David.

Re: Why are our logs being truncated to 10,000 characters?

inventsekar — Mon, 12 Nov 2018 06:42:27 GMT

this will be done in test or dev or prod?!?!

Re: Why are our logs being truncated to 10,000 characters?

richgalloway — Mon, 12 Nov 2018 12:00:12 GMT

The truncation is happening on the heavy forwarder. Restart it to apply the props.conf setting and all should be OK.

Re: Why are our logs being truncated to 10,000 characters?

davidmills — Mon, 12 Nov 2018 23:16:47 GMT

That worked - thanks.