Does Splunk have a good place to see Truncated Data from an input?

[jaredlaney]

I have an input that is being JSON. There are some files that get truncated and others that don't.

My config in props looks like this:

[test_json]
KV_MODE = json
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
TRUNCATE = 999999

The file is pretty large and each json doc in the file is on one line. When I do a line count, sometimes the numbers match Splunk and sometimes they don't. Should Splunk put up a bulletin message when it truncates data??

[maciep]

I don't think Splunk tells you when data is truncated or whether any of the normal "health" apps like SoS, DMC, Fire Brigade, etc have panels with that info.

We use a simple search below in one of our own dashboards to help identify which sourcetypes are being truncated, so we can address appropriately in props.conf. The splunkd logs know how long the event was and the limit that it exceeded which caused it to be truncated.

Maybe you could find it helpful....

index=_internal sourcetype=splunkd truncating | rex "limit of (?<limit>\S+).+\>=\s+(?<length>\S+)" | stats max(length) as length max(limit) as limit count by data_source data_sourcetype | sort - count

[GauravSplunxter]

This search gave me exactly what i needed. Thanks.