topic Re: Does TRUNCATE specify the ultimate size of an event? in Getting Data In

Does TRUNCATE specify the ultimate size of an event?

danielbb — Wed, 19 Feb 2020 17:06:47 GMT

We are not clear whether setting TRUNCATE to a certain value guarantees that the event won't exceed this size in bytes. If not, can we specify the maximum length of an event somewhere?

Re: Does TRUNCATE specify the ultimate size of an event?

manjunathmeti — Wed, 19 Feb 2020 17:18:35 GMT

Splunk truncates events in bytes, from Splunk documentation on props.conf:

TRUNCATE = <non-negative integer>
* The default maximum line length, in bytes.
* Although this is in bytes, line length is rounded down when this would
  otherwise land mid-character for multi-byte characters.
* Set to 0 if you never want truncation (very long lines are, however, often
  a sign of garbage data).
* Default: 10000

Re: Does TRUNCATE specify the ultimate size of an event?

danielbb — Wed, 19 Feb 2020 18:22:30 GMT

Ok, according to this documentation, TRUNCATE specifies the length of a line. Then we can merge lines, right? I wonder if or where we can specify the max length of an event?

Beacuae I run -

index=<index name>
| eval len = len(_raw)
| stats max(len)

And it returns 3,461,805. I would like to have a concrete limit for the length of an event.

Re: Does TRUNCATE specify the ultimate size of an event?

manjunathmeti — Wed, 19 Feb 2020 18:54:24 GMT

You can try applying regex to truncate the _raw field. Below regex gets first 3000 characters.

props.conf:

[sourcetype]
TRUNCATE = 10000
TRANSFORMS-truncate_raw = truncate_raw

transforms.conf

[truncate_raw]
SOURCE_KEY  = _raw
REGEX = ^.{0,3000}
DEST_KEY = _raw
FORMAT = $1

Re: Does TRUNCATE specify the ultimate size of an event?

danielbb — Thu, 20 Feb 2020 19:24:54 GMT

Nice @manjunathmeti - are there any combinations where TRUNCATE does specify the length of the event?

Re: Does TRUNCATE specify the ultimate size of an event?

danielbb — Thu, 27 Feb 2020 17:36:32 GMT

We are fine with TRUNCATE for the max line's length and MAX_EVENTS for max number of lines.