Getting Data In
Highlighted

Does TRUNCATE specify the ultimate size of an event?

Motivator

We are not clear whether setting TRUNCATE to a certain value guarantees that the event won't exceed this size in bytes. If not, can we specify the maximum length of an event somewhere?

Tags (2)
0 Karma
Highlighted

Re: Does TRUNCATE specify the ultimate size of an event?

Influencer

Splunk truncates events in bytes, from Splunk documentation on props.conf:

TRUNCATE = <non-negative integer>
* The default maximum line length, in bytes.
* Although this is in bytes, line length is rounded down when this would
  otherwise land mid-character for multi-byte characters.
* Set to 0 if you never want truncation (very long lines are, however, often
  a sign of garbage data).
* Default: 10000
Highlighted

Re: Does TRUNCATE specify the ultimate size of an event?

Motivator

Ok, according to this documentation, TRUNCATE specifies the length of a line. Then we can merge lines, right? I wonder if or where we can specify the max length of an event?

Beacuae I run -

index=<index name>
| eval len = len(_raw)
| stats max(len)

And it returns 3,461,805. I would like to have a concrete limit for the length of an event.

0 Karma
Highlighted

Re: Does TRUNCATE specify the ultimate size of an event?

Influencer

You can try applying regex to truncate the _raw field. Below regex gets first 3000 characters.

props.conf:

[sourcetype]
TRUNCATE = 10000
TRANSFORMS-truncate_raw = truncate_raw

transforms.conf

[truncate_raw]
SOURCE_KEY  = _raw
REGEX = ^.{0,3000}
DEST_KEY = _raw
FORMAT = $1
Highlighted

Re: Does TRUNCATE specify the ultimate size of an event?

Motivator

Nice @manjunathmeti - are there any combinations where TRUNCATE does specify the length of the event?

0 Karma
Highlighted

Re: Does TRUNCATE specify the ultimate size of an event?

Motivator

We are fine with TRUNCATE for the max line's length and MAX_EVENTS for max number of lines.

0 Karma