Getting Data In

Does TRUNCATE specify the ultimate size of an event?

danielbb
Motivator

We are not clear whether setting TRUNCATE to a certain value guarantees that the event won't exceed this size in bytes. If not, can we specify the maximum length of an event somewhere?

Tags (2)
0 Karma

manjunathmeti
SplunkTrust
SplunkTrust

Splunk truncates events in bytes, from Splunk documentation on props.conf:

TRUNCATE = <non-negative integer>
* The default maximum line length, in bytes.
* Although this is in bytes, line length is rounded down when this would
  otherwise land mid-character for multi-byte characters.
* Set to 0 if you never want truncation (very long lines are, however, often
  a sign of garbage data).
* Default: 10000

danielbb
Motivator

Ok, according to this documentation, TRUNCATE specifies the length of a line. Then we can merge lines, right? I wonder if or where we can specify the max length of an event?

Beacuae I run -

index=<index name>
| eval len = len(_raw)
| stats max(len)

And it returns 3,461,805. I would like to have a concrete limit for the length of an event.

0 Karma

manjunathmeti
SplunkTrust
SplunkTrust

You can try applying regex to truncate the _raw field. Below regex gets first 3000 characters.

props.conf:

[sourcetype]
TRUNCATE = 10000
TRANSFORMS-truncate_raw = truncate_raw

transforms.conf

[truncate_raw]
SOURCE_KEY  = _raw
REGEX = ^.{0,3000}
DEST_KEY = _raw
FORMAT = $1

danielbb
Motivator

Nice @manjunathmeti - are there any combinations where TRUNCATE does specify the length of the event?

0 Karma

danielbb
Motivator

We are fine with TRUNCATE for the max line's length and MAX_EVENTS for max number of lines.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...