topic Using Transforms.conf truncates my data at 3925 characters - Why? in Getting Data In

Using Transforms.conf truncates my data at 3925 characters - Why?

kendrickt — Tue, 17 Feb 2015 12:51:30 GMT

Ladies and Gents,

I'm struggling trying to transform some of my data.

props.conf

[st1]
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TRUNCATE = 0
TRANSFORMS-1 = t1, t2
BREAK_ONLY_BEFORE_DATE = false

transforms.conf

[t1]
REGEX = ^(.*)data(data I want to capture)data(.*)$
FORMAT = $1<newfield>$2</newfield>$3
DEST_KEY = _raw

[t2]
REGEX = ^(.*)different data(different data I want to capture)different data(.*)$
FORMAT = $1<newfield1>$2</newfield2>$3
DEST_KEY = _raw

There problem I have is when the data is passed to transforms.conf, the resulting event is truncated at 3925 characters. whereas the data that isn't modified by transforms.conf remains fine.

Please help

Thanks

Re: Using Transforms.conf truncates my data at 3925 characters - Why?

kendrickt — Wed, 18 Feb 2015 15:04:30 GMT

Just figured out the answer - Splunk was truncating at 4096 characters. So I added LOOKAHEAD = 20000 to my tranforms.conf and it works!

Re: Using Transforms.conf truncates my data at 3925 characters - Why?

saurabh_tek11 — Wed, 20 Dec 2017 07:28:01 GMT

Thanks for sharing your finding