topic Re: How can I keep only first 6k bytes of single line event by transforms.conf in Splunk Search

How can I keep only first 6k bytes of single line event by transforms.conf

Masa — Fri, 04 Sep 2015 21:45:15 GMT

How can I keep only first 6k bytes of single line event.

I have syslog type of data. They are single line and sometimes more than 64k Byte long.
But, I need only first 6k bytes of an event.
So, I created the following transforms.conf. But, it does not work.
I know SEDCMD works to do the same job.
But, why does transforms.conf not work?

props.conf

[syslog-cef]
SHOULD_LINEMERGE = false
TRANSFORMS-keep6k = keep6k
transforms.conf

[keep6k]
REGEX = ^(.{6144})
DEST_KEY = _raw
FORMAT = $1
Data is not truncated to 6k.

Re: How can I keep only first 6k bytes of single line event by transforms.conf

woodcock — Fri, 04 Sep 2015 22:03:21 GMT

It is easier to use SEDCMD like this:

props.conf

[syslog-cef]
SEDCMD-keepFirst6k = s/^(.{0,6144}).*/\1/

Re: How can I keep only first 6k bytes of single line event by transforms.conf

Masa — Fri, 04 Sep 2015 22:23:33 GMT

Thanks for quick response.
If I used SEDCMD with the following regex. This works.

SEDCMD-keepFirst6k = s/^(.{6144})/\1/

I'm looking for transforms.conf so that I can use the same regex for multiple sourcetypes by calling this transform name.

Re: How can I keep only first 6k bytes of single line event by transforms.conf

woodcock — Fri, 04 Sep 2015 22:33:07 GMT

Your initial solution looks fine to me; did you restart all Splunk instances on your Indexers and Heavy Forwarders?

Re: How can I keep only first 6k bytes of single line event by transforms.conf

woodcock — Fri, 04 Sep 2015 22:34:57 GMT

IMHO, you are gaining nothing except in the case that you decide to change from 6144 character to some other number (in that case you would have to edit every props.conf copy). If you are not planning on doing this, then the SEDCMD solution is shorter, better, and has every other benefit.

Re: How can I keep only first 6k bytes of single line event by transforms.conf

Masa — Sun, 06 Sep 2015 23:24:34 GMT

Hi, woodcok. Thanks you for your advice. Yes, I did restarted. In my test, I used a UF and indexer. But, it is same result even if I have a standalone instance to monitor the same file.

Re: How can I keep only first 6k bytes of single line event by transforms.conf

MuS — Mon, 07 Sep 2015 08:29:12 GMT

Hi Masa,

take a look at this answer http://answers.splunk.com/answers/306418/how-can-i-remove-partial-string-of-single-line-eve-1.html#answer-306465 🙂

cheers, MuS

Re: How can I keep only first 6k bytes of single line event by transforms.conf

Masa — Mon, 07 Sep 2015 17:17:40 GMT

Thanks, MuS.