Splunk Search

How can I keep only first 6k bytes of single line event by transforms.conf

Masa
Splunk Employee
Splunk Employee

How can I keep only first 6k bytes of single line event.

I have syslog type of data. They are single line and sometimes more than 64k Byte long.
But, I need only first 6k bytes of an event.
So, I created the following transforms.conf. But, it does not work.
I know SEDCMD works to do the same job.
But, why does transforms.conf not work?

  • props.conf

    [syslog-cef]
    SHOULD_LINEMERGE = false
    TRANSFORMS-keep6k = keep6k

  • transforms.conf

    [keep6k]
    REGEX = ^(.{6144})
    DEST_KEY = _raw
    FORMAT = $1
    Data is not truncated to 6k.

0 Karma
1 Solution

MuS
SplunkTrust
SplunkTrust

MuS
SplunkTrust
SplunkTrust

Masa
Splunk Employee
Splunk Employee

Thanks, MuS.

0 Karma

woodcock
Esteemed Legend

It is easier to use SEDCMD like this:

props.conf

[syslog-cef]
SEDCMD-keepFirst6k = s/^(.{0,6144}).*/\1/
0 Karma

woodcock
Esteemed Legend

Your initial solution looks fine to me; did you restart all Splunk instances on your Indexers and Heavy Forwarders?

0 Karma

Masa
Splunk Employee
Splunk Employee

Hi, woodcok. Thanks you for your advice. Yes, I did restarted. In my test, I used a UF and indexer. But, it is same result even if I have a standalone instance to monitor the same file.

0 Karma

Masa
Splunk Employee
Splunk Employee

Thanks for quick response.
If I used SEDCMD with the following regex. This works.

SEDCMD-keepFirst6k = s/^(.{6144})/\1/

I'm looking for transforms.conf so that I can use the same regex for multiple sourcetypes by calling this transform name.

0 Karma

woodcock
Esteemed Legend

IMHO, you are gaining nothing except in the case that you decide to change from 6144 character to some other number (in that case you would have to edit every props.conf copy). If you are not planning on doing this, then the SEDCMD solution is shorter, better, and has every other benefit.

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!