topic Can you rename fields that were automatically extracted with KV_mode=auto using transformations? in Getting Data In

Can you rename fields that were automatically extracted with KV_mode=auto using transformations?

rjthibod — Mon, 16 May 2016 22:39:43 GMT

If I have Key-Value pair events and fields that are automatically extracted with KV_MODE=auto in props.conf, can I apply a field transformation to an extracted field?

For example, I have a field UserName that appears in the raw events like (e.g., ... UserName="ryan" ...). I want a field user to appear at search-time, but I don't want to use and EVAL- or a FIELDALIAS- clause in props.conf, because I don't to overload the server and how it looks for fields (see https://splunkbase.splunk.com/app/2871/ and explanation about how litsearch works).

I have tried using this in props.conf

[my_src_type]
KV_MODE = auto
REPORT-extractions = RenameUser,ExtractSessionType

And the following in transforms.conf

[RenameUser]
SOURCE_KEY = UserName
REGEX = (.+)
FORMAT = user::"$1"
[ExtractSessionType]
REGEX = SessionName="(?<SessionType>\w+(-\w+)*)\S*"

The "SessionType" field extractions from the "SessionName" field are successful, but the "UserName" field is never renamed to "user".

Is this possible with the Key-Value extractions being applied first?

I have looked in the job inspector and found no mention of errors or issues.

Re: Can you rename fields that were automatically extracted with KV_mode=auto using transformations?

rjthibod — Tue, 17 May 2016 01:50:17 GMT

My own experimenting seems to say that using "EXTRACT-" clauses won't work either. I am guessing this has to do with the use of KV_MODE being set to "auto". I am probably abandoning it for now, but would welcome any response from someone that can give a definitive answer.

Re: Can you rename fields that were automatically extracted with KV_mode=auto using transformations?

woodcock — Sun, 29 May 2016 14:13:05 GMT

The only way to do what you are asking (and I would not do it) is to change it in the raw event using SEDCMD. Check it out here:

http://docs.splunk.com/Documentation/Splunk/6.2.0/Data/Anonymizedatausingconfigurationfiles

IMHO, the proper policy should be that it is OK to have one alias per field but ONLY one (preferably none). The exploding litsearch is a problem but Splunk is aggressively working on it and it isn't (usually) as bad as it seems (except for with ES).