How to automatically extract key value pairs after using translatefix

[isha_rastogi]

I have used translatefix to decode the fix messages logs and it worked fine. But Splunk is not able to automatically extract key-value pairs. I don't want to write extract kvdelims and pairdelims in each search. How can I automatically extract key-values using configuration files

My Original data:
19:14:59.338308 outgoing: 8=FIX.4.X|9=12|35=0|34=123456|49=ABC1|52=20170406-23:14:59.338|56=XYZ1|10=123|

After using translatefix as command:
index=abc sourcetype=xyz | translatefix
data is decoded as:

19:14:59.338308 outgoing: BeginString=FIX.4.X BodyLength=12 MsgType=Heartbeat MsgSeqNum=123456 SenderCompID=ABC1 SendingTime=20170406-23:14:59.338 TargetCompID=XYZ1 CheckSum=123

To extract key value pair I have to use extract command in search like below:

ndex="abc" sourcetype=xyz | translatefix | extract pairdelim=" " kvdelim="=" .
Is there any way I can configure to extract automatically using conf files (props.conf, transforms.conf). So I dont have to write extract in each search query and data is extracted automatically.

I tried using delims and regex (\w+)=([^[\s]+) in transforms.conf. It worked for original data but not working after using translatefix command.

[lguinn2 · Accepted solution]

Okay, I gave a dumb answer because I didn't realize that you are using translatefix inline. If you are reformatting the data on the fly (which is how translatefix works), you can't put the field extractions in props.conf or transforms.conf based on the NEW format.

However, you can certainly add the field extractions based on the original/stored format. Of course, in the original format, you don't get the "real" field names, instead you get weird names like "10" and "56." To make the names pretty, you would need to do a bunch of renames, which doesn't really help either.

I suggest a macro. Let's call it infixed_xyz() and let the body of the macro be

sourcetype=xyz | translatefix | extract pairdelim=" " kvdelim="="

now you can write a search like this

index=abc `infixed_xyx` | where TargetCompID=XYZ1 | stats or whatever

[isha_rastogi]

As translatefix is configured in commands.conf and I'm using it as command. Can I configure it in configuration file so that it will be automatically applied instead of writing it inline.

[lguinn2]

In the props.conf (on the search head or wherever users log in), add the following:

[xyz]
KV_MODE = auto

And the key-value pairs (separated by 😃 will be extracted. You do not need the transforms.conf.

[isha_rastogi]

Tried to put KV_MODE =auto. No success :(. ( Tried on new indexed data)

[adonio]

can you share sample data after it was decoded? (in plain text)

[isha_rastogi]

Yes Sure.

My Original data:
19:14:59.338308 outgoing: 8=FIX.4.X|9=12|35=0|34=123456|49=ABC1|52=20170406-23:14:59.338|56=XYZ1|10=123|

After using translatefix as command:
index=abc sourcetype=xyz | translatefix
data is decoded as:

19:14:59.338308 outgoing: BeginString=FIX.4.X BodyLength=12 MsgType=Heartbeat MsgSeqNum=123456 SenderCompID=ABC1 SendingTime=20170406-23:14:59.338 TargetCompID=XYZ1 CheckSum=123

To extract key value pair I have to use extract command in search like below:

ndex="abc" sourcetype=xyz | translatefix | extract pairdelim=" " kvdelim="=" .
Is there any way I can configure to extract automatically using conf files (props.conf, transforms.conf)