Field extraction with kv/extract

[davidha]

Hi,

I am trying to extract fields of the form [key1=value with spaces] [key2=value with spaces] using the kv search command, as demonstrated on http://blogs.splunk.com/2008/02/12/delimiter-based-key-value-pair-extraction/.

I am using the search command:

... | kv pairdelim="[]" kvdelim="="

This doesn't seem to work as expected; I get instead extracted fields such as key1=value key2=value instead of what I was expecting, key1=value with spaces.

Any help would be appreciated.

Thanks!

[Lowell · Accepted solution]

I don't think your example is "delimiter based" exactly.

I think the best approach for this would be to add a transforms.conf entry for this. (You can often add regex-based extractions more quickly using EXTRACT- as of Splunk 4.0+, but unfortunately this is one case where you have to do make a full-blown transforms.conf stanza because we need repeating matches, which EXTRACT doesn't support.) Also, the rex search command does not work because you are extracting both the field name and the field value; so using a transforms entry is really your best option.

Something like this should do the trick:

[my_kv_pairs_with_spaces]
REGEX = \[([^=]*)=([^\]]*)\]
FORMAT = $1::$2
MV_ADD = true

Then make sure your add an entry in your props.conf file that points to your transformer. For example:

[my_sourcetype]
REPORTS-my_fields = my_kv_pairs_with_spaces

Note: I'm assuming since you trying to use the kv command that splunk isn't just extracting these correctly out of the box for you. If you haven't tried letting Splunk do the extractions automatically, I would definitely start there.

[davidha]

Thanks, that worked a treat. Yeah, Splunk wasn't automatically extracting fields of that form.

[ftk]

Can you post some sample data that you are trying to extract the values out of?