topic Field extraction with kv/extract in Splunk Search

Field extraction with kv/extract

davidha — Wed, 14 Apr 2010 21:10:33 GMT

Hi,

I am trying to extract fields of the form [key1=value with spaces] [key2=value with spaces] using the kv search command, as demonstrated on http://blogs.splunk.com/2008/02/12/delimiter-based-key-value-pair-extraction/.

I am using the search command:

... | kv pairdelim="[]" kvdelim="="

This doesn't seem to work as expected; I get instead extracted fields such as key1=value key2=value instead of what I was expecting, key1=value with spaces.

Any help would be appreciated.

Thanks!

Re: Field extraction with kv/extract

ftk — Wed, 14 Apr 2010 21:16:54 GMT

Can you post some sample data that you are trying to extract the values out of?

Re: Field extraction with kv/extract

Lowell — Wed, 14 Apr 2010 21:29:23 GMT

I don't think your example is "delimiter based" exactly.

I think the best approach for this would be to add a transforms.conf entry for this. (You can often add regex-based extractions more quickly using EXTRACT- as of Splunk 4.0+, but unfortunately this is one case where you have to do make a full-blown transforms.conf stanza because we need repeating matches, which EXTRACT doesn't support.) Also, the rex search command does not work because you are extracting both the field name and the field value; so using a transforms entry is really your best option.

Something like this should do the trick:

[my_kv_pairs_with_spaces]
REGEX = \[([^=]*)=([^\]]*)\]
FORMAT = $1::$2
MV_ADD = true

Then make sure your add an entry in your props.conf file that points to your transformer. For example:

[my_sourcetype]
REPORTS-my_fields = my_kv_pairs_with_spaces

Note: I'm assuming since you trying to use the kv command that splunk isn't just extracting these correctly out of the box for you. If you haven't tried letting Splunk do the extractions automatically, I would definitely start there.

Re: Field extraction with kv/extract

davidha — Wed, 21 Apr 2010 16:28:38 GMT

Thanks, that worked a treat. Yeah, Splunk wasn't automatically extracting fields of that form.