Splunk Search

Field extraction with kv/extract

davidha
New Member

Hi,

I am trying to extract fields of the form [key1=value with spaces] [key2=value with spaces] using the kv search command, as demonstrated on http://blogs.splunk.com/2008/02/12/delimiter-based-key-value-pair-extraction/.

I am using the search command:

... | kv pairdelim="[]" kvdelim="="

This doesn't seem to work as expected; I get instead extracted fields such as key1=value key2=value instead of what I was expecting, key1=value with spaces.

Any help would be appreciated.

Thanks!

0 Karma
1 Solution

Lowell
Super Champion

I don't think your example is "delimiter based" exactly.

I think the best approach for this would be to add a transforms.conf entry for this. (You can often add regex-based extractions more quickly using EXTRACT- as of Splunk 4.0+, but unfortunately this is one case where you have to do make a full-blown transforms.conf stanza because we need repeating matches, which EXTRACT doesn't support.) Also, the rex search command does not work because you are extracting both the field name and the field value; so using a transforms entry is really your best option.

Something like this should do the trick:

[my_kv_pairs_with_spaces]
REGEX = \[([^=]*)=([^\]]*)\]
FORMAT = $1::$2
MV_ADD = true

Then make sure your add an entry in your props.conf file that points to your transformer. For example:

[my_sourcetype]
REPORTS-my_fields = my_kv_pairs_with_spaces

Note: I'm assuming since you trying to use the kv command that splunk isn't just extracting these correctly out of the box for you. If you haven't tried letting Splunk do the extractions automatically, I would definitely start there.

View solution in original post

Lowell
Super Champion

I don't think your example is "delimiter based" exactly.

I think the best approach for this would be to add a transforms.conf entry for this. (You can often add regex-based extractions more quickly using EXTRACT- as of Splunk 4.0+, but unfortunately this is one case where you have to do make a full-blown transforms.conf stanza because we need repeating matches, which EXTRACT doesn't support.) Also, the rex search command does not work because you are extracting both the field name and the field value; so using a transforms entry is really your best option.

Something like this should do the trick:

[my_kv_pairs_with_spaces]
REGEX = \[([^=]*)=([^\]]*)\]
FORMAT = $1::$2
MV_ADD = true

Then make sure your add an entry in your props.conf file that points to your transformer. For example:

[my_sourcetype]
REPORTS-my_fields = my_kv_pairs_with_spaces

Note: I'm assuming since you trying to use the kv command that splunk isn't just extracting these correctly out of the box for you. If you haven't tried letting Splunk do the extractions automatically, I would definitely start there.

View solution in original post

davidha
New Member

Thanks, that worked a treat. Yeah, Splunk wasn't automatically extracting fields of that form.

0 Karma

ftk
Motivator

Can you post some sample data that you are trying to extract the values out of?

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!