topic How to extract each key=value as a new field from a multikv field at search-time? in Splunk Search

How to extract each key=value as a new field from a multikv field at search-time?

zeophlite — Wed, 06 Jul 2016 10:13:38 GMT

At search-time, I've been able to massage my data into a multikv field like so:

Is it possible to extract each key=value as a new field into my event, without specifying the key or the mvindex ?

i.e. , I can manually extract individual fields using a method like this:

| eval "Target type"=mvindex(kvpairs, 1)
| eval "Target name"=mvindex(kvpairs, 2)

But I don't know all possible key's beforehand (they are provided by a 3rd party system).

Re: How to extract each key=value as a new field from a multikv field at search-time?

javiergn — Wed, 06 Jul 2016 10:29:27 GMT

Try this:

your base search here
| rex field=kvpairs max_match=0 "(?mi)(?<keyvalue>[^\n]+)"
| mvexpand keyvalue
| rex field=keyvalue "(?i)^(?<key>[^=]+)=(?<value>.*)$"
| eval {key} = value
| fields - kvpairs, keyvalue, key, value
| stats first(*) as * by _raw

Example:

| stats count | fields - count
| eval _raw = "
Host=iorsdb;Target type=Cluster Database;Target name=<a href=\"https://poem12.
"
| eval kvpairs = split(_raw, ";")
| rex field=kvpairs max_match=0 "(?mi)(?<keyvalue>[^\n]+)"
| mvexpand keyvalue
| rex field=keyvalue "(?i)^(?<key>[^=]+)=(?<value>.*)$"
| eval {key} = value
| fields - kvpairs, keyvalue, key, value
| stats first(*) as * by _raw
| fields - _raw

Output: see picture below

Re: How to extract each key=value as a new field from a multikv field at search-time?

zeophlite — Wed, 06 Jul 2016 11:06:52 GMT

Is there a way to do this without the mvexpand ? I want to keep the raw events together, just augment them with additional fields

Re: How to extract each key=value as a new field from a multikv field at search-time?

javiergn — Wed, 06 Jul 2016 11:19:16 GMT

You can revert mvexpand afterwards with stats.
If you don't separate your kvfields into different events you can't use eval {key} = value in the right way.

Try without the mvexpand and see what happens.

As an alternative, give this a go:

your base search
| rex field=kvpairs max_match=0 "(?mi)(?<keyvalue>[^\n]+)"
| mvexpand keyvalue
| rex field=keyvalue "(?i)^(?<key>[^=]+)=(?<value>.*)$"
| eval {key} = value
| fields - keyvalue, key, value
| stats values(kvpairs) as kvpairs,  first(*) as * by _raw

Re: How to extract each key=value as a new field from a multikv field at search-time?

sundareshr — Wed, 06 Jul 2016 12:33:30 GMT

Have you tried the extract command. Something like this should work

.... | extract pairdelim="\n" kvdelim="=" | ...

http://docs.splunk.com/Documentation/Splunk/6.4.1/SearchReference/extract

Re: How to extract each key=value as a new field from a multikv field at search-time?

zeophlite — Wed, 06 Jul 2016 12:38:42 GMT

Hi @sundareshr , extract only works on _raw , I want to run it on kvpairs , which is a multivalue field