Splunk Search

How to extract each key=value as a new field from a multikv field at search-time?

zeophlite
New Member

At search-time, I've been able to massage my data into a multikv field like so:
alt text

Is it possible to extract each key=value as a new field into my event, without specifying the key or the mvindex ?

i.e. , I can manually extract individual fields using a method like this:

| eval "Target type"=mvindex(kvpairs, 1)
| eval "Target name"=mvindex(kvpairs, 2)

But I don't know all possible key's beforehand (they are provided by a 3rd party system).

0 Karma

sundareshr
Legend

Have you tried the extract command. Something like this should work

.... | extract pairdelim="\n" kvdelim="=" | ...

http://docs.splunk.com/Documentation/Splunk/6.4.1/SearchReference/extract

zeophlite
New Member

Hi @sundareshr , extract only works on _raw , I want to run it on kvpairs , which is a multivalue field

0 Karma

javiergn
Super Champion

Try this:

your base search here
| rex field=kvpairs max_match=0 "(?mi)(?<keyvalue>[^\n]+)"
| mvexpand keyvalue
| rex field=keyvalue "(?i)^(?<key>[^=]+)=(?<value>.*)$"
| eval {key} = value
| fields - kvpairs, keyvalue, key, value
| stats first(*) as * by _raw

Example:

| stats count | fields - count
| eval _raw = "
Host=iorsdb;Target type=Cluster Database;Target name=<a href=\"https://poem12.
"
| eval kvpairs = split(_raw, ";")
| rex field=kvpairs max_match=0 "(?mi)(?<keyvalue>[^\n]+)"
| mvexpand keyvalue
| rex field=keyvalue "(?i)^(?<key>[^=]+)=(?<value>.*)$"
| eval {key} = value
| fields - kvpairs, keyvalue, key, value
| stats first(*) as * by _raw
| fields - _raw

Output: see picture below

alt text

0 Karma

zeophlite
New Member

Is there a way to do this without the mvexpand ? I want to keep the raw events together, just augment them with additional fields

0 Karma

javiergn
Super Champion

You can revert mvexpand afterwards with stats.
If you don't separate your kvfields into different events you can't use eval {key} = value in the right way.

Try without the mvexpand and see what happens.

As an alternative, give this a go:

your base search
| rex field=kvpairs max_match=0 "(?mi)(?<keyvalue>[^\n]+)"
| mvexpand keyvalue
| rex field=keyvalue "(?i)^(?<key>[^=]+)=(?<value>.*)$"
| eval {key} = value
| fields - keyvalue, key, value
| stats values(kvpairs) as kvpairs,  first(*) as * by _raw
0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...