Multiple key value pair extractions

huaraz — Fri, 09 Sep 2011 14:55:05 GMT

I have a logfile with the following format:

LOG: : ; : ; .....

If I had only one key value pair I think could do

[mylog]

REGEX = LOG: (\S+): (.*);

Format = $1::$2

or two pairs

[mylog]

REGEX = LOG: (\S+): (.); (\S+): (.);

Format = $1::$2 $3::$4

but what can I do if I have an unknown number of pairs ?

Thank you

Markus

Re: Multiple key value pair extractions

borisalves — Sat, 11 Feb 2012 01:03:09 GMT

I just had to deal with a similar issue for Json data. I am running 4.2 and do not have the spath() available this is what I did:

Initial search :

BesRttSorterImpl

result=
[2012-02-10 14:31:12 PST] BesRttSorterImpl - DEBUG: RTT Site scores={"AAA":39.88540275,"BBB":65.32070525,"CCC":148.4583085}

First thing to pay attention is that json always returns a structure field, in this case scores is the main field. So first addintion to the search is ignore any results without scores

Search 1:BesRttSorterImpl scores

The overall goal now, is to have each value pair show as a field, in this case called multi-valued field, also used as “mv”. In order to do that, we execute what splunk does best, remove what we do not want to see.

scores={"AAA":39.88540275,"BBB":65.32070525,"CCC":148.4583085}

We will use the function eval for that. What to know about eval, it executes functions and returns to another field,

So it works some what like this:

Search | eval result=function (X,Y)

For start we will replace {“ with blank to a field called results

| eval result=replace(scores,"{\"","")

Note the use of \ to mean the value of “ instead of using quote as a delimiter

Then we will remove the other }

| eval result2=replace(result,"}","")

We will now remove all “ from the results

| eval result3=replace(result2,"\"","")

Now we replace the Json delimiter with “=”

| eval result4=replace(result3,":","=")

Final step is t make subfields using split

| eval result5=split(result4,",")

Now we have all the values in result5, to be present then we use table.

Table result5

Putting all together:

BesRttSorterImpl scores | eval result=replace(scores,"{\"","") | eval result2=replace(result,"}","") | eval result3=replace(result2,"\"","") | eval result4=replace(result3,":","=") | eval result5=split(result4,",") | table result5

will show:
AAA=value
BBB=Value
CCC=Value

If the elements are not in the order you want, then Further manipulation is required to have all element match Using If() and Match() can do that.

Not elegant, but worked for me.

Re: Multiple key value pair extractions

willthames2 — Mon, 28 Sep 2020 11:23:29 GMT

Have you looked at DELIMS rather than REGEX?

http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Createandmaintainsearch-timefieldextractionsthroughconfigurationfiles#Configuring_delimiter-based_field_extraction

Not sure how that would cope with the LOG at the start of the line, but I imagine you could always strip it off before DELIMS.

topic Re: Multiple key value pair extractions in Getting Data In

Multiple key value pair extractions

Re: Multiple key value pair extractions

Re: Multiple key value pair extractions