Splunk Search

How to extract a repeating field from an event

New Member

I have data that looks like this:

AA=value1,BB=value2,BB=value3,BB=value4
AA=value5,BB=value6,BB=value7
AA=value8,BB=value9
AA=value10,BB=value11,BB=value12

How do I extract all the BB fields for each AA field?

Tags (1)
0 Karma
1 Solution

Legend

The following will automatically extract all key-value pairs (AA=value and BB=value,etc). This will include the CC fields, too.

What makes this different that the automatic field extraction is that it allows for multiple values for any field. So if there is more than one AA or BB or CC in the same event, the field will have multiple values.

In props.conf

[yoursourcetypehere]
KV_MODE = none
REPORT-ref = extract_fields

In transforms.conf

[extract_fields]
MV_ADD = true
DELIMS = ",", "="

Finally, if you want to see if this works, just run a short search like this:

yoursearchhere
| table AA BB

View solution in original post

SplunkTrust
SplunkTrust

With your data:

<your search> | rex "AA=(?P<aa>[^,]+),[^B]*BB=(?P<bb_1>[^,]+)(,[^B]*BB=(?P<bb_2>[^,]+)(,[^B]*BB=(?P<bb_3>[^,]+))?)?"

If you need to have a "B" in your values, this may not work. If you have more than 3 sets of BB entries, you will have to add to this. If you could have another value (like CC=) before a BB, then you will have to change it a little, too.

Multi value stuff doesn't seem to me to work as well for this, because there isn't a list, like BB=1,2,3;CC=5,6,7, so this seems to be a reasonable alternative.

0 Karma

Legend

The following will automatically extract all key-value pairs (AA=value and BB=value,etc). This will include the CC fields, too.

What makes this different that the automatic field extraction is that it allows for multiple values for any field. So if there is more than one AA or BB or CC in the same event, the field will have multiple values.

In props.conf

[yoursourcetypehere]
KV_MODE = none
REPORT-ref = extract_fields

In transforms.conf

[extract_fields]
MV_ADD = true
DELIMS = ",", "="

Finally, if you want to see if this works, just run a short search like this:

yoursearchhere
| table AA BB

View solution in original post

New Member

If I implement this on our production splunk infrastructure will this 'break' any existing reports and dashboards?

0 Karma

Legend

Probably not. The transformation will extract all the same fields that the automatic extraction finds; it just adds the multi-value fields ability.

However, if you have reports or dashboards that use the BB and CC fields, they might give a different answer now. But they were probably giving a wrong answer before...

0 Karma

Communicator

Do you know how to do this if they weren't in KV pairs?

0 Karma

New Member

Actually...

The data looks a bit more like this;

AA=value1,BB=value2,CC=value3,BB=value4,BB=value5
AA=value6,BB=value7,BB=value8
AA=value9,BB=value10,CC=value11
AA=value12,BB=value13,CC=value14,BB=value15,CC=value16

There are other fields (CC) that I am not interested in that are mixed up with the BB fields. There is always only one AA field per event. Each BB field may have an associated CC field.

I am only interested in the BB fields per AA

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!