Multiple key value pairs during search

sc0tt — Thu, 07 Nov 2013 11:47:11 GMT

In order to reduce the amount of data being indexed, I am using a sed script to strip away all XML tags and to format the data as key=value. Sometimes there are multiple key-value pairs like in the below sample event.

2013-11-07 03:20:24,637 Outgoing UserId="555555555555" Type="Main" To="user@example.com"  Key="1" Text="Value 1" Key="2" Text="Value 2" Key="3" Text="Value 3" SessionId="1000"

I want the search to include all occurrences of multiple key-value pairs (i.e. search..| stats count by Key). Currently, I will only get Key=1 in my results. I have added MV_ADD to transforms.conf but it is not working. Below is my current configuration.

props.conf
[source::/path/to/test.txt]
SHOULD_LINEMERGE = False
SEDCMD-trim = s/<([^\s\>]*)[^\>]*\>([^<]*)\<\/\1\>/ \1="\2"/g
REPORT-mvfield = mv-field 

transforms.conf
[mv-field]
MV_ADD = true

What am I missing?

Update:

Based on this post I was able to determine that I needed to add a transforms for each field.

[mv-field-key]
REGEX = Key=\"(.+?)\"
FORMAT = Key::$1
MV_ADD = true

[mv-field-text]
REGEX = \bText\b=\"(.+?)\"
FORMAT = Text::$1
MV_ADD = true

Is it possible to have a single stanza so that I don't have to specify each field that I want to apply MV_ADD?

Re: Multiple key value pairs during search

_d_ — Thu, 07 Nov 2013 14:44:25 GMT

Yes, you can do this instead of individual extractions:

[mv-field]
REGEX  = (?<_KEY_1>\w+)=\"(?<_VAL_1>.*?)\"
MV_ADD = true

_KEY_1 and _VAL_1 have special meanings in transforms.conf

Edit: Change regexes in the named capture groups to fit your needs.

Re: Multiple key value pairs during search

sc0tt — Thu, 07 Nov 2013 16:16:14 GMT

Perfect, thanks!

topic Multiple key value pairs during search in Splunk Search

Multiple key value pairs during search

Re: Multiple key value pairs during search

Re: Multiple key value pairs during search