Splunk Search

Multiple key value pair extraction

timbCFCA
Path Finder

I have multiple key value pairs in a line like so:
summary=" Policy Rule modified" summary=" Policy Rule number 2 modified" details="Details line 1" details="Details line 2" ... details="Details line X".

Nothing I'm doing provides an easy way to either concatenate or preferably create keys with increasing numerical sequence, eg summary=, summary2=, etc. I've tried the MV_ADD and REPEAT_MATCH transform options without result. What do I need? It feels like something basic I'm missing but I can't put my finger on it.

[mytype]
REGEX = details=(\S+)
FORMAT = details::$1
REPEAT_MATCH = true

REGEX = summary=(\S+)
FORMAT = summary::$1
MV_ADD = true
Tags (1)
1 Solution

hazekamp
Builder

timbCFCA,

The MV_ADD transforms settings is what you want here. REPEAT_MATCH is only valid @ index time. Given the following sample the following configs will be valid:

summary=" Policy Rule modified" summary=" Policy Rule number 2 modified" details="Details line 1" details="Details line 2" ... details="Details line X"

## props.conf
[<your_sourcetype>]
REPORT-details_for_your_sourcetype = details_for_your_sourcetype
REPORT-summary_for_your_sourcetype = summary_for_your_sourcetype

## transforms.conf
[details_for_your_sourcetype]
# Assuming quoted strings
REGEX = details=\"([^"]+)
FORMAT = details::$1
MV_ADD = True

[summary_for_your_sourcetype]
# Assuming quoted strings
REGEX = summary=\"([^"]+)
FORMAT = summary::$1
MV_ADD = True

View solution in original post

supersleepwalke
Communicator

I am experiencing this same issue. I would have thought that using "| extract mv_add=true" would fix it in the search interface, but that appears to have no effect. Has anybody experienced that?

I'm able to easily solve this using "| rex ... max_match=50", but that seems like it ought to be unnecessary.

0 Karma

hazekamp
Builder

timbCFCA,

The MV_ADD transforms settings is what you want here. REPEAT_MATCH is only valid @ index time. Given the following sample the following configs will be valid:

summary=" Policy Rule modified" summary=" Policy Rule number 2 modified" details="Details line 1" details="Details line 2" ... details="Details line X"

## props.conf
[<your_sourcetype>]
REPORT-details_for_your_sourcetype = details_for_your_sourcetype
REPORT-summary_for_your_sourcetype = summary_for_your_sourcetype

## transforms.conf
[details_for_your_sourcetype]
# Assuming quoted strings
REGEX = details=\"([^"]+)
FORMAT = details::$1
MV_ADD = True

[summary_for_your_sourcetype]
# Assuming quoted strings
REGEX = summary=\"([^"]+)
FORMAT = summary::$1
MV_ADD = True

timbCFCA
Path Finder

hazedav, many thanks. I realized that I needed to define two separate entries in props.conf just as I received your response. I went a touch different route for the regex. I'll post just in case anyone comes across this.

[forechange_details]
REGEX = details=\"(.+?)\"
FORMAT = details::$1
MV_ADD = true

[forechange_summary]
REGEX = summary=\"(.+?)\"
FORMAT = summary::$1
MV_ADD = true
Get Updates on the Splunk Community!

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

(view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...