Splunk Search

Multi-value Field extraction

tallasky
Explorer

Hello,

I would like to create a multi-value field for my data, how can i do that?

here's a sample of my data (Starts at QAM)

event1 = QAM 32209 Prog 238 Path PATH_MW

event2 = QAM 23001 Prog 25 Path PATH_MG PATH_VERTE

event3 = QAM 46201 Prog 180 Path PATH_MD_1 PATH_JAUNE

The field i am trying to extract is Path but as you can see in the sample above, path can have a single value in the case of event1(PATH_MW) or multiple values in the case of event2 (PATH _MG PATH _VERTE) and event3 (PATH _MD _1 PATH _JAUNE)

This regular expression finds them but considers multiple values as one (?i) Path (?P[a-z_]+.[a-z_])

Here the values for event 2 and 3 are considered as one even if we have two values, what can i do so that it stores the two values in the path field for the same event but see them as two different values.

Thank you in advance.

0 Karma

tallasky
Explorer

Hello Chris, thanks again,

It's tricky to acces those files in a professional environement with many levels of security and different teams using the same Splunk, but i found a way around it:
I use this regular expression (?i) Path (?P[a-z_-]+.[a-z_-]) to get my path values wheter it One(event1) or many(event2,event3) and i use the makemv function to separate them before my research by using the space between them as delimiter

Sourcetype = ...| makemv delim = " " path

Thanks you.

chris
Motivator

Hi, I'm glad you found a solution

0 Karma

tallasky
Explorer

Hi Chris, thanks for getting back to me:

Now, what i'm trying to do is use the extract field option so this is done automatically with new data coming in everyday, your solution with rex implies that i have to use the search bar? i don't want that.

Thank you for your help.

0 Karma

chris
Motivator

the rex command has a max_matches option so you could try something like:

| rex "Path (?.*)" | rex field=temporary_field max_match=10 "(?[a-z_]+)"


Update

To do this automatically you will have to use the regular expression in a configs file called props.conf & transforms.conf. You can place those files in $SPLUNK_HOME/etc/system/local to start.
props.conf

[my_sourcetype]
TRANSFORMS-tempfield = tempfield
TRANSFORMS-mv_field = mv_field

transforms.conf

[tempfield]
REGEX=Path (.*)
FORMAT=temporary_field::$1

[mv_field]
SOURCE_KEY=temporary_field
REGEX=([a-z_]+)
FORMAT=mv_field::$1
MV_ADD=true

There is more information here --> documentation. I also suggest that you read about Technology Add ons the ESS and PCI Apps/Suites from Splunk are built on this approach. The idea is to put all the configuration that is necessary to parse & extract fields for a technology into an app that can be used by all the splunk users/apps at your site. Does that make sense?

chris
Motivator

Hi, I updated the answer let me know if this works for you

tallasky
Explorer

Hi Chris, thanks for getting back to me:

Now, what i'm trying to do is use the extract field option so this is done automatically with new data coming in everyday, your solution with rex implies that i have to use the search bar? i don't want that.

Thank you for your help.

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...