topic Re: Multi Value Fields Extraction using Props and Transform in Splunk Search

Multi Value Fields Extraction using Props and Transform

shayhibah — Wed, 08 Jan 2020 18:06:32 GMT

Hi,

I have log in the following format:

time=12345678|hostname=shayh|product=blade1<>blade2<>blade3|username:sha@gmail.com|

In order to extract these fields, I used both props.conf and transform.conf:

props.conf:

[my_format]
REPORT-my_format = my_format

transform.conf:

[my_format]
FORMAT = $1::$2
REGEX = ([a-zA-Z0-9_]+)=([^|]+)
MV_ADD = true

I am able to get the following fields:

time=12345678
hostname=shayh
product=blade1<>blade2<>blade3
username:sha@gmail.com

I noticed that I have an issue only with multi-value fields extraction.
multi-value fields may contains many values separated by "<>".

How can I change my settings to support current behavior with multi-value fields extraction?

UPDATE:
I succeeded to split it using fields.conf and TOKENIZER but on Splunk UI I still see it as:
product=blade1<>blade2<>blade3

Is it possible to solve it too? maybe by replacing <> with \n)?

Re: Multi Value Fields Extraction using Props and Transform

richgalloway — Wed, 08 Jan 2020 18:36:34 GMT

Please share some sample data, anonymized as necessary.

Re: Multi Value Fields Extraction using Props and Transform

oscar84x — Wed, 08 Jan 2020 20:21:18 GMT

If the format is field1=value1|field2=value2|... you can try:

TRANSFORMS
    [pipe_eq]
    DELIMS = "|", "="

PROPS
   [my_format]
   TRANSFORMS-delims = pipe_eq

Re: Multi Value Fields Extraction using Props and Transform

shayhibah — Wed, 08 Jan 2020 20:56:52 GMT

@richgalloway
Hi,

Here is an example:
time=12345678|hostname=shayh|product=blade1<>blade2<>blade3|username:sha@gmail.com

Right now the pairs are perfect but I would like that field 'product' (for example) will be multi-value

Re: Multi Value Fields Extraction using Props and Transform

richgalloway — Thu, 09 Jan 2020 00:59:02 GMT

It looks like your current REGEX will work except for the 'username' field since it is not in K=V format. I would let Splunk extract the 'product' field as-is and use the split function to break it up at search time.

Re: Multi Value Fields Extraction using Props and Transform

shayhibah — Thu, 09 Jan 2020 06:05:52 GMT

I need it to be done automatically using configuration files and not on the search bar itself.

Re: Multi Value Fields Extraction using Props and Transform

shayhibah — Thu, 09 Jan 2020 06:06:37 GMT

its not working.
The current parsing is OK, I just need to support multi-value fields as well as single values.

Re: Multi Value Fields Extraction using Props and Transform

to4kawa — Thu, 09 Jan 2020 11:39:38 GMT

| makeresults 
| eval _raw="time=12345678|hostname=shayh|product=blade1<>blade2<>blade3|username:sha@gmail.com|" 
| kv 
| eval product=split(product,"<>")

Hi, @shayhibah

please try Fields » Calculated fields » Add new

eval expression is #4

Define calc fields