topic Regex Extraction in Splunk Dev

Regex Extraction

bhavneeshvohra9 — Tue, 12 Jan 2021 12:11:17 GMT

HI All,

PLease help in extracting the below logs as per the conditions stated

key1=value1,key2=value2\,key 3=value3\,value4\,value5\,value6\,key4={key5=value5\,key6= value6\,},key7=val10(key8=val11\,key9=val12),key20=val20

i have a log of the above format

i want to extract key value pairs without backslash

also key 3=value3\,value4\,value5\,value6\, is a multivalue field and should be extracted as :-

key 3=value3,value4,value5,value6

and key4={key5=value5\,key6= value6\,}, is a json object which should be extracted as:-

key5= value5 , key6= value 6(i.e key 4={ should not be extracted)

and key7=val10(key8=val11\,key9=val12) should be extracted individually as :-

key7=val10 ,key8=val11,key9=val12

Re: Regex Extraction

richgalloway — Tue, 12 Jan 2021 15:15:30 GMT

Regular expressions like to work with consistent data. So, are these keys always the same and always in the same order? Are any of them optional?

Can you go back to the developers and ask them to log in a more consistent format, like all-JSON or all K=V? Something like this would be far easier to parse.

key1=value1;key2=value2;key 3=value3,value4,value5,value6;key5=value5;key6= value6;key7=val10;key8=val11;key9=val12;key20=val20

Re: Regex Extraction

bhavneeshvohra9 — Tue, 12 Jan 2021 15:22:40 GMT

Hi ,

Thank you for your response .. it is not possible to change the data format

everything in the log format is in forms of key value pairs like key1= val1

but for some keys there is a '\'gettinga dded ti the values like key2=val2\,

also there are some keys having multiple values like key3=val3\,val4\,val5\, for which we need extarction like key3=val3,val4,val5

also there are scenarios where its in form of object like key4= {key5=val5\, key6=val6},

for which we only need the values inside { like--> key5= val5 and key6= val6

Re: Regex Extraction

richgalloway — Tue, 12 Jan 2021 16:38:26 GMT

Thank you for restating the problem.

Regular expressions like to work with consistent data. So, are these keys always the same and always in the same order? Are any of them optional?

Re: Regex Extraction

scelikok — Tue, 12 Jan 2021 17:22:44 GMT

Hi @bhavneeshvohra9,

Can you try following? The only change you should do is "key_3" field name. I needed to use second transform for multi-value fields. I hope multi-value field names are specific.

props.conf [your_sourcetype] REPORT-kv = extract_kv, extract_mv EVAL-key_3 = split(replace(key_3,"\\\\",""),",") KV_MODE = none transforms.conf [extract_mv] FORMAT = key_3::$1 REGEX = key\s3=(.+?),\w+= [extract_kv] FORMAT = $1::$2 REGEX = ([^=,{\s]+)=([^=,\\{()]+)