Regex Extraction

[bhavneeshvohra9]

HI All,

PLease help in extracting the below logs as per the conditions stated

key1=value1,key2=value2\,key 3=value3\,value4\,value5\,value6\,key4={key5=value5\,key6= value6\,},key7=val10(key8=val11\,key9=val12),key20=val20

i have a log of the above format

i want to extract key value pairs without backslash

also key 3=value3\,value4\,value5\,value6\,  is a multivalue field and should be extracted  as  :-

key 3=value3,value4,value5,value6

and key4={key5=value5\,key6= value6\,}, is a json object  which should be extracted as:-

  key5= value5 , key6= value 6(i.e key 4={ should not be extracted)

 and key7=val10(key8=val11\,key9=val12) should be extracted individually as :-

key7=val10 ,key8=val11,key9=val12

[scelikok]

Hi @bhavneeshvohra9,

Can you try following? The only change you should do is "key_3" field name. I needed to use second transform for multi-value fields. I hope multi-value field names are specific.

props.conf
[your_sourcetype]
REPORT-kv = extract_kv, extract_mv
EVAL-key_3 = split(replace(key_3,"\\\\",""),",")
KV_MODE = none

transforms.conf
[extract_mv]
FORMAT = key_3::$1
REGEX = key\s3=(.+?),\w+=

[extract_kv]
FORMAT = $1::$2
REGEX = ([^=,{\s]+)=([^=,\\{()]+)

[richgalloway]

Regular expressions like to work with consistent data.  So, are these keys always the same and always in the same order?  Are any of them optional?

Can you go back to the developers and ask them to log in a more consistent format, like all-JSON or all K=V?  Something like this would be far easier to parse.

key1=value1;key2=value2;key 3=value3,value4,value5,value6;key5=value5;key6= value6;key7=val10;key8=val11;key9=val12;key20=val20

[bhavneeshvohra9]

Hi ,

 Thank you for your response .. it is not possible to change the data format

everything in the log format is  in forms of key value pairs like key1= val1

but  for some keys there is a '\'gettinga dded ti the values like key2=val2\,

also there are some keys having multiple values like key3=val3\,val4\,val5\,   for which we need extarction like key3=val3,val4,val5

also there are scenarios where its in form of object like key4= {key5=val5\, key6=val6},

for which we only need the values inside { like--> key5= val5 and key6= val6

[richgalloway]

Thank you for restating the problem.

Regular expressions like to work with consistent data.  So, are these keys always the same and always in the same order?  Are any of them optional?