Hello, I would like to ask about problem with parsing log using regex with lookahead. I have this log: Oct 10 04:18:31 ATLAS Threat Categories|Blocked Host|7|rt=1633832250000 src=122.226.102.59...
See more...
Hello, I would like to ask about problem with parsing log using regex with lookahead. I have this log: Oct 10 04:18:31 ATLAS Threat Categories|Blocked Host|7|rt=1633832250000 src=122.226.102.59 cs3Label=Match Type dpt=23 cn2=13 proto=TCP dst=193.85.146.63 cn1=21129644 spt=39528 cs2Label=Protection Group Name cs1Label=IOC Pattern cn1Label=Element Id cn2Label=Protection Group ID cs7Label=Threat Category cs7=Malware cs6=Telnet Bruteforce cs1=122.226.102.59 cs6Label=Threat Name cs3=ip cs2=Default Protection Group As you can see, there are a number of parameters for which two fields are always used, eg "cs1Label" (parameter name) and "cs1" (parameter value). My goal is to create a new field in search time parsing phase, which will be the value of the field "cs1Label" and the value will be the value of the field "cs1". For example: IOC Pattern = 122.226.102.59 The problem is that the log does not have a fixed structure, the order of the individual fields changes. Therefore, "normal" regex cannot be used. So I created the following regex using a lookahead that parses the appropriate values (I tested it in a regex101.com tester and it works): ^(?=.*\bcs1Label=\b((.*?)((\s\w+\=)|($))))(?=.*\bcs1=\b((.*?)((\s\w+\=)|($)))) Unfortunately, when I use it in Splunk, in transforms.conf, it doesn't work. My transforms.conf looks like this: [combined_field_cs1]
SOURCE_KEY = _raw
REGEX = ^(?=.\bcs1Label=\b((.?)((\s\w+=)|($))))(?=.\bcs1=\b((.?)((\s\w+=)|($))))
FORMAT = $2::$7 props.conf REPORT-combined_field_cs1 = combined_field_cs1 Strictly speaking, when I simply want to look at the messages in a given index, the search freezes, it does not display any messages and I have to close it manually. I made some testing and it obvious that REGEX is the problem. It is not clear to me why the regex in Splunk does not work. Or did I choose the completely wrong path and need to use a completely different way to achieve my goal? Could yomeone more experinced help? Any help will be highly appreciated. Best regards Lukas Mecir