topic Re: Regex Help Required in Getting Data In

Regex Help Required

anandhalagaras1 — Thu, 03 Dec 2020 08:06:37 GMT

Hi Team,

I need a help to extract all the fields in the Wineventlog post the message information in the log. And all the data are getting delimited based on "=".

Sample Event:

10/26/2020 04:44:22 PM
LogName=xyz
SourceName=abc
EventCode=ddd
EventType=d
Type=Warning
ComputerName=xyz.abc.com
User=NOT_TRANSLATED
Sid=x-d-d-dd
SidType=d
TaskCategory=xxxx
OpCode=xxxx
RecordNumber=dddd
Keywords=xxxxxxx
Message=An infection has been found
Date/time of event = 2020-10-26 16:44:22
Event Severity Level = xxxx
Scan Rule = xx yy zz
URL = no_path
File name = yy.com
File status = xxxxxx
Component name = xxxxxx.com
Component disposition = abc
Virus name = abc xxx yyy
Virus ID = 00000
Virus definitions = 000000.000
Client IP = xxx.xx.xxx.xx
Scan Duration (sec) = x.xxx
Connect Duration (sec) = x.xxx
Symantec Protection Engine IP address = xx.xxx.xxx.xxx
Symantec Protection Engine Port number = xxxx
Uptime (in seconds) = xxxxxxx
Uber Category = xyz
Sub Category Name = abc
Sub Category ID = c
Sub Category Description = Programs that infect other programs, files, or areas of a computer by inserting themselves or attaching themselves to that medium.
Cumulative Risk Rating = xyz
Performance impact = xyz
Privacy impact = xyz
Ease of removal = xyz
Stealth = xyz
Date/time of event(with millisec) = 2020-10-26 16:44:22:617
Symantec Protection Engine Host Name = xxxxxx

So if i use the below mentioned props & transforms I can able to extract the fields till "Message" only and after which the fields are not getting extracted so kindly help to check and update my regex so that i should be able to extract all the fields post Message field in the log.

i.e. Till Message field there is no "space" included but after the Message field we can see space character is allowed.

props.conf
[yoursourcetype]
REPORT-ZZcustom_msg_kv = custom_msg_kv

transforms.conf
[custom_msg_kv]
SOURCE_KEY = message
REGEX = ([a-zA-Z]\w+)=(.*?)(?=\s+[a-zA-Z]\w+=|$)
FORMAT = $1::$2

Also correct me if the props and transforms are correct and the format is also correct or not?

@FrankVl Kindly help on my query.

Re: Regex Help Required

rnowitzki — Thu, 03 Dec 2020 08:46:40 GMT

Hi @anandhalagaras1 ,

This seems to work:

(.*)=(.*?)(?=\s+[a-zA-Z]\w+=|$)

BR
Ralph

Re: Regex Help Required

anandhalagaras1 — Thu, 03 Dec 2020 09:10:55 GMT

@rnowitzki

Thank you for your swift response. So I had just validated with the https://regex101.com/ and its working perfectly.

So I need another help i.e. I don't want to extract the fields based on sourcetype information since all the events are coming in a single sourcetype so do we have any possibility to extract the fields based on SourceName.

So consider an example my sourcename as "xyz"

So I have just gave a try so kindly check and correct me if i am wrong. So kindly validate me props and transforms and update me.

props.conf
[SourceName:xyz]
REPORT-ZZcustom_msg_kv = custom_msg_kv

transforms.conf
[custom_msg_kv]
SOURCE_KEY = message
REGEX = (.*)=(.*?)(?=\s+[a-zA-Z]\w+=|$)
FORMAT = $1::$2

Thank you.

Re: Regex Help Required

rnowitzki — Thu, 03 Dec 2020 10:18:45 GMT

The stanza to refer to the source should be

[source::xyz]

Besides that, it looks fine.

Maybe test it local with a single file, before pushing to production.

BR
Ralph