topic Re: How to use regex to filter out logs? in Splunk Search

How to use regex to filter out logs?

martaBenedetti — Fri, 02 Jul 2021 08:21:45 GMT

Hi community,

I have the need to exclude AIX logs containing a certain field value.

This is the regex the parser is using to extract vendor_action filed:

^\w+\s+\w+\s+\d+\s+\d+\:\d+\:\d+\s+\d+\s+(?<pid>\d+)\s+(?<ppid>\d+)\s+(?<user>\S+)\s+(?<process>\S+)\s+(?<vendor_action>\S+)\s+(?<status>\S+)

I'm trying to exclude events that contain vedor_action=FILE_Unlink and these are my conf file located on Heavy Forwarder:

props.conf

[aix:audit] TRANSFORMS-null= setnull

transforms.conf

[setnull] REGEX = ^\w+\s+\w+\s+\d+\s+\d+\:\d+\:\d+\s+\d+\s+\d+\s+\d+\s+\S+\s+\S+\s+FILE_Unlink\s+\S+ DEST_KEY = queue FORMAT = nullQueue

There are sample logs: the first one should be excluded while the second one no:

Fri Jul 02 10:01:49 2021 34078844 8520050 dbloader rm FILE_Unlink OK Not supported filename /tmp/CSI_ODS_M_SIA__INFO_RILANCIO.txt Fri Jul 02 10:01:46 2021 34930828 4587668 root root lsvg FILE_Unlink OK filename /dev/__pv17.0.34930828

When I restart spunk all logs are excluded, so I think something is wrong with my REGEX even if on regex101 seems to work fine.

Any ideas?

Thanks a lot

Marta

Re: How to use regex to filter out logs?

ITWhisperer — Fri, 02 Jul 2021 09:17:40 GMT

The first one has 2 words between the numbers and FILE_Unlink whereas the second one has 3 words - your regex only caters for the first case

Re: How to use regex to filter out logs?

martaBenedetti — Tue, 06 Jul 2021 15:50:23 GMT

@ITWhisperer that's right: the first one should be excluded with nullQueue and the second one should be indexed.

The problem though is that all logs are excluded.

Re: How to use regex to filter out logs?

martaBenedetti — Tue, 06 Jul 2021 15:49:17 GMT

@ITWhisperer Do you have suggestion on how to do so?

That is filter out the first kind of log?

Thanks

Re: How to use regex to filter out logs?

ITWhisperer — Tue, 06 Jul 2021 16:17:33 GMT

I can't anything wrong with what you have posted. Which version of splunk are you using?

Re: How to use regex to filter out logs?

martaBenedetti — Wed, 07 Jul 2021 07:25:15 GMT

@ITWhisperer on HFW there is Splunk Enterprise 7.1.3.

Thought you were thinking about something 🙂

Thanks anyway!!