topic Re: Field Extraction : regex works fine with search using "rex" command but not with Field extraction in Splunk Search

Field Extraction : regex works fine with search using "rex" command but not with Field extraction

Flo-Paris — Wed, 21 Apr 2021 13:03:42 GMT

Hello,

I'm trying to analyze WatchGuard firewall logs received by Splunk using syslog on udp 514 port.

I was able to find a well working regex to use in a search using the following rex command in order to extract needed fields :

*
| rex field=_raw ".*\s(?<HOSTNAME>\S+)\s(?<PROCESS>\S+):\s.*\s(?<DISPOSITION>(Allow|Deny))\s(?<SRC_INT>\S+)\s(?<DST_INT>\S+)\s.*(?<PR>(icmp|igmp|tcp|udp)).*\s(?<SRC_IP>[[octet]](?:\.[[octet]]){3})\s(?<DST_IP>[[octet]](?:\.[[octet]]){3})\s(?<SRC_PORT>\d{1,5})\s(?<DST_PORT>\d{1,5})\s.*$(?P<RULE_NAME>.*)?(-00)$$"
| table HOSTNAME,PROCESS,DISPOSITION,SRC_INT,DST_INT,PR,SRC_IP,DST_IP,SRC_PORT,DST_PORT,RULE_NAME

Result is a table as we can see in attachment.

Now, in order to optimize all of that, i would like to be able to extract all these fields automatically without having the need to use a rex command in each search i run...

i tryed using the Splunk Field extraction wizard, both using the automatic regex generator and by copy paste my search regex, but no success...

i suppose i missed something somewhere ?

thanks for your help

Florent

Re: Field Extraction : regex works fine with search using "rex" command but not with Field extraction

Flo-Paris — Wed, 21 Apr 2021 13:06:16 GMT

Exemple of original log received :

Apr 21 15:04:33 10.40.1.254 Apr 21 15:04:33 FRPARXXX0001.mydomain.local firewall: msg_id="3000-0151" Allow Firebox EXT-FIBER-XXX-100 udp 1XX.XXX.XXX.1 1.XXX.XXX.10 39010 53 dst_user="administrator@mydomain.local" duration="32" sent_bytes="68" rcvd_bytes="128" (Any From Firebox-00)

Re: Field Extraction : regex works fine with search using "rex" command but not with Field extraction

Flo-Paris — Wed, 21 Apr 2021 13:08:19 GMT

Only "RULE_NAME" seems to be correctly extracted by default (see attachment), i don't know why...

Re: Field Extraction : regex works fine with search using "rex" command but not with Field extraction

Flo-Paris — Wed, 21 Apr 2021 13:15:01 GMT

Even if Splunk field extraction wizard seems to match my fields already...

Re: Field Extraction : regex works fine with search using "rex" command but not with Field extraction

Flo-Paris — Wed, 21 Apr 2021 13:17:46 GMT

Here are my existing Field Extractions in the Splunk Settings / Fields / Field Extractions menu