topic Field extraction using regular expression under field extraction page in Splunk Enterprise

Field extraction using regular expression under field extraction page

sandeepduppalli — Wed, 15 Jul 2020 17:33:18 GMT

I need to write a common regex to match all the below patterns

My regular expression written so far is

(?P<timestamp>\w+\s+\d+\s+\d+:\d+:\d+)\s+(?P<src>\w+)\s+(?P<daemon>\w+):\s+(?P<message>(.*?)$)|(?J)(?P<timestamp>\w+\s+\d+\s+\d+:\d+:\d+)\s+(?P<src>\w+)\s+(?P<daemon>\w+)\[(?P<process_id>\d+)\]:\s+(?P<entry_type>\w+):\s+(?P<service_id>\w+)\s+\w+=((?P<status>0)|(?<pid>\d+))(\s+from=(?P<origin_ip>(.*?)$))

This matches 1st and 2nd pattern in regex101.com but when I put it in splunk it doesn't work matching unintended fields. Please help how to go with this

Jul 15 14:01:32 jiufc1fe330 xinetd[82352]: START: nrpe pid=151239 from=::ffff:14.956.44.41
Jul 15 12:30:36 dyue29200 systemd: Removed slice User Slice of root.
Jul 15 12:30:21 dtg280419 xinetd[16211]: EXIT: nrpe status=0 pid=8924 duration=0(sec)

Re: Field extraction using regular expression under field extraction page

richgalloway — Wed, 15 Jul 2020 19:35:45 GMT

Is this for index-time or search-time field extraction?
Can you be more specific about how Splunk is failing? What unintended fields are matching?

Re: Field extraction using regular expression under field extraction page

sandeepduppalli — Thu, 16 Jul 2020 01:26:11 GMT

This is search time field extraction. I am using few captured named group for field names like timestamp,src etc.., twice with (?J) option but splunk recognizes them as different fields. How to use the same field name here?My basic aim to have a single regex for all log patterns mentioned.

Re: Field extraction using regular expression under field extraction page

richgalloway — Thu, 16 Jul 2020 13:45:45 GMT

Since this is a search-time extraction, consider using multiple rex commands. That's often easier than crafting a single regex for all cases.

Also, try putting the (?J) flag at the beginning of your regex.

Re: Field extraction using regular expression under field extraction page

sandeepduppalli — Thu, 16 Jul 2020 13:49:57 GMT

When using rex command with same named group it is giving the following error

Error in 'rex' command: Encountered the following error while compiling the regex '(?P<timestamp>\w+\s+\d+\s+\d+:\d+:\d+)\s+(?P<src>\w+)\s+(?P<daemon>\w+)\[(?P<process_id>\d+)\]:\s+pid=(?P<process_id>\d+)': Regex: two named subpatterns have the same name (PCRE2_DUPNAMES not set).

Re: Field extraction using regular expression under field extraction page

richgalloway — Thu, 16 Jul 2020 14:28:07 GMT

That's what the (?J) flag is for. Why did you leave it out?