topic Re: Why is my rex statement unable to extract the field? in Splunk Search

Why is my rex statement unable to extract the field?

jsiker — Tue, 29 Sep 2020 07:11:40 GMT

I have this rex with an assigned field:

regex _raw="(?<total_GC_time>0?.\d+)"

I'm searching lines like this:

2015-09-10T16:46:31.320-0400: 861067.833: Total time for which application threads were stopped: 0.0010090 seconds

and I'm trying to capture the 0.0... for all lines.

The events come up fine, but when I try to table total_GC_time, all the fields are empty. Is it a problem with the regex _raw call?

Re: Why is my rex statement unable to extract the field?

MuS — Thu, 10 Sep 2015 21:58:08 GMT

Hi jsiker,

try this regex:

regex _raw="(?<total_GC_time>0\.\d+)"

this will capture only the seconds after .. threads were stopped:

Hope this helps ...

cheers, MuS

Re: Why is my rex statement unable to extract the field?

MuS — Thu, 10 Sep 2015 22:03:36 GMT

btw, you can test your regex with Splunk directly like this:

$SPLUNK_HOME/bin/splunk cmd pcregextest mregex="(?<total_GC_time>0\.\d+)" test_str="2015-09-10T16:46:31.320-0400: 861067.833: Total time for which application threads were stopped: 0.0010090 seconds"

and the result will look like this:

Original Pattern: '(?<total_GC_time>0\.\d+)'
Expanded Pattern: '(?<total_GC_time>0\.\d+)'
Regex compiled successfully. Capture group count = 1. Named capturing groups = 1.
SUCCESS - match against: '2015-09-10T16:46:31.320-0400: 861067.833: Total time for which application threads were stopped: 0.0010090 seconds'

#### Capturing group data ##### 
Group |            Name | Value
--------------------------------------
    1 |   total_GC_time | 0.0010090

Re: Why is my rex statement unable to extract the field?

somesoni2 — Thu, 10 Sep 2015 22:03:51 GMT

Try something like this

your base search | rex "(?<total_GC_time>[0-9\.]+)\s*seconds$"

Re: Why is my rex statement unable to extract the field?

jsiker — Thu, 10 Sep 2015 22:44:08 GMT

awesome! thanks, both work. i realize now i hadn't had a pipe b/w the rex and the rest of my search. great tip for the testing too, didn't know you could do that!

Re: Why is my rex statement unable to extract the field?

jsiker — Thu, 10 Sep 2015 23:09:36 GMT

do i do this in the normal search box? i've been unable to get this to work.

Re: Why is my rex statement unable to extract the field?

MuS — Thu, 10 Sep 2015 23:11:47 GMT

Re: Why is my rex statement unable to extract the field?

jsiker — Fri, 11 Sep 2015 19:00:31 GMT

haha, if i had access to our Splunk server, life would be grand. sadly i don't. 😞

Re: Why is my rex statement unable to extract the field?

somesoni2 — Fri, 11 Sep 2015 19:16:09 GMT

Similar, but not as good as @MuS's testing method, specially if you don't have server access

| gentimes start=-1 | eval _raw="2015-09-10T16:46:31.320-0400: 861067.833: Total time for which application threads were stopped: 0.0010090 seconds" | rex "(?<total_GC_time>[0-9\.]+)\s*seconds$"

This can run in any splunk instance, and I use this for testing my regex.

Re: Why is my rex statement unable to extract the field?

MuS — Fri, 11 Sep 2015 19:41:06 GMT

@jsiker here comes Web-cli App https://splunkbase.splunk.com/app/1607/ to the rescue 🙂

Re: Why is my rex statement unable to extract the field?

Runals — Sat, 12 Sep 2015 11:26:52 GMT

I always just use https://regex101.com/

Depending on the complexity and variability in the logs I'm trying to extract fields from I might do something like this to get at the data

sourcetype = foo | dedup punct | head 10 | table _raw