topic How to write regex to extract a field's values and pass them to a new field using rex? in Splunk Search

How to write regex to extract a field's values and pass them to a new field using rex?

johntopley — Mon, 01 Sep 2014 07:40:35 GMT

How can I use the value from a field named geog in the regular expression passed to the rex command? In the example below, I'd like foo to be substitued by whatever value geog has.

rex field=_raw "foo:(?<area>[^&]*)"

Thanks in advance.

Re: How to write regex to extract a field's values and pass them to a new field using rex?

rsennett_splunk — Mon, 01 Sep 2014 08:31:00 GMT

By "it will only be one value" do you mean that it is static text? Because then you would just use the text...

Re: How to write regex to extract a field's values and pass them to a new field using rex?

johntopley — Mon, 01 Sep 2014 08:46:32 GMT

No, it's not static text. It's a field value. I've edited by question accordingly.

Re: How to write regex to extract a field's values and pass them to a new field using rex?

kristian_kolb — Tue, 02 Sep 2014 09:02:04 GMT

... | eval foo = geog |

gives the value of the field geog to the field foo.

Somehow, I think that this is not what you're after. Please provide some sample events, your search, and your desired outcome.

Re: How to write regex to extract a field's values and pass them to a new field using rex?

Runals — Tue, 02 Sep 2014 09:55:42 GMT

You are on the right track I think - just replace | rex field=_raw with | rex field=geog.

Re: How to write regex to extract a field's values and pass them to a new field using rex?

johntopley — Tue, 02 Sep 2014 10:03:05 GMT

No, geog is not the field I want rex to extract the information from. I just want to dynamically build up my rex regular expression to use whatever value the geog field has rather than hard-coding a value in the regex.

Re: How to write regex to extract a field's values and pass them to a new field using rex?

Runals — Tue, 02 Sep 2014 10:16:15 GMT

I'm confused - you want to use the value in the geog field but geog is not the field you want to extract the information from? Can you post an example? You can can have multiple capturing groups in a single rex command or have multiple rex commands. For example | rex field=geog "(?[^:]+):(?[^&]*)" | stats values(area) by foo

Re: How to write regex to extract a field's values and pass them to a new field using rex?

johntopley — Tue, 02 Sep 2014 11:21:28 GMT

It's a URL query string like this:

querystring=geog:2011WARDH&totals:false&dm/2011WARDH:E06000016,E12000004,E06000016&etc...

The geog field is extracted and returns 2011WARDH in this example. I want to extract E06000016,E12000004,E06000016 into a new area field. As you can see, they're prefixed with the 2011WARDH value from the geog field. This is not a fixed value, so I need it to vary within the regular expression as it varies within the geog field.

Re: How to write regex to extract a field's values and pass them to a new field using rex?

rsennett_splunk — Tue, 02 Sep 2014 14:58:52 GMT

Here is what you want:

rex "geog:([^\&]+)&([^\/]+)\\/\1:(?P<area>[^\&]+)"
(be sure to escape the forward slash...the markdown is not allowing that to show.)

The first capturing group grabs the value of geog and then later, you reference the first capturing group with the \1

this worked for me.

You can see exactly how it works if you put your event and the regex into something like regex101.com

Re: How to write regex to extract a field's values and pass them to a new field using rex?

johntopley — Wed, 03 Sep 2014 09:56:39 GMT

That did the trick - thanks!

Re: How to write regex to extract a field's values and pass them to a new field using rex?

rsennett_splunk — Wed, 03 Sep 2014 15:10:10 GMT

Awesome! Thank you for accepting the answer. Be sure to vote it up as well so that it is more likely to bubble to the top when other folks are looking for something similar.