topic Re: Regex from variable in Splunk Search

Regex from variable

bobweinerjr — Tue, 02 Jul 2019 10:20:52 GMT

I would like to store a regex pattern in a variable and use it to extract data. I've seen lots of similar questions but haven't been able to figure this out.

I can do the following

| makeresults count=1 | eval val=4 | rex field=val "(?<dig>\d)"

but I cannot

| makeresults count=1 | eval val=4 | eval ptn="(?<dig>\d)" | rex field=val ptn

Ultimately, I would have regex patterns stored in a CSV file and use lookup to get the correct pattern for a given query. It seems the above would a minimal implementation of this strategy.

Re: Regex from variable

jkat54 — Tue, 02 Jul 2019 12:18:47 GMT

You can use map

| makeresults count=1 
| eval val=4 
| eval ptn="(?<dig>\d)" 
| map [ search | rex field=val $ptn$]

Re: Regex from variable

bobweinerjr — Tue, 02 Jul 2019 12:41:16 GMT

Didn't know about map. That seems useful.

This search did not work for me, though.

Re: Regex from variable

oscar84x — Tue, 02 Jul 2019 13:24:24 GMT

You could use a transforms.conf stanza with the extract command to accomplish this.
Transforms would be your storage for your regex pattern and then you'd invoke it with extract during your search, or you can apply it automatically in props.conf

https://docs.splunk.com/Documentation/Splunk/6.5.2/SearchReference/Extract

Re: Regex from variable

cpetterborg — Tue, 02 Jul 2019 13:46:17 GMT

It would actually be:

| makeresults count=1 
| eval val=4 
| eval ptn="(?<dig>\d)"
| map search="| rex field=val $ptn$"

Except that the search results don't go into the map command for val in that way, and you can't send the val value into the search like this:

| makeresults count=1 
| eval val=4 
| eval ptn="(?<dig>\d)"
| map search="| rex field=$val$ $ptn$"

because the val value isn't a field name. So you are stuck between a rock and a hard place. The rex command requires a quoted string for the regex that it will use, not a field. I don't know of a way that you can do what you are wanting to do.

Re: Regex from variable

woodcock — Tue, 02 Jul 2019 14:09:27 GMT

This is probably more what you are looking for:

https://answers.splunk.com/answers/386488/regex-in-lookuptable.html

Re: Regex from variable

jkat54 — Tue, 02 Jul 2019 17:01:25 GMT

I always mess up the syntax of map... apologies

Re: Regex from variable

bobweinerjr — Tue, 02 Jul 2019 18:13:41 GMT

quite alright. I appreciate the input and will learn from it anyway.
unfortunately, we had a power outage on campus this morning and Splunk is not the first thing restored so it won't be today 😞

Re: Regex from variable

woodcock — Tue, 02 Jul 2019 19:38:29 GMT

Also note that both match() and replace() will pull RegEx from inside of a field name.

Re: Regex from variable

jkat54 — Tue, 02 Jul 2019 20:05:04 GMT

Here's what I meant to post:

 | makeresults count=1 
 | eval val=4 
 | eval ptn="(?<dig>\d)" 
 | map search="search index=yourindex | rex field=val $ptn$"

OR:

 | inputlookup yourlookup.csv 
 | map search="search index=yourindex | rex field=val $regexFieldInLookup$"

Re: Regex from variable

bobweinerjr — Thu, 04 Jul 2019 20:07:25 GMT

That's great. Going to try that out.

Re: Regex from variable

woodcock — Fri, 05 Jul 2019 13:46:29 GMT

Be sure to UpVote over there and come back here to Accept an answer if it works out.