Matching regular expressions in lookup table against field in index

[madcow]

I have a lookup table containing a list of regular expressions, and am trying see if there are matches against a field in one of my index. 

I can't figure how to do it as it is not a direct comparison of values. 

Appreciate any help on this.

[yuanliu]

Take a look at psuedo code in Can I save mvexpand when matching a multivalue lookup? I use regex in one of my lookups and the manipulation is crazy, so much so I named the intermediate field crazystring to this day. (Perhaps carefully read the entire discussion.)  I have practical considerations to want to use regex for this purpose.  Think carefully if that is really necessary. (Basically you are using lookup to store code.  This is not really how Splunk is designed.)

With JSON functions introduced in Splunk 8.1, today this problem can be solved with more semantic expressions.  But the method will be the same.

[bowesmana]

You can do this - here's a simple example where the lookup regexes.csv contains two rows, the first with an IPv6 regex and the second with an IPv4, i.e. made with this SPL

| makeresults 
| fields - _time
| eval regex=split("([A-Za-z0-9]{1,4}:){7}##([0-9]{1,3}\.){3}[0-9]{1,3}", "##")
| mvexpand regex
| fields regex
| outputlookup regexes.csv

This SPL then creates 3 ip values and matches them against the regexes

| makeresults 
| fields - _time
| eval ipv4="bla 10.1.2.3 bla"
| eval ipv6="bla 2021:1431:aaaa:bbbb:cccc:dddd:1234:0 bla"
| eval ipvbad="bla not an ip address bla"
``` Show how the regexes are evaluated ```
| eval regexes=[ | inputlookup regexes.csv | stats values(regex) as regex | eval regex="\"(".mvjoin(regex, "|").")\"" | return $regex ]

| foreach ipv* [ | eval ipv<<MATCHSTR>>_match_direct_from_lookup=if(match(<<FIELD>>, [ | inputlookup regexes.csv | stats values(regex) as regex | eval regex="\"(".mvjoin(regex, "|").")\"" | return $regex ]), 1, 0), 
                        ipv<<MATCHSTR>>_match_from_field=if(match(<<FIELD>>, regexes), 1, 0)
]
| transpose 0

You can see the ipv4 and 6 match but the bad one does not.

[madcow]

| inputlookup regexes.csv | stats values(regex) as regex | eval regex="\"(".mvjoin(regex, "|").")\"" 

Apologies if I am misinterpreting, the above portion combines my regular expressions into a single value? 

I tried a to do a "where match(field_value, regex)", but gotten a regular expression is too large error 

[bowesmana]

That mvjoin simply turns your list of regexes into 

(A|B|C|D|E|F|...)

i.e. A OR B OR C OR D...

How many regexes do you have - I suspect there is a practical limit and you've probably reached it.