Splunk Search

How do I match a regex query in a CSV?

JohnGilmour
New Member

Hello,

I have a CSV file full of regex queries. What I am looking at doing is matching those with a regex in the CSV.

Ideas?

0 Karma
1 Solution

dmarling
Builder

This should provide better performance depending on the size of the lookup file:

| where
    [| inputlookup regexlookup.csv
    | eval matcher="match(subject,\"".regex."\")"
    | stats values(matcher) as search
    | eval search=mvjoin(search. " OR ")]

You can append that to any search and it will add a where statement that is populated by a serious of matches from the regex in your lookup file. If it's a massive lookup file this may cause some issues, but if it's not too big it should work alright. You'll need to substitute the name of the lookup file where I have regexlookup.csv and the name of the field you want to match on where I have subject.

Edit: I forgot to mention that you'll also need to make sure to update the regex field in the match statement to match the field name of your regex in the lookup file too.

If this comment/answer was helpful, please up vote it. Thank you.

View solution in original post

0 Karma

woodcock
Esteemed Legend

I believe that you are looking for this, right?

https://answers.splunk.com/answers/386488/regex-in-lookuptable.html

0 Karma

dmarling
Builder

This should provide better performance depending on the size of the lookup file:

| where
    [| inputlookup regexlookup.csv
    | eval matcher="match(subject,\"".regex."\")"
    | stats values(matcher) as search
    | eval search=mvjoin(search. " OR ")]

You can append that to any search and it will add a where statement that is populated by a serious of matches from the regex in your lookup file. If it's a massive lookup file this may cause some issues, but if it's not too big it should work alright. You'll need to substitute the name of the lookup file where I have regexlookup.csv and the name of the field you want to match on where I have subject.

Edit: I forgot to mention that you'll also need to make sure to update the regex field in the match statement to match the field name of your regex in the lookup file too.

If this comment/answer was helpful, please up vote it. Thank you.

View solution in original post

0 Karma

dmarling
Builder

@JohnGilmour if this answer or the other one assisted you, please make sure to accept either of them as the accepted answer. Thank you!

If this comment/answer was helpful, please up vote it. Thank you.
0 Karma

richgalloway
SplunkTrust
SplunkTrust

The performance will be awful, but you could probably do it with map.

| inputlookup regex.csv | map search="index=foo SUBJECT=* | where match(SUBJECT, "$regex$")"
---
If this reply helps you, an upvote would be appreciated.
0 Karma

dmarling
Builder

Can you provide a bit more information such as an example of the data in the csv and how you are attempting to use it in a dashboard/query?

If this comment/answer was helpful, please up vote it. Thank you.
0 Karma

JohnGilmour
New Member

Hello - I'm looking at a dashboard for Spam, with the email SUBJECT (Field) matching a regex in a file.

AKA if the subject matches one of the regex rules in the file.

E.G
e-?ma[il1]+ .{0,10}suspen
e-?ma[il1]{1,} user
from.helpdesk
fu[il1]{2,}.*ma[il1]+[ -]?box

Thanks,

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!