Splunk Search

regular expression in my lookup table


hi,all,here is my problem:

here is my search:

mysearch | table fields1 fields2

and I got:

fields1 fields2

foofoo abcccd

barbar asdddf

the lookup table I define in lookups is as below,the keywords is regular expression which I want match the fields2

keyword fields3

abccc\w+ 10

asddd\w+ 20

what I want is

fields1 fields2 fields3

foofoo abcccd 10

barbar asdddf 20

so how can I get this done?

Super Champion

I just thought it may be worth pointing out that the mvrex command which is implemented by the SA-cim_validator app may be something worth taking a look at. While the command itself doesn't deal with lookups, values pulled back from lookups are send through this command on at least one of the dashboards:


Anyways, the combo of regex within lookups is pretty rare. Thought this may give some future readers some ideas to think about.

0 Karma

New Member

Hi all we have some trouble with this python script
Splunk error code
"returned error code 1"

Please Help

0 Karma


There is no regex support in static lookup tables unfortunately. You could achieve this by writing a dynamic lookup script that does this, the obvious drawback obviously being that it's a bit more hassle to roll up your sleeves and start coding.

I've written this kind of dynamic lookup for this exact purpose and have it lying around somewhere, but don't know where right now - let me know if you want it and I'll have another look.

EDIT: So, looked around and found it. DISCLAIMER, I'm by no means a real Python coder 🙂


# A dynamic lookup that takes CSV as input, performs a regex match against another CSV, then returns the CSV results                                                          
import csv
import sys
import re
import os
import glob

def inlookup(inf, inval, outf):
        # The app makes the assumption that a directory in the form <customer>_indexer_config exists. If multiple
        # directories matching this template exist for some weird reason, only the first one is used.
        config_app_path = os.path.join(os.environ['SPLUNK_HOME'],'etc','apps','yourapp')
        csvname = "yourlookup.csv"
        csvpath = os.path.join(config_app_path,'lookups',csvname)
    except Exception as e:
        sys.stderr.write("No %s file found." % csvname)

        c = open(csvpath, 'rb')
        f = csv.DictReader(c)

        for row in f:
            if re.search(row[inf], inval):
                return row[outf]

    except Exception as e:
        return []

def main():
    if len(sys.argv) != 3:
        print "Usage: %s <in field> <out field>" % (sys.argv[0])

    inf = sys.argv[1]
    outf = sys.argv[2]
    r = csv.DictReader(sys.stdin)
    w = csv.DictWriter(sys.stdout, r.fieldnames)

    for result in r:
        # If all fields are already present, there's no need
        # to look anything up
        if len(result[inf]) and len(result[outf]):

        elif len(result[inf]):
            outvalue = inlookup(inf, result[inf], outf)
            result[outf] = outvalue


As you can see in the start of the inlookup function you need to specify your path and lookup filename explicitly. As far as I know there's unfortunately no way of providing an argument for a lookup to consume it that way, so it needs to be hardcoded.



external_cmd = regexpython.py Id,Name
external_type = python
fields_list = Id,Name


LOOKUP-UniqueID_Lookup = UniqueID_Lookup Id AS Id OUTPUTNEW Name AS Name

0 Karma


I want the Name UserDefinedCategory should be displayed in the category...but this is not workin ?? am i missin something ??

0 Karma


Hi Ayn,

Can you pls give me the steps in exucting this ?

I have done the following , but this seems not working

  1. I have copied above in my app's bin directory i.e /opt/splunk/etc/apps/MY_APP/bin/regexpython.py with the filename regexpython.py
  2. I have my lookup file name lookup_UniqueId.csv , which has fields Id, Name

Id is the value that comes in the logs, and correspondingly it matches the Name that are present in the lookup file

  1. Now with ur code of regex . i have added this line in my lookup Id,Name ^2\d+6$,"UserDefinedCategory"

ie. if my Id is starting with 2 and ends with 6

0 Karma


thanks,it is very helpful!

0 Karma


Amended my answer with the code I found lying around... 😉

0 Karma


yeah,I really appreciate it if you could have another look,the problem I mentioned is a real case in my work and I stuck here.By the way I write some python script in my daily work,thanks in advance if you could provide the answer!

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!