topic Re: Lookup issues in Splunk Search

Lookup issues

hartfoml — Mon, 28 Nov 2011 21:01:11 GMT

I have a table of bad IP's that I want to use in a search agnest my firewall logs

in the past I have done this low tech search

sourcetype="cisco_syslog" ("x.x.x.x" OR "y.y.y.y" OR z.z.z.z) # this sometimes takes a long time #

this search would find all the firewall logs that have any of the bad IP's. As the list became longer I wanted to use the list from SANS.org. I can automaticly generate the list OK, but how do I use the list in a search?

Re: Lookup issues

Ayn — Mon, 28 Nov 2011 21:22:00 GMT

Create a lookup file and put it for instance in an appropriate directory (for instance $SPLUNK_HOME/etc/system/lookups), then search for IP numbers found in it using a subsearch. Let's say you call the lookup sanslist.csv and use the field name "ip":

sourcetype="cisco_syslog" [| inputlookup sanslist.csv | rename ip AS query | fields query]

Some information on the reason for renaming the "ip" field to "query": a subsearch works much like backticks in many *NIX shells, in that it executes first of all and then returns its results to the outer search, which uses this output. Normally if you have a subsearch with "| fields foo" at the end, the subsearch will return something like this:

((foo="val1") OR (foo="val2") OR (foo="val3") ... OR foo="val42"))

query is a special field that causes the subsearch to return pure free-text searches rather than searching for values in a particular field. So if foo were to be renamed to query, the subsearch would instead return something like this:

("val1" OR "val2" OR "val3" ... OR "val42")

In your case you want to search for the IP numbers as free-text searches, so that's why the renaming is needed.

Re: Lookup issues

hartfoml — Tue, 29 Nov 2011 15:22:11 GMT

I think I'm missing something "the field name"

The sanslist.csv is only a list of IP's there is no field name in the one column table. Who do I get the field name in the table or lookup.

I am doing a rex piped to a table command to get the IP from a scripted downloaded file and then pipe the table to "outputlookup sanslist.csv" How do I get the field name into this?

Re: Lookup issues

Ayn — Tue, 29 Nov 2011 15:55:14 GMT

If you are passing values to outputlookup you are feeding it with some field value as well. Have a look at this question: http://splunk-base.splunk.com/answers/5521/specify-fields-for-outputlookup-or-outputcsv

Or check yourself in the resulting .csv file. The fieldname is in the headers on the first line of the CSV file.