Splunk Search

Using lookup table matches to limit Web datamodel results

adamblock2
Path Finder

I have created a lookup table which contains iocs, a subset of which are IPv4 addresses. I am trying to use events from the Web datamodel to alert on matches in the lookup table. The following is part of a search which I created that does return results. Unfortunately, it loads full number of datamodel events before attempting to match records in the lookup.

| tstats  summariesonly=true   earliest(_time) AS firstEvent latest(_time) AS lastEvent values(Web.action) AS action  values(Web.http_method) AS http_method values(Web.http_user_agent) AS http_user_agent count from datamodel=Web where nodename=Web.Proxy by Web.app Web.src Web.dest
| rename Web.* AS *
| lookup ioc_entries_lookup ioc_string AS src
| table firstEvent lastEvent action src app http_method http_user_agent dest url
| convert ctime(*Event) timeformat="%m/%d/%Y %H:%M:%S"

I am looking for a way to include the lookup as one of the "where" clauses in the tstats command. I tried using inputlookup as a kind of subsearch. Being that there are close to 200,000 records in the lookup, it failed.

| tstats  summariesonly=true   earliest(_time) AS firstEvent latest(_time) AS lastEvent values(Web.action) AS action  values(Web.http_method) AS http_method values(Web.http_user_agent) AS http_user_agent count from datamodel=Web where [|inputlookup ioc_entries_lookup 
    | fields ioc_string 
    | rename ioc_string AS Web.src
    | table Web.src]

Is there another way that this can be done? Is there a way that this can be done using the "lookup" command?

Thank you.

Tags (1)
0 Karma

starcher
Influencer

As you said too big. just use the lookup as a lookup after your stats and use a ... | where isnotnull()

| tstats  summariesonly=true   earliest(_time) AS firstEvent latest(_time) AS lastEvent values(Web.action) AS action  values(Web.http_method) AS http_method values(Web.http_user_agent) AS http_user_agent count from datamodel=Web where nodename=Web.Proxy by Web.app Web.src Web.dest
 | rename Web.* AS *
 | lookup ioc_entries_lookup ioc_string AS src OUTPUTNEW ioc_string as isFound
 | where isnotnull(isFound)
0 Karma
Get Updates on the Splunk Community!

New Year. New Skills. New Course Releases from Splunk Education

A new year often inspires reflection—and reinvention. Whether your goals include strengthening your security ...

Splunk and TLS: It doesn't have to be too hard

Overview Creating a TLS cert for Splunk usage is pretty much standard openssl.  To make life better, use an ...

Faster Insights with AI, Streamlined Cloud-Native Operations, and More New Lantern ...

Splunk Lantern is a Splunk customer success center that provides practical guidance from Splunk experts on key ...