Splunk Search

Using lookup table matches to limit Web datamodel results

adamblock2
Path Finder

I have created a lookup table which contains iocs, a subset of which are IPv4 addresses. I am trying to use events from the Web datamodel to alert on matches in the lookup table. The following is part of a search which I created that does return results. Unfortunately, it loads full number of datamodel events before attempting to match records in the lookup.

| tstats  summariesonly=true   earliest(_time) AS firstEvent latest(_time) AS lastEvent values(Web.action) AS action  values(Web.http_method) AS http_method values(Web.http_user_agent) AS http_user_agent count from datamodel=Web where nodename=Web.Proxy by Web.app Web.src Web.dest
| rename Web.* AS *
| lookup ioc_entries_lookup ioc_string AS src
| table firstEvent lastEvent action src app http_method http_user_agent dest url
| convert ctime(*Event) timeformat="%m/%d/%Y %H:%M:%S"

I am looking for a way to include the lookup as one of the "where" clauses in the tstats command. I tried using inputlookup as a kind of subsearch. Being that there are close to 200,000 records in the lookup, it failed.

| tstats  summariesonly=true   earliest(_time) AS firstEvent latest(_time) AS lastEvent values(Web.action) AS action  values(Web.http_method) AS http_method values(Web.http_user_agent) AS http_user_agent count from datamodel=Web where [|inputlookup ioc_entries_lookup 
    | fields ioc_string 
    | rename ioc_string AS Web.src
    | table Web.src]

Is there another way that this can be done? Is there a way that this can be done using the "lookup" command?

Thank you.

Tags (1)
0 Karma

starcher
Influencer

As you said too big. just use the lookup as a lookup after your stats and use a ... | where isnotnull()

| tstats  summariesonly=true   earliest(_time) AS firstEvent latest(_time) AS lastEvent values(Web.action) AS action  values(Web.http_method) AS http_method values(Web.http_user_agent) AS http_user_agent count from datamodel=Web where nodename=Web.Proxy by Web.app Web.src Web.dest
 | rename Web.* AS *
 | lookup ioc_entries_lookup ioc_string AS src OUTPUTNEW ioc_string as isFound
 | where isnotnull(isFound)
0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...