Splunk Search

Expire Rows in a Lookup After 30 Days


Hi Splunkheads, 

Need some advice here. I have built a simple lookup table and simple search for known bad ip addresses. My search runs across the lookup table, and returns a table for any matches across the environment. 

Here is my search:


| tstats summariesonly=t fillnull_value="MISSING" count from datamodel=Network_Traffic.All_Traffic by All_Traffic.src, All_Traffic.dest, All_Traffic.dest_port, _time, All_Traffic.action, All_Traffic.bytes, index, sourcetype
| lookup ioc_ip.csv ioc_ip as All_Traffic.src OUTPUT ioc_ip as src_found
| lookup ioc_ip.csv ioc_ip as All_Traffic.dest OUTPUT ioc_ip as dest_found
| where !isnull(src_found) OR !isnull(dest_found)
| fields - src_found, dest_found
| sort -_time


I have been asked to auto-expire rows in the lookup after 30 days. The logic would be something like:

If date ioc_email_date older than 30 days:
Delete row
Run search
I have added dates to my lookup table. Here is a dummy example of my lookup table:
Splunk Screenshot 2021-05-21 134637.png
My questions:
1. Best format for the ioc_ip_date column? Would it be best to use Epoch time? Currently using this "2021-18-05" as per above. Happy to convert to any format.
2. Any suggestions on how to add the logic to the above search sample?
Labels (2)
Tags (4)
0 Karma


Since you are going to be doing some sort of calculation and comparison on the date, epoch time would be best, although you could store both just to make it easier to check.

| inputlookup ...
| where now - date < 30*60*60*24
| outputlookup ...

The lookup file will only get smaller with this process, so you need to add additional events to this process or have an additional process to keep the file current.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...