Splunk Search

Expire Rows in a Lookup After 30 Days

dmbr
Explorer

Hi Splunkheads, 

Need some advice here. I have built a simple lookup table and simple search for known bad ip addresses. My search runs across the lookup table, and returns a table for any matches across the environment. 

Here is my search:

 

| tstats summariesonly=t fillnull_value="MISSING" count from datamodel=Network_Traffic.All_Traffic by All_Traffic.src, All_Traffic.dest, All_Traffic.dest_port, _time, All_Traffic.action, All_Traffic.bytes, index, sourcetype
| lookup ioc_ip.csv ioc_ip as All_Traffic.src OUTPUT ioc_ip as src_found
| lookup ioc_ip.csv ioc_ip as All_Traffic.dest OUTPUT ioc_ip as dest_found
| where !isnull(src_found) OR !isnull(dest_found)
| fields - src_found, dest_found
| sort -_time

 

I have been asked to auto-expire rows in the lookup after 30 days. The logic would be something like:

If date ioc_email_date older than 30 days:
Delete row
else:
Run search
 
I have added dates to my lookup table. Here is a dummy example of my lookup table:
Splunk Screenshot 2021-05-21 134637.png
My questions:
1. Best format for the ioc_ip_date column? Would it be best to use Epoch time? Currently using this "2021-18-05" as per above. Happy to convert to any format.
2. Any suggestions on how to add the logic to the above search sample?
 
 
Labels (2)
Tags (4)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Since you are going to be doing some sort of calculation and comparison on the date, epoch time would be best, although you could store both just to make it easier to check.

| inputlookup ...
| where now - date < 30*60*60*24
| outputlookup ...

The lookup file will only get smaller with this process, so you need to add additional events to this process or have an additional process to keep the file current.

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...