Splunk Search

How to update certain time fields of lookup table without overwriting old table entries?

JakeInfoSec
Explorer

So, I put together a search not too long ago, with help from the community on here, that would run hourly to update a lookup table I have running. What this table holds is a list of suspicious IPs that have a field saying last date seen. I previously had a search that was doing exactly what I wanted where it would update that field with the most recent date seen. But, for some reason, that is no longer working, and I can't seem to figure out why. Instead of updating the latest date, it is showing a date from almost a month ago despite it still running and being seen as recent as today. Here is the layout of the search I had used last time.

 sourcetype=blah [| inputlookup suspect_list=csv | table Susp_IP | rename Susp_IP as src_ip ]
 | search Ticket_num=* | rename src_ip as Susp_IP | eval date_last_seen=_time  | table Susp_IP, Ticket_num, date_last_seen 
 |inputlookup append=t suspect_list.csv
 | dedup Susp_IP
 | outputlookup suspect_list.csv

Essentially, it is supposed to be inputting the lookup, searching on those IP's and updating the date last seen field, and then inputting the lookup again so that it will still keep old entries in the event. Those IPs haven't shown up in the last hour and not be removed. Then, it combines them and outputs it back to that same lookup.

thambisetty
SplunkTrust
SplunkTrust
  sourcetype=blah [| inputlookup suspect_list=csv | table Susp_IP | rename Susp_IP as src_ip ]
  | search Ticket_num=* | rename src_ip as Susp_IP | eval date_last_seen=_time  | table Susp_IP, Ticket_num, date_last_seen 
  |inputlookup append=t suspect_list.csv
  | stats max(date_last_seen) as date_last_seen by Susp_IP
  | outputlookup suspect_list.csv
————————————
If this helps, give a like below.
0 Karma

JakeInfoSec
Explorer

This still seems to only have the older dates. For instance there is an IP that is showing up in logs right now, but the date being shown for it in this search is still from over a month ago.

0 Karma

thambisetty
SplunkTrust
SplunkTrust

Are you getting latest results in your search?

————————————
If this helps, give a like below.
0 Karma

thambisetty
SplunkTrust
SplunkTrust

Give me date_last_seen value from your search results and lookup also.

I assume that you have date_last _seen value from results in the format of unix epoch since you have copied _time to date_last_seen.

What about date_last_seen value format from lookup?

————————————
If this helps, give a like below.
0 Karma

JakeInfoSec
Explorer

so now that I have the following search:

  sourcetype=blah [| inputlookup suspect_list=csv | table Susp_IP | rename Susp_IP as src_ip ]
  | rename src_ip as Susp_IP | eval date_last_seen=_time  | table Susp_IP, Ticket_num, date_last_seen 
  |inputlookup append=t suspect_list.csv
  | outputlookup suspect_list.csv

I have a table showing the whole lookup but also table entries that include the IP and date_last_seen with the date_last_seen field being the most updated(today's date). So now I just need to figure out how to dedup those so that the table has one row per IP and that the date_last_seen is the most recent date, whether it be today's date since it is showing up in events or it could be 2 days ago if that is the most recent time we have seen that IP.

0 Karma

thambisetty
SplunkTrust
SplunkTrust

here the problem is with date which you are collecting from logs. I think thats not being parse correctly. if it was parsed correctly, date_last_seen value would be in unix epoch time format.

can you post the result of the below query: I need to see time format of _time and date_last_seen
The below query will return only one result and you can put some garbage values other than _time and date_last_seen and post result here.
sourcetype=blah [| inputlookup suspect_list=csv | table Susp_IP | rename Susp_IP as src_ip ]
| rename src_ip as Susp_IP | eval date_last_seen=_time | table Susp_IP, Ticket_num, date_last_seen,_time | head 1

————————————
If this helps, give a like below.
0 Karma

JakeInfoSec
Explorer

so for one particular IP. The date_last_seen within the lookup table is "8/8/2018 9:24
". Now that I look at that search you sent me before, and the one I was working with previously, the only results showing up in the table seem to be the lookup table itself. I believe that is the case because there are no actual events, rather just a table within the stats tab. Now if I get rid of everything after the first inputlookup so that I only have the following for a search:sourcetype=blah [| inputlookup suspect_list=csv | table Susp_IP | rename Susp_IP as src_ip ]
Then I get events that have the recent events for the IP's on the list. So it may be the search Ticket_num=* that is screwing me up here.

0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...