Eval results from two time ranges

[marshalll3302]

Splunk sirs, 

I am trying to add a boolean column to my data called 'new_IP_detected' which will tell me whether an answer IP is new compared to answer IPs from a previous time range. Both searches are from the same index and sourcetype, and I only want to compare whether or not an answer IP from -24h to now is in the list of answer IPs from -30d to -24h. My search so far:

index=[sample index] sourcetype=[sample sourcetype] earliest=-24h latest=now
NOT
[ search index=[sample index] sourcetype=[sample sourcetype] earliest=-30d latest=-24h
| stats count by answer
| table answer]
| stats count by answer
| table answer

As of right now I am getting no results which I believe is expected (meaning there are no new IPs in the last 24 hrs). How would I add 'new_IP_detected' column over the last 30 days?

[PickleRick · Accepted solution]

This is one of the approaches. Another one would be to list all data and categorize it, then summarize and pick only matching ones.

So in your case you probably can do something like

<your_search> earliest=-30d

to list all events and

| eval state=if(_time<now()-86400,"old","new")

to categorize it. But this approach will work only because you have a single "type of search" and only the time differs so the events are easily distinguishable. In more complicated case you can use another approach:

<your search> earliest=-30d latest=-24h | eval state="old"
| append
     [ <your search> earliest=-24h | eval state="new" ]

Of course this one has limitations from the append command so you might use multisearch instead.

Anyway.

As you now have your search results, you can stats them

| stats values(state) by answer

so you know whether each answer is included in the old or new set. Now all that's left is to filter the result to only see those you want. For example if you want only those that are in the "new" period, but not in the "old" one you simply do

| where state="new" AND NOT state="old"

One caveat - matching multivalued fields can be a bit unintuitive since a condition is matched on each value from the mvfiled separately so

| where state="new" AND state!="old"

is a completely different condition (and I'll leave it as an exercise for the reader to find out what it matches).

[marshalll3302]

Great solution - I like how it also takes out need for subsearch 🙂 Thank you!!

[ITWhisperer]

It looks like if you get any results in answer, they will be new - you could test this by shortening your subsearch to earliest=-25h latest=-24h which should show new addresses if they occur in the last 24h but not in the hour before that

[marshalll3302]

Great way to sanity check - didn't think of this til you mentioned it. Ty!!