Splunk Search

Eval results from two time ranges

marshalll3302
Explorer

Splunk sirs, 

I am trying to add a boolean column to my data called 'new_IP_detected' which will tell me whether an answer IP is new compared to answer IPs from a previous time range. Both searches are from the same index and sourcetype, and I only want to compare whether or not an answer IP from -24h to now is in the list of answer IPs from -30d to -24h. My search so far:

index=[sample index] sourcetype=[sample sourcetype] earliest=-24h latest=now
NOT
[ search index=[sample index] sourcetype=[sample sourcetype] earliest=-30d latest=-24h
| stats count by answer
| table answer]
| stats count by answer
| table answer

As of right now I am getting no results which I believe is expected (meaning there are no new IPs in the last 24 hrs). How would I add 'new_IP_detected' column over the last 30 days?

Labels (2)
0 Karma
1 Solution

PickleRick
SplunkTrust
SplunkTrust

This is one of the approaches. Another one would be to list all data and categorize it, then summarize and pick only matching ones.

So in your case you probably can do something like

<your_search> earliest=-30d

to list all events and

| eval state=if(_time<now()-86400,"old","new")

to categorize it. But this approach will work only because you have a single "type of search" and only the time differs so the events are easily distinguishable. In more complicated case you can use another approach:

<your search> earliest=-30d latest=-24h | eval state="old"
| append
     [ <your search> earliest=-24h | eval state="new" ]

Of course this one has limitations from the append command so you might use multisearch instead.

Anyway.

As you now have your search results, you can stats them

| stats values(state) by answer

so you know whether each answer is included in the old or new set. Now all that's left is to filter the result to only see those you want. For example if you want only those that are in the "new" period, but not in the "old" one you simply do

| where state="new" AND NOT state="old"

One caveat - matching multivalued fields can be a bit unintuitive since a condition is matched on each value from the mvfiled separately so

| where state="new" AND state!="old"

is a completely different condition (and I'll leave it as an exercise for the reader to find out what it matches).

 

View solution in original post

0 Karma

PickleRick
SplunkTrust
SplunkTrust

This is one of the approaches. Another one would be to list all data and categorize it, then summarize and pick only matching ones.

So in your case you probably can do something like

<your_search> earliest=-30d

to list all events and

| eval state=if(_time<now()-86400,"old","new")

to categorize it. But this approach will work only because you have a single "type of search" and only the time differs so the events are easily distinguishable. In more complicated case you can use another approach:

<your search> earliest=-30d latest=-24h | eval state="old"
| append
     [ <your search> earliest=-24h | eval state="new" ]

Of course this one has limitations from the append command so you might use multisearch instead.

Anyway.

As you now have your search results, you can stats them

| stats values(state) by answer

so you know whether each answer is included in the old or new set. Now all that's left is to filter the result to only see those you want. For example if you want only those that are in the "new" period, but not in the "old" one you simply do

| where state="new" AND NOT state="old"

One caveat - matching multivalued fields can be a bit unintuitive since a condition is matched on each value from the mvfiled separately so

| where state="new" AND state!="old"

is a completely different condition (and I'll leave it as an exercise for the reader to find out what it matches).

 

0 Karma

marshalll3302
Explorer

Great solution - I like how it also takes out need for subsearch 🙂 Thank you!!

0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

It looks like if you get any results in answer, they will be new - you could test this by shortening your subsearch to earliest=-25h latest=-24h which should show new addresses if they occur in the last 24h but not in the hour before that

0 Karma

marshalll3302
Explorer

Great way to sanity check - didn't think of this til you mentioned it. Ty!!

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...