Splunk Search

Compare values of most recent event to event before -- Show only the difference?

dmbr
Explorer
How do I compare the values of the most recent event to the event before that and show only the difference?

In one example, I am looking at o365 management activity with multivalue fields.
I want to see the difference and know when a domain has been added to an inbound Spam Policy.

Here is my base search:

index=idm_o365 sourcetype=o365:management:activity Workload="Exchange" Operation="Set-HostedContentFilterPolicy"
| eval a=mvfind('Parameters{}.Name', "AllowedSenderDomains"), AllowedSenderDomains=mvindex('Parameters{}.Value', a)
| table _time user_email ObjectId AllowedSenderDomains | sort - _time

The last two events will be this:
2022-08-15 00:00:00 user@example.com SpamPolicyName A.com;B.com;C.com
2022-08-10 00:00:00 user@example.com SpamPolicyName A.com;B.com

I would like to compare these two events and only show the difference, i.e. that "C.com" was added:
2022-08-15 00:00:00 user@example.com SpamPolicyName C.com
Tags (4)
0 Karma
1 Solution

bowesmana
SplunkTrust
SplunkTrust

Using streamstats and mvmap.

| makeresults 
| eval data=split("2022-08-15 00:00:00 user@example.com SpamPolicyName A.com;B.com;C.com,2022-08-10 00:00:00 user@example.com SpamPolicyName A.com;B.com", ",")
| mvexpand data
| rex field=data "(?<t>.{19}) (?<email>[^ ]*) (?<domains>.*)"
| eval _time=strptime(t, "%F %T")
| table _time email domains
``` Now here is what you need ```
``` Sort thenm in ascending time order ```
| sort _time
| eval domains=split(domains, ";")
``` Now copy previous date to later date ```
| streamstats window=1 current=f values(domains) as prev_domains
``` and remove 'first' event with no previous ```
| where isnotnull(prev_domains)
``` Now find additions ```
| eval additions=mvmap(domains, if(isnull(mvfind(prev_domains, domains)), domains, null()))
``` this will also find removals ```
| eval removals=mvmap(prev_domains, if(isnull(mvfind(domains, prev_domains)), prev_domains, null()))

First part sets up your example. Note that this also finds any 'removals'. i.e. where latest data does NOT have domains from earlier date.

 

View solution in original post

bowesmana
SplunkTrust
SplunkTrust

Using streamstats and mvmap.

| makeresults 
| eval data=split("2022-08-15 00:00:00 user@example.com SpamPolicyName A.com;B.com;C.com,2022-08-10 00:00:00 user@example.com SpamPolicyName A.com;B.com", ",")
| mvexpand data
| rex field=data "(?<t>.{19}) (?<email>[^ ]*) (?<domains>.*)"
| eval _time=strptime(t, "%F %T")
| table _time email domains
``` Now here is what you need ```
``` Sort thenm in ascending time order ```
| sort _time
| eval domains=split(domains, ";")
``` Now copy previous date to later date ```
| streamstats window=1 current=f values(domains) as prev_domains
``` and remove 'first' event with no previous ```
| where isnotnull(prev_domains)
``` Now find additions ```
| eval additions=mvmap(domains, if(isnull(mvfind(prev_domains, domains)), domains, null()))
``` this will also find removals ```
| eval removals=mvmap(prev_domains, if(isnull(mvfind(domains, prev_domains)), prev_domains, null()))

First part sets up your example. Note that this also finds any 'removals'. i.e. where latest data does NOT have domains from earlier date.

 

Get Updates on the Splunk Community!

Splunk Observability Cloud's AI Assistant in Action Series: Auditing Compliance and ...

This is the third post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how to ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...