topic Display only differences in values, between 2 events in Splunk Search

Display only differences in values, between 2 events

chengka — Tue, 29 Sep 2020 13:02:55 GMT

Hello,
I'm looking events that track changes to a configuration. The first event is the "before" state the newest event is the "after" state. There events are in json format and there are > 80 fields. I have a search that will display all of the values for each field present, but it's not practical to display a table with 80 columns and expect the user to locate the columns with multiple values.

Here are 2 very sanitized events.

After Mod Event --
{
"EventTime" : "02/23/2017-09:07:47",
"EventName" : "ChangeObject",
"EventType" : "Configuration",
"MQLONG MsgSeqNumber" : "2(0x00000002)",
"MQCA_ALTERATION_DATE" : "2017-02-23",
"MQCA_ALTERATION_TIME" : "09.07.47",
}

Before Mod Event --
{
"EventTime" : "02/23/2017-09:07:47",
"EventName" : "ChangeObject",
"EventType" : "Configuration",
"MQLONG MsgSeqNumber" : "1(0x00000001)",
"MQCA_ALTERATION_DATE" : "2017-02-23",
"MQCA_ALTERATION_TIME" : "09.01.47",
}

This stats command gives me the values for each field.
... | stats values(*) AS *

All I want to see are the fields that are different, in this case the "MQLONG MsgSeqNumber" and "MQCA_ALTERATION_TIME". Amny of the fields are all text. Again, there can be over 80 continually changing fields, so I can't write anything that lists specific fields. This seems like something that should be fairly common, but I can't find any threads that answer my specific need.

Hoping someone can share what they've done or tackle this challenge 🙂

Re: Display only differences in values, between 2 events

chengka — Tue, 28 Feb 2017 21:49:27 GMT

I missed an , should be "| stats values() AS *"

Re: Display only differences in values, between 2 events

woodcock — Tue, 28 Feb 2017 22:04:00 GMT

Like this:

... | stats values(*) AS * count(*) AS count* dc(*) AS dc* | foreach count* [eval '<<MATCHSTR>>'=if(($<<FIELD>>$=$dc<<MATCHSTR>>$), null(), $<<MATCHSTR>>$)] | fields - count* dc* | table *

Re: Display only differences in values, between 2 events

chengka — Tue, 28 Feb 2017 22:25:14 GMT

Thank you. I saw that answer in another thread, but it doesn't work for me. I get this error. I don't even have that field.. stumped..
Failed to parse templatized search for field 'counttag::eventtype'

Re: Display only differences in values, between 2 events

somesoni2 — Tue, 28 Feb 2017 22:51:10 GMT

Give this a try. Assuming you've only two events so, if the value is same for a field, values(field) will give a single value (mvcount will be 1).

your base search | stats values(*) as *  | eval temp=1 
| untable temp fieldname fieldvalue | where mvcount(fieldvalue)!=1
| xyseries temp fieldname fieldvalue | fields - temp

Re: Display only differences in values, between 2 events

woodcock — Wed, 01 Mar 2017 02:15:48 GMT

Try my updated answer to account for the colons in your field names. It will work on the search bar but not in a dashboard panel without extra modification.

Re: Display only differences in values, between 2 events

DalJeanis — Wed, 01 Mar 2017 05:26:16 GMT

I was thinking something like that. Required assumption is that there are exactly two events returned from the underlying search.

What's the advantage of using xyseries here?

Re: Display only differences in values, between 2 events

chengka — Wed, 01 Mar 2017 15:45:34 GMT

Thank you, I will try it, but I don't see any colons in my field names, just the separator.

Re: Display only differences in values, between 2 events

woodcock — Wed, 01 Mar 2017 15:47:57 GMT

It is right there in the error: a field named tag::eventtype.

Re: Display only differences in values, between 2 events

somesoni2 — Wed, 01 Mar 2017 15:52:56 GMT

xyseries was to revert to original result format (stats values(*) as * gives a column for each field). Filtering is easier with rows so I untabled columns to row,, filtered it and then reverted to original format.

Re: Display only differences in values, between 2 events

chengka — Tue, 29 Sep 2020 13:05:05 GMT

Thank you for your patience. I ran the search and it returns all fields, not only those that are different. This is what I saw when I tried this solution from another thread.

source="splunkanswers.txt" index=testing | stats values() AS * count() AS count* dc() AS dc | foreach count* [eval '<>'=if(($<>$=$dc<>$), null(), $<>$)] | fields - count* dc* | table *

Here is partial result exported to csv. Several of the fields shown only have 1 value.
EventName EventTime MQCA_ALTERATION_DATE MQCA_ALTERATION_TIME MQLONG MsgSeqNumber date_hour date_mday
ChangeObject 02/23/2017-09:07:47 42789 09.01.47 09.07.47 1(0x00000001) 2(0x00000002) 9 23

Re: Display only differences in values, between 2 events

woodcock — Wed, 01 Mar 2017 16:29:13 GMT

Give the other answer a try.

Re: Display only differences in values, between 2 events

chengka — Tue, 29 Sep 2020 13:05:11 GMT

The idea of value counts was the way I was thinking. I tried your solution and it doesn't find any results, buy trimming back the search I can see that it lists the values, but the where clause doesn't return any data. Maybe I need to reencode the untabled data as MV..

temp fieldname fieldvalue
1 EventName ChangeObject
1 EventTime 02/23/2017-09:07:47
1 MQCA_ALTERATION_DATE 2017-02-23
1 MQCA_ALTERATION_TIME 09.01.47 09.07.47
1 MQLONG MsgSeqNumber 1(0x00000001) 2(0x00000002)

Re: Display only differences in values, between 2 events

somesoni2 — Wed, 01 Mar 2017 16:41:37 GMT

Lets give this a try

your base search | stats values(*) as *  delim="##" | eval temp=1 
 | untable temp fieldname fieldvalue | makemv fieldvalue delim="##" | where mvcount(fieldvalue)!=1
 | xyseries temp fieldname fieldvalue | fields - temp

Re: Display only differences in values, between 2 events

chengka — Wed, 01 Mar 2017 20:01:23 GMT

That worked! I added some OR statements to the where, so I can see other relevant fields that identify the resource. Thank you!