Splunk Search

Display only differences in values, between 2 events

chengka
Explorer

Hello,
I'm looking events that track changes to a configuration. The first event is the "before" state the newest event is the "after" state. There events are in json format and there are > 80 fields. I have a search that will display all of the values for each field present, but it's not practical to display a table with 80 columns and expect the user to locate the columns with multiple values.

Here are 2 very sanitized events.

After Mod Event --
{
"EventTime" : "02/23/2017-09:07:47",
"EventName" : "ChangeObject",
"EventType" : "Configuration",
"MQLONG MsgSeqNumber" : "2(0x00000002)",
"MQCA_ALTERATION_DATE" : "2017-02-23",
"MQCA_ALTERATION_TIME" : "09.07.47",
}

Before Mod Event --
{
"EventTime" : "02/23/2017-09:07:47",
"EventName" : "ChangeObject",
"EventType" : "Configuration",
"MQLONG MsgSeqNumber" : "1(0x00000001)",
"MQCA_ALTERATION_DATE" : "2017-02-23",
"MQCA_ALTERATION_TIME" : "09.01.47",
}

This stats command gives me the values for each field.
... | stats values(*) AS *

All I want to see are the fields that are different, in this case the "MQLONG MsgSeqNumber" and "MQCA_ALTERATION_TIME". Amny of the fields are all text. Again, there can be over 80 continually changing fields, so I can't write anything that lists specific fields. This seems like something that should be fairly common, but I can't find any threads that answer my specific need.

Hoping someone can share what they've done or tackle this challenge 🙂

Tags (1)
0 Karma
1 Solution

somesoni2
Revered Legend

Give this a try. Assuming you've only two events so, if the value is same for a field, values(field) will give a single value (mvcount will be 1).

your base search | stats values(*) as *  | eval temp=1 
| untable temp fieldname fieldvalue | where mvcount(fieldvalue)!=1
| xyseries temp fieldname fieldvalue | fields - temp

View solution in original post

somesoni2
Revered Legend

Give this a try. Assuming you've only two events so, if the value is same for a field, values(field) will give a single value (mvcount will be 1).

your base search | stats values(*) as *  | eval temp=1 
| untable temp fieldname fieldvalue | where mvcount(fieldvalue)!=1
| xyseries temp fieldname fieldvalue | fields - temp

DalJeanis
SplunkTrust
SplunkTrust

I was thinking something like that. Required assumption is that there are exactly two events returned from the underlying search.

What's the advantage of using xyseries here?

0 Karma

somesoni2
Revered Legend

xyseries was to revert to original result format (stats values(*) as * gives a column for each field). Filtering is easier with rows so I untabled columns to row,, filtered it and then reverted to original format.

chengka
Explorer

The idea of value counts was the way I was thinking. I tried your solution and it doesn't find any results, buy trimming back the search I can see that it lists the values, but the where clause doesn't return any data. Maybe I need to reencode the untabled data as MV..

temp fieldname fieldvalue
1 EventName ChangeObject
1 EventTime 02/23/2017-09:07:47
1 MQCA_ALTERATION_DATE 2017-02-23
1 MQCA_ALTERATION_TIME 09.01.47 09.07.47
1 MQLONG MsgSeqNumber 1(0x00000001) 2(0x00000002)

0 Karma

somesoni2
Revered Legend

Lets give this a try

your base search | stats values(*) as *  delim="##" | eval temp=1 
 | untable temp fieldname fieldvalue | makemv fieldvalue delim="##" | where mvcount(fieldvalue)!=1
 | xyseries temp fieldname fieldvalue | fields - temp
0 Karma

chengka
Explorer

That worked! I added some OR statements to the where, so I can see other relevant fields that identify the resource. Thank you!

0 Karma

woodcock
Esteemed Legend

Like this:

... | stats values(*) AS * count(*) AS count* dc(*) AS dc* | foreach count* [eval '<<MATCHSTR>>'=if(($<<FIELD>>$=$dc<<MATCHSTR>>$), null(), $<<MATCHSTR>>$)] | fields - count* dc* | table *

chengka
Explorer

Thank you. I saw that answer in another thread, but it doesn't work for me. I get this error. I don't even have that field.. stumped..
Failed to parse templatized search for field 'counttag::eventtype'

0 Karma

woodcock
Esteemed Legend

Try my updated answer to account for the colons in your field names. It will work on the search bar but not in a dashboard panel without extra modification.

0 Karma

chengka
Explorer

Thank you, I will try it, but I don't see any colons in my field names, just the separator.

0 Karma

woodcock
Esteemed Legend

It is right there in the error: a field named tag::eventtype.

0 Karma

chengka
Explorer

Thank you for your patience. I ran the search and it returns all fields, not only those that are different. This is what I saw when I tried this solution from another thread.

source="splunkanswers.txt" index=testing | stats values() AS * count() AS count* dc() AS dc | foreach count* [eval '<>'=if(($<>$=$dc<>$), null(), $<>$)] | fields - count* dc* | table *

Here is partial result exported to csv. Several of the fields shown only have 1 value.
EventName EventTime MQCA_ALTERATION_DATE MQCA_ALTERATION_TIME MQLONG MsgSeqNumber date_hour date_mday
ChangeObject 02/23/2017-09:07:47 42789 09.01.47 09.07.47 1(0x00000001) 2(0x00000002) 9 23

0 Karma

woodcock
Esteemed Legend

Give the other answer a try.

0 Karma

chengka
Explorer

I missed an , should be "| stats values() AS *"

0 Karma
Get Updates on the Splunk Community!

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...

Getting Started with AIOps: Event Correlation Basics and Alert Storm Detection in ...

Getting Started with AIOps:Event Correlation Basics and Alert Storm Detection in Splunk IT Service ...

Register to Attend BSides SPL 2022 - It's all Happening October 18!

Join like-minded individuals for technical sessions on everything Splunk!  This is a community-led and run ...