topic Re: How to loop through the record to get the diff between them in Splunk Search

How to loop through the record to get the diff between them

zhanweiw — Tue, 16 Nov 2021 07:20:37 GMT

Hi there,

I am trying to diff the new version against the one version older record and extract the diff from them.

For example,

ver 1.3 against 1.2 and ver 1.2 against 1.1 to only extract the diff between them.

I hope to do it in a flexible ways as in future I may have ver 1.4 and so on...

I also want to limit the results to only latest 5 version diff.

For example,

I got 1.1, 1.2, ..., 1.10 version, but I only want the result for 1.6, 1.7, ..., 1.10 when diff against the previous one version.

Is that possible?

Currently I have data like this:

records:

============================================

index=a, ver=1.1, a="halo", b="haha", c="nana"

index=a, ver=1.1, a="testing", b="haha", c="nana"

index=a, ver=1.1, a="halo", b="kaka", c="testing"

index=a, ver=1.2, a="halo", b="haha", c="nana"

index=a, ver=1.2, a="lala", b="haha", c="nana"

index=a, ver=1.2, a="halo", b="kaka", c="TESTING"

index=a, ver=1.3, a="halo", b="haha", c="nana"

index=a, ver=1.3, a="lala", b="haha", c="tata"

index=a, ver=1.3, a="halo", b="kaka", c="lala"

index=a, ver=1.3, a="halo", b="kaka", c="kakaka"

============================================

Result expected when comparing ver 1.2 against 1.1 and ver1.3 against 1.2:

ver

added record (merging a b c using ",")

removed record (merging a b c using ",")

1.2

lala,haha,nana

halo,kaka,TESTING

testing,haha,nana

halo,kaka,testing

1.3

lala,haha,tata

halo,kaka,lala

halo,kaka,kakaka

lala,haha,nana

halo,kaka,TESTING

Re: How to loop through the record to get the diff between them

ITWhisperer — Tue, 16 Nov 2021 07:15:32 GMT

Are these separate events in splunk?

You seem to be comparing the first event in version 1.1 with the first event in version 1.2, and the second event in version 1.1 with the second event in version 1.2, and so on. Is this correct? Is there some other way to identify which events should be compared against each other?

Why does lala, haha, tata not feature as an addition in version 1.3 since it doesn't appear in version 1.2?

Re: How to loop through the record to get the diff between them

zhanweiw — Tue, 16 Nov 2021 07:28:12 GMT

Thanks for pointing out, I have updated the post.

You seem to be comparing the first event in version 1.1 with the first event in version 1.2, and the second event in version 1.1 with the second event in version 1.2, and so on. Is this correct?

Ya, I am comparing version 1.2 against 1.1 then 1.3 against 1.2.

Is there some other way to identify which events should be compared against each other?

No other way. Not sure is it possible to use mvindex to get list of uniq version then do zipping like below and compare against each other.

[(1.2, 1.1)
(1.3, 1.2)]

Re: How to loop through the record to get the diff between them

ITWhisperer — Tue, 16 Nov 2021 07:55:13 GMT

| makeresults | eval _raw="index=a, ver=1.1, a=\"halo\", b=\"haha\", c=\"nana\" index=a, ver=1.1, a=\"testing\", b=\"haha\", c=\"nana\" index=a, ver=1.1, a=\"halo\", b=\"kaka\", c=\"testing\" index=a, ver=1.2, a=\"halo\", b=\"haha\", c=\"nana\" index=a, ver=1.2, a=\"lala\", b=\"haha\", c=\"nana\" index=a, ver=1.2, a=\"halo\", b=\"kaka\", c=\"TESTING\" index=a, ver=1.3, a=\"halo\", b=\"haha\", c=\"nana\" index=a, ver=1.3, a=\"lala\", b=\"haha\", c=\"tata\" index=a, ver=1.3, a=\"halo\", b=\"kaka\", c=\"lala\" index=a, ver=1.3, a=\"halo\", b=\"kaka\", c=\"kakaka\"" | multikv noheader=t | table _raw | kv | streamstats count as row by ver | sort 0 row ver | streamstats list(a) as previous_a list(b) as previous_b list(c) as previous_c window=1 current=f by row | where ver!="1.1" | where a!=previous_a OR b!=previous_b OR c!=previous_c OR isnull(previous_a) OR isnull(previous_b) OR isnull(previous_c) | sort 0 ver row | eval added=a.",".b.",".c | eval removed=previous_a.",".previous_b.",".previous_c | stats list(added) as added list(removed) as removed by ver

Re: How to loop through the record to get the diff between them

zhanweiw — Tue, 16 Nov 2021 08:07:19 GMT

Hi ITWhisperer,

Thanks for your respond but not sure why the result is empty after I run it.

Can you help to confirm?

Re: How to loop through the record to get the diff between them

ITWhisperer — Tue, 16 Nov 2021 08:23:03 GMT

Here is a screenshot of it working

What are you doing differently?

Re: How to loop through the record to get the diff between them

zhanweiw — Tue, 16 Nov 2021 08:34:06 GMT

Hi ITWhisperer,

Not sure why after I copy again and paste it to the search engine.

It is working as expected.

Thank for your help 🙂