topic Re: Compare values of 2 columns in a table in Splunk Search

Compare values of 2 columns in a table

ramonnegronvz — Fri, 01 May 2020 19:24:04 GMT

I have a table that has 2 columns with Transaction ID's shown by a stats values() as below:

| stats values(E-TransactionID) as E-TransactionID values(R-TransactionID) as R-TransactionID

I'd like to compare the values of both columns and only show the Transaction ID's from R-TransactionID that does NOT appear in the E-TransactionID column.

I've made the following attempts after the stats values() with no luck. Any help is GREATLY appreciated.

Attempt 1 (had to try this anyway):
| table R-TransactionID E-TransactionID
| where R-TransactionID != E-TransactionID

Attempt 2:
| eval match=if(R-TransactionID=E-TransactionID, "EQUAL", R-TransactionID)
| stats values(match) as TransactionID

Attempt 3:
| foreach R-TransactionID
[eval match=if(R-TransactionID!=E-TransactionID, R-TransactionID, "MATCH")]
| stats values(R-TransactionID) as R-TransactionID values(E-TransactionID) as E-TransactionID values(match) as TransactionID

Attempt 4 (similar to previous, but with table instead😞
| foreach R-TransactionID
[eval match=if(R-TransactionID!=E-TransactionID, R-TransactionID, "MATCH")]
| stats values(R-TransactionID) as R-TransactionID values(E-TransactionID) as E-TransactionID values(match) as TransactionID

Re: Compare values of 2 columns in a table

richgalloway — Fri, 01 May 2020 20:12:14 GMT

One problem is the values function returns a multi-value field which does not work well with other commands and functions. Usually, mvexpand helps, but you have two multi-value fields you're trying to match up.

Perhaps you could take a step back and describe the original problem you want to solve.

Re: Compare values of 2 columns in a table

ramonnegronvz — Wed, 30 Sep 2020 05:11:49 GMT

Thanks @richgalloway
Within the same index/sourcetype, there are 3 messages which I need to track in order to come up with the Transaction ID's that "matter". In the below query, I'm correlating 2 of the strings within the log events to come up with the E-TransactionID's (from the values of transid). I'm doing another search (string_3) to come up with R-TransactionID, which tells me about the Transaction ID's that "matter", but I want to "ignore" them if the Transaction ID matches E-TransactionID. In other words, my goal is to find the Transaction ID's that appear on R-TransactionID but NOT on E-TransactionID.

From your comment, do you think if I change values to table, I'll be able to accomplish what I need?

Here's a "dummy-down" search query I'm running to come up with both E-TransactionID and R-TransactionID:

index=index_p sourcetype=st_p message="string_1"
| join transid
[ search index=index_p sourcetype=st_p message="string_2"
| table transid]
| stats values(transid) as E-TransactionID
| append
[search index=index_p sourcetype=st_p message="string_2"
| stats values(transid) as R-TransactionID]
| stats values(E-TransactionID) as E-TransactionID values(R-TransactionID) as R-TransactionID

Re: Compare values of 2 columns in a table

to4kawa — Fri, 01 May 2020 21:49:59 GMT

sample:

| makeresults count=20
| streamstats count
| stats values(count) as R
| eval E=split("1,4,6",",")
| eval tmp=mvsort(mvappend(E,R))
| stats count by tmp
| where count < 2

try mvappend and stats by

but,

index=index_p sourcetype=st_p message="string_1" OR message="string_2"
| stats dc(message) as flag count(eval(message="string_1")) as message1  by  transid
| where flag < 2 AND message1 > 0
| table transid

Your query does not need them.

Re: Compare values of 2 columns in a table

ramonnegronvz — Fri, 01 May 2020 22:18:22 GMT

Thanks @to4kawa
Let me see how can I attempt using mvappend and stats to fit my need here.

Re: Compare values of 2 columns in a table

to4kawa — Fri, 01 May 2020 22:49:55 GMT

Hi @ramonnegronvz
my answer is updated.

Re: Compare values of 2 columns in a table

ramonnegronvz — Sun, 03 May 2020 22:27:52 GMT

I must admit, that at the beginning I was trying to figure out how would this help, but after trying it out, and testing it out, it is perfect! Thanks so much @to4kawa!