topic Re: How to compare the results of 2 searches? in Splunk Search

How to compare the results of 2 searches?

ckutach — Mon, 21 Aug 2023 20:01:09 GMT

I have splunk logs that are of 2 types, successes and failures. They contain 2 things:

"SUCCESS" "ID: <IDNumber>"

"FAILURE" "ID: <IDNumber>"

My goal is to find IDs that are identified with failures that are not also identified with a success. So for the data:

"SUCCESS" "ID: 0000", "FAILURE" "ID: 0000", "SUCCESS" "ID: 1111", "FAILURE" "ID: 2222", "SUCCESS" "ID: 3333", "FAILURE" "ID: 4444"

the result would be the IDs 2222 and 4444

My current search is:

index=sampleindex source=samplesource "SUCCESS" | rex field=_raw "ID: (?<id1>+)" | join [search index=sampleindex source=samplesource "FAILURE" | rex field=_raw "ID: (?<id2>+)"] | table id1, id2

I am able to perform each of these searches separately and output the ids, but when I combine them I cannot get the results of either id1 or id2, so I am not able to compare them

Does anyone know how I can structure my search to achieve my final goal?

Re: How to compare the results of 2 searches?

yuanliu — Mon, 21 Aug 2023 22:37:55 GMT

You can perform a set operation on the two searches as illustrated. But a far more efficient method is to perform just one search and use mvmap.

index=sampleindex source=samplesource (SUCCESS OR FAILURE) | rex field=data max_match=0 "\"SUCCESS\" \"ID:\s*(?<success>\d+)" | rex field=data max_match=0 "\"FAILURE\" \"ID:\s*(?<failure>\d+)" | eval failed = mvmap(failure, if(failure != success, failure, null()))

(In SPL, equality test of a scalar and a vector iterates over the vector.) The field failed now contains all the failure IDs that is not found in success.

Re: How to compare the results of 2 searches?

bowesmana — Tue, 22 Aug 2023 00:42:05 GMT

I am assuming that you will have 2 separate events, one for success and one for failure, so in that case, you'll need a slightly different version of @yuanliu SPL

index=sampleindex source=samplesource (SUCCESS OR FAILURE) | rex field=data "\"(?<status>SUCCESS)\" \"ID:\s*(?<id>\d+)" | rex field=data "\"(?<status>FAILURE)\" \"ID:\s*(?<id>\d+)" | stats values(status) as status by id | where mvcount(status)=1 AND status="FAILURE"

so the stats command combines all status values for the same id and then the where clause filters out only those ids where the status is FAILURE only