Visually correlate two searches by field and count

[Ted-Splunk]

There is an async process that logs first when something is created, then again when it is picked up by a service that will send a notification. I would like to create a visual that easily identifies customers with dropped notifications. (The two events occur within a minute.) I have two searches that share a common ID and identify the events above. When things are working well the counts by customer per day will be the same from both searches.  Is it possible to correlate the two searches to show any differences in these counts?

In this sanitized example MyIdA is the ID from search A while MyIdB is the same ID from search B

(search A) OR (search B)
| rex "cust (?<MyIdA>\d+)"
| rex "\"custId\",\"value\":\"(?<MyIdB>\d+)"

[gcusello]

Hi @Ted-Splunk ,

you should correlate the two searches using the common key (using eval coalesce) and check the presence in both searches (e.g. if they are in different indexes), something like this:

(search A) OR (search B)
| rex "cust (?<MyIdA>\d+)"
| rex "\"custId\",\"value\":\"(?<MyIdB>\d+)"
| eval key=coalesce(MyIdA,MyIdB)
| stats dc(index) AS inex_count BY key
| where index_count>1

 modify the condition based on the way to identify the two searches.

Ciao.

Giuseppe

[yuanliu]

I think you meant index_count < 2, not > 1 because OP wants to find keys with differing counts.

The formula is suitable if maximum number each key can appear in an index is 1.  If the key can appear more than once and the OP wants to find keys that have differing counts, the formula should change to

(search A) OR (search B)
| rex "cust (?<MyIdA>\d+)"
| rex "\"custId\",\"value\":\"(?<MyIdB>\d+)"
| eval key=coalesce(MyIdA,MyIdB)
``` above is the same as gcusello's ```
| stats count by key index
| eval index_count = index . ": " . count
| stats values(index_count) as index_count dc(count) as _distinct_count by key
| where _distinct_count > 1 OR mvcount(index_count) < 2

In the following scenario, for example,

index	key
index1	A
index1	B
index2	A
index2	B
index2	C
index2	B

simply counting indices with condition index_count < 2 will give

key	index_count
C	1

This misses the differing counts for key B.  The more nuanced counts presented here will give

key	index_count
B	index1: 1 index2: 2
C	index2: 1

As as side, the original formula by simple count of indices with index_count > 1 will give

key	index_count
A	2
B	2

This is also incorrect because A is perfectly balanced while B is not.