Why does search return incorrect dc count?

[Dzmitry]

Hi Team,

I noticed that for some hosts search returns incorrect dc count:

1) the query to dc count ids when status is failed
index=".." exec_mode="..." host_name="test_host" status="failed" | stats  dc(id) AS failed BY host_name | table host_name failed (returns 1)

2)  the query to dc count ids when status is skipped or passed

index="..." exec_mode="..." host_name="test_host" (status="skipped" OR status="passed") | stats  dc(id) AS pass_skip BY host_name | table host_name pass_skip (returns 234)

3) the query to dc count every id 

index="..." exec_mode="..." host_name="test_host" | stats dc(id) AS executed BY host_name | table host_name executed (returns 234)

but I expect that query #3 returns sum queries 1 (failed) and 2 (skipped and passed) : 1 + 234 = 235
the I try to play with statuses in the query to get the total ids dc count
4) index="..." exec_mode="..." host_name="test_host" (status="failed" OR status!="failed") | stats  dc(id) AS failed BY host_name | table host_name failed ( it returns also 234)

Only 10% of hosts have such odd search behavior,  for another 90% total = failed + passed/skipped

Thank you in advance!

[gcusello]

Hi @Dzmitry,

which time period did you used?

don't use a rtime period containing latest=now, but a closed time period: e.g. yesterday, earliest=-h@h latest=@h.

To be sure that the number of events is fixed.

Ciao.

Giuseppe

[Dzmitry]

Hi @gcusello

I use "Last 7 days" 

I tried "Date range" and "Date & Time range" "Advanced" and got same issue

[gcusello]

Hi @Dzmitry ,

please try, always using a fixed and closed time (e.g. last week) the following search and see if the results are correct:

index="..." exec_mode="..." host_name="test_host" (status="failed" OR status="skipped" OR status="passed") 
| stats  dc(id) AS pass_skip BY host_name 
| table host_name pass_skip

if they are correct means that you have events outside the three types you're using in your searches.

ciao.

Giuseppe

[Dzmitry]

Unfortunately, it doesn't help.
I get the same result for queries with (status="skipped" OR status="passed") and (status="skipped" OR status="passed" OR status="failed"), tried all fixed time ranges. 

[gcusello · Accepted solution]

Hi @Dzmitry,

what's the result (always in the same time period) of :

index="..." exec_mode="..." host_name="test_host" (status="failed" OR status="skipped" OR status="passed") 
| stats  dc(id) AS pass_skip BY host_name status
| stats sum(pass_skip) AS total BY status

Ciao.

Giuseppe

[Dzmitry]

Thank you and Merry Christmas😃
final version 
index="..." exec_mode="..." host_name="..." (status="skipped" OR status="passed" OR status="failed") | stats dc(id) AS executed BY host_name status | stats sum(executed ) AS total BY host_name

[gcusello]

Hi @Dzmitry,

good for you, see next time!

Please accept one answer for the other people of Community

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉