Splunk Cloud Platform

Is this a Splunk Bug? tstats + PREFIX() over a summary index doesn't return all DISTINCT COUNTS?

isaiz
Loves-to-Learn Lots

Hello again.

 

I am testing a "light" version of an index completely compatible with the tstats + PREFIX() method (selecting only the fields I work with and removing all major breakers of field values from the _raw) as an alternative to datamodels, since it's waaaay faster.

newraw_test gen.PNG

 

My first test has been computing the distinct count value of a field (sessionid) with extremely high cardinality but without major breakers (so prefix compatible) both in the original index and my summary index for a given hour.

ORIGINAL INDEX:  48.692.463 distinct session ids

fortinet_data dc.PNG

SUMMARY INDEX: 6.016.022 distinct session ids

newraw_test dc.PNG

 

However, if I do the alternative way of doing DC (count by sessionid so for each different sessionid it generates a row and then I count all the rows) it gives me the correct result.

SUMMARY INDEX with count of counts method: 48.692.463 distinct session ids
newraw_test count count.PNG

 

So the problem is in the DC function. It seems the issue occurs when splunk gathers the DC chunks to generate the final result, but tuning chunk_size parameter has no effect whatsoever. When I do the same test with smaller time ranges so distinct sessions >1.000.000 both original index and summary index DCs give me the same result.

 

How can I solve the problem? Is this a Splunk bug?

Labels (2)
0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July Tech Talks, Office Hours, and Webinars!

What are Community Office Hours?Community Office Hours is an interactive 60-minute Zoom series where ...

Updated Data Type Articles, Anniversary Celebrations, and More on Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

A Prelude to .conf25: Your Guide to Splunk University

Heading to Boston this September for .conf25? Get a jumpstart by arriving a few days early for Splunk ...