topic Re: Why am I getting different results for "stats count" and "tstats count"? in Splunk Search

Why am I getting different results for "stats count" and "tstats count"?

thippeshaj — Fri, 11 Dec 2015 05:57:25 GMT

Hi All,

I'm getting a different values for stats count and tstats count.
Sometimes the data will fix itself after a few days, but not always. I can’t use the data displayed on the dashboard AS is, reason being it’s not reliable, unless I manually do a reconciliation, and if it doesn’t tally, there is pretty much nothing I can do to get the correct data except to wait for a few more days and check again.

Is there anything that I need to correct ??
please help me.

Re: Why am I getting different results for "stats count" and "tstats count"?

woodcock — Fri, 11 Dec 2015 06:42:09 GMT

What are your search strings?

Re: Why am I getting different results for "stats count" and "tstats count"?

thippeshaj — Fri, 11 Dec 2015 11:53:24 GMT

Normal Statistic
Query : host=xyz sourcetype=abcd | timechart span=1s count

Tstats
Query : | tstats count as "count" WHERE index=main sourcetype=abcd host=xyz by _time span=1s

Re: Why am I getting different results for "stats count" and "tstats count"?

woodcock — Sat, 12 Dec 2015 04:52:08 GMT

The problem is that Splunk has bucketed your data in spans bigger than the 1s value that you have specified in your span.

When I do this:

| tstats count WHERE index=_internal sourcetype=splunkd host="MyHost" BY _time | delta _time AS tstatsBucketSpanSecs | stats values(tstatsBucketSpanSecs)

Then I get this:

values(tstatsBucketSpanSecs)
60

This means that I must not specify a value for span that is smaller than the size that you find splunk is bucketing for your data or it will not work correctly.
So try changing your span to fit what it is doing and I suspect that it will work the same for both.

Re: Why am I getting different results for "stats count" and "tstats count"?

thippeshaj — Mon, 14 Dec 2015 06:59:55 GMT

Hi woodcock
thanks,

Evan after changing the span still total number of events are not matching

Re: Why am I getting different results for "stats count" and "tstats count"?

alacercogitatus — Mon, 14 Dec 2015 12:44:16 GMT

tstats can support spans of 1s as seen in the picture. This was generated from the run-anywhere search:

index=_internal |timechart span=1s count as time_count | join _time type=left [ | tstats count as tstats_count WHERE index=_internal  BY _time span=1s ]

The problem is probably stemming from the tokenization of your data. According to the docs (http://docs.splunk.com/Documentation/Splunk/6.3.1/SearchReference/Tstats 😞

You might see a count mismatch in the events retrieved when searching tsidx files. This is because it is not possible to distinguish between indexed field tokens and raw tokens in tsidx files. On the other hand, it is more explicit to run tstats on accelerated datamodels or from a tscollect, where only the fields and values are stored and not the raw tokens.

Therefore, the counts might not always be the same. I'm not 100% sure on how to solve this yet, but are you using tscollect or an accelerated datamodel? Those two will return the correct fields, at the cost of pre-processing via tscollect or the cost of datamodel acceleration.

Re: Why am I getting different results for "stats count" and "tstats count"?

acharlieh — Mon, 14 Dec 2015 13:51:09 GMT

Out of curiosity, do you have multiple indexes? I notice your tstats query has index=main, but your normal query does not.

Re: Why am I getting different results for "stats count" and "tstats count"?

martin_mueller — Mon, 14 Dec 2015 17:47:08 GMT

Run these two:

index=main host=xyz sourcetype=abcd | timechart span=1s count
| tstats count WHERE index=main host=xyz sourcetype=abcd  by _time span=1s prestats=t | timechart span=1s count

Two changes: I've added index=main to the first search, and prestats=t | timechart ... to the second. The index makes sure you're really using the same input data for both searches. The prestats/timechart makes sure you're producing the same type of output - for example, with empty buckets filled.

Re: Why am I getting different results for "stats count" and "tstats count"?

ppablo — Tue, 15 Dec 2015 19:20:47 GMT

Hi @thippeshaj

I noticed you upvoted this answer, but didn't click "Accept", so I accepted it for you for now, but can you confirm if this actually did solve your question? Did you check the possible default searched index issue suggested by @acharlieh or run the test searches suggested by @martin_mueller in the comments under your question? Please comment with follow-up so other users are aware.

Re: Why am I getting different results for "stats count" and "tstats count"?

thippeshaj — Wed, 16 Dec 2015 06:27:36 GMT

Yes achrlieh we have multiple indexes

Re: Why am I getting different results for "stats count" and "tstats count"?

thippeshaj — Wed, 16 Dec 2015 06:30:32 GMT

@martin_mueller
I tried but still got the different count....