Why am I getting different results for "stats count" and "tstats count"?

[thippeshaj]

Hi All,

I'm getting a different values for stats count and tstats count.
Sometimes the data will fix itself after a few days, but not always. I can’t use the data displayed on the dashboard AS is, reason being it’s not reliable, unless I manually do a reconciliation, and if it doesn’t tally, there is pretty much nothing I can do to get the correct data except to wait for a few more days and check again.

Is there anything that I need to correct ??
please help me.

[alacercogitatus · Accepted solution]

tstats can support spans of 1s as seen in the picture. This was generated from the run-anywhere search:

index=_internal |timechart span=1s count as time_count | join _time type=left [ | tstats count as tstats_count WHERE index=_internal  BY _time span=1s ]

The problem is probably stemming from the tokenization of your data. According to the docs (http://docs.splunk.com/Documentation/Splunk/6.3.1/SearchReference/Tstats 😞

You might see a count mismatch in the events retrieved when searching tsidx files. This is because it is not possible to distinguish between indexed field tokens and raw tokens in tsidx files. On the other hand, it is more explicit to run tstats on accelerated datamodels or from a tscollect, where only the fields and values are stored and not the raw tokens. 

Therefore, the counts might not always be the same. I'm not 100% sure on how to solve this yet, but are you using tscollect or an accelerated datamodel? Those two will return the correct fields, at the cost of pre-processing via tscollect or the cost of datamodel acceleration.

[ppablo]

Hi @thippeshaj

I noticed you upvoted this answer, but didn't click "Accept", so I accepted it for you for now, but can you confirm if this actually did solve your question? Did you check the possible default searched index issue suggested by @acharlieh or run the test searches suggested by @martin_mueller in the comments under your question? Please comment with follow-up so other users are aware.

[woodcock]

The problem is that Splunk has bucketed your data in spans bigger than the 1s value that you have specified in your span.

When I do this:

| tstats count WHERE index=_internal sourcetype=splunkd host="MyHost" BY _time | delta _time AS tstatsBucketSpanSecs | stats values(tstatsBucketSpanSecs)

Then I get this:

values(tstatsBucketSpanSecs)
60

This means that I must not specify a value for span that is smaller than the size that you find splunk is bucketing for your data or it will not work correctly.
So try changing your span to fit what it is doing and I suspect that it will work the same for both.

[thippeshaj]

Hi woodcock
thanks,

Evan after changing the span still total number of events are not matching

[woodcock]

What are your search strings?

[thippeshaj]

Normal Statistic
Query : host=xyz sourcetype=abcd | timechart span=1s count

Tstats
Query : | tstats count as "count" WHERE index=main sourcetype=abcd host=xyz by _time span=1s

[acharlieh]

Out of curiosity, do you have multiple indexes? I notice your tstats query has index=main, but your normal query does not.

[thippeshaj]

Yes achrlieh we have multiple indexes

[martin_mueller]

Run these two:

index=main host=xyz sourcetype=abcd | timechart span=1s count
| tstats count WHERE index=main host=xyz sourcetype=abcd  by _time span=1s prestats=t | timechart span=1s count

Two changes: I've added index=main to the first search, and prestats=t | timechart ... to the second. The index makes sure you're really using the same input data for both searches. The prestats/timechart makes sure you're producing the same type of output - for example, with empty buckets filled.

[thippeshaj]

@martin_mueller
I tried but still got the different count....