Splunk Search

Why do Stats and Evenstats return different results when calculating percentiles?

MattLingwood
Engager

I am trying to work out the 99th percentile of some response times and after seeing discrepancies in a couple of search queries I have narrowed it down to the | Stats and | Eventstats command.

My Eventstats query:
source="script-output" name="MyScriptResponses"
| rename response{}.ms as response_time_ms
| mvexpand response_time_ms
| eventstats p99(response_time_ms) as "99th response time (ms)"
| table name "99th response time (ms)"
returns 200

My Stats query:
source="http-simple" name="MyScriptResponses"
| rename response{}.ms as response_time_ms
| mvexpand response_time_ms
| stats p99(response_time_ms) as "99th response time (ms)"
| table name "99th response time (ms)"
returns 190

Some info about my data:
There are 3828 response times.
It is a set time of "Previous Week"
I tried the same with Average and got the same answers.
I tried different percentiles; 75, 90, 95, 99 and only 90 returned the same data.

0 Karma
1 Solution

hliakathali_spl
Splunk Employee
Splunk Employee

"Generate summary statistics of all existing fields in your search results and save them as values in new fields. Eventstats is similar to the stats command, except that aggregation results are added inline to each event and only if the aggregation is pertinent to that event."

In the limits.conf file, the max_mem_usage_mb parameter is used to limit how much memory the stats and eventstats commands use to keep track of information. If the eventstats command reaches this limit, the command stops adding the requested fields to the search results. You can increase the limit, contingent on the available system memory.

Also, here are the link to the documentation files for stats and evenstats respectively:

http://docs.splunk.com/Documentation/Splunk/5.0.3/SearchReference/Stats
http://docs.splunk.com/Documentation/Splunk/5.0.3/SearchReference/Eventstats

View solution in original post

hliakathali_spl
Splunk Employee
Splunk Employee

"Generate summary statistics of all existing fields in your search results and save them as values in new fields. Eventstats is similar to the stats command, except that aggregation results are added inline to each event and only if the aggregation is pertinent to that event."

In the limits.conf file, the max_mem_usage_mb parameter is used to limit how much memory the stats and eventstats commands use to keep track of information. If the eventstats command reaches this limit, the command stops adding the requested fields to the search results. You can increase the limit, contingent on the available system memory.

Also, here are the link to the documentation files for stats and evenstats respectively:

http://docs.splunk.com/Documentation/Splunk/5.0.3/SearchReference/Stats
http://docs.splunk.com/Documentation/Splunk/5.0.3/SearchReference/Eventstats

MattLingwood
Engager

As a Splunk cloud user I have no control over the limits.conf file. Would you recommend which is better/More accurate by default? eventstats or stats?
Thanks

0 Karma

hliakathali_spl
Splunk Employee
Splunk Employee

If you are using Splunk Cloud and want to change this limit, file a Support ticket.

http://docs.splunk.com/Documentation/Splunk/5.0.3/SearchReference/Eventstats

0 Karma
Get Updates on the Splunk Community!

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...