I have an accelerated data model where all events contain a duration field (ReqTot). In addition, some events include a field indicating an experiment that was applied to the given transaction. I want to timechart the exactperc99 of the duration field, grouped by the experiment values. The "no experiment" case is valid and should be represented in the results. However, tstats seems to drop some of the relevant data for events that don't have an experiment field, so I only get results for the experiment cases.
I've tried this several ways, but this is the most straightforward version of the query, as I'd hope it would work.
| tstats exactperc99(sla.ReqTot) from datamodel=Datamodel where (nodename=sla) groupby _time sla.Experiments prestats=true
| timechart span=5m exactperc99(sla.ReqTot) by sla.Experiments
When I run this query, tstats seems to discard the values of ReqTot for any events where Experiments was not present. As a result, I only see correct results for the experiment cases, but no results at all for the case with no experiment.
If I run in verbose mode I can see that the events still appear to be present, but they don't have values for ReqTot. A similar query that determines a count by Experiments value produces correct results, because the value of ReqTot doesn't matter. And if I remove Experiments from the groupby clause, I get a correct result for all events; in verbose mode, all events show a proper histogram of values for ReqTot. The inclusion of the missing field seems to cause the values of all other fields to be excluded.
Why is tstats discarding my data, and is there any way to make this query work correctly?
... View more