Datamodel slower than TSIDX

[btorresgil]

Working on my app, I have converted all of the TSIDX-based dashboards over to an accelerated Data model. So instead of using "|tstats FROM tsidxindex" everywhere, now I'm using either "|tstats FROM datamodel=mydatamodel" or "|pivot" to create all of the charts on the dashboards. The datamodel is accelerated and the accelerated index is 100% built.

However...

On the same exact servers with roughly the same amount of data, I've noticed that the dashboards take much longer to populate the graphs and tables when using the accelerated datamodel than they did when using just TSIDX summary indexes. On the order of 5-20 seconds longer for the datamodel. This makes navigating the dashboards feel slow and clunky as the charts and tables slowly populate over time. There is very little data in my test environment, less than 1 Gig.

On some servers the charts and tables will even hang around 50% loaded and won't complete even after 30 seconds of waiting. I'm not sure if this is the same issue or a different issue, because it only affects some servers while the slowness affects all of them. But the hanging issue also never affected the TSIDX summary indexes, only since I switched to datamodel.

[Marklar]

I'm not sure what's going on with the hanging issue, but I can respond to the perceived slowness. My #1 guess is the misconception that mixed-mode is "slowing things down" when running tstats on an accelerated datamodel.

Datamodel acceleration is transparent, meaning any data that has not been accelerated yet (always the case for live streaming data) will still be captured in queries by falling back to normal search for that data. Keep in mind we always process all summaries first, so they still get all the answers from accelerated data right away, then we fill in with search.

Old-school tscollect-style tstats never had this feature, and was always out of date by comparison. To match this old behavior (seemingly only for perception's sake), you can send the argument 'summariesonly=t' to your tstats commands that select from accelerated datamodels.

As for 100% completeness of the acceleration, this percentage does not include hot buckets, as they are never truly "finished" as live data can always stream in. 100% complete means you've accelerated all buckets within the acceleration range that are non-hot.

[btorresgil]

'summariesonly=t' helps. In testing page response times, tsidx takes 9 seconds, datamodel takes 15 seconds, datamodel with summariesonly takes 11 seconds. Is there any way to do summariesonly with the |pivot command instead of the |tstats command? Or do I have to convert all my |pivots to |tstats?