Pivot and Stats and disappearing make my data disappear

[stephenho]

Hi All,

I'm playing around with data models at the moment and I came across this strange issue. This is similar to my query

Base search

| pivot Data_Model Outbound max(value) AS "value" SPLITROW hostname AS "hostname" SPLITROW _time AS "_time" PERIOD minute SPLITROW group AS "group" SPLITROW metric AS "metric" FILTER metric is *myfilter* SORT 100 hostname

Post Process

|   streamstats current=t global=f window=2 earliest(value) as curr, latest(value) as next, earliest(_time) as currTime, latest(_time) as nextTime by hostname, metric, group 
| eval timeDiff=nextTime-currTime | eval curr=if(timeDiff>600,NULL,curr)
| eval curr=if(next<curr, NULL, curr)
| eval delta=next-curr
| eval Gigabits=(delta*8/1000/1000/1000)
| eval Gigabits=if(Gigabits > 10000, NULL, Gigabits) 
| eval Gigabits_per_second=Gigabits/timeDiff 
| stats max(Gigabits_per_second) by group

So there seems to be a problem with using the stats command in this way. Everytime I run it, I initially get 5 rows returned, but as the search progresses, the search rows drop. So, I'll start with 10 rows, and then it will drop to 5 rows, and by the end of the search, I will only have 1 row.

Replacing the stats command with a

table Gigabits_per_second, group

I'm thought the problem might be with the streamstats command but as I can get the correct values when using a table command, seems to rule that out. Anyone seen this issue before?

Thanks

Steve

[Lucas_K · Accepted solution]

There is some sort of interaction from the pivots "sort 100" and that last stats command. Putting sort 0 will fix it but someone with a better understanding of the search pipeline can hopefully explain what is occurring.