Splunk removes all items from chart apart from OTHER

[bowesmana]

I have a timechart that shows

timechart span=5s fixedrange=f limit=20 perc90(time_taken) by request

over 70 minutes. Used for performance testing.

It's in a real time dashboard and it shows perfectly well. However, I redraw the complete dashboard to cover the 70 minutes at the end of a test and while the chart fills in, just before the end of drawing all events, all the lines in the chart disappear, leaving only the metrics for OTHER...???

I have opened up the chart in its own search and it will do the same

However, this does not always happen for different runs of the same test, but appears consistent with some test runs.

There are around 18 million events.
I changed the limit= to 15, then 5, but still the same problem. I then changed the limit back to 20 and the span from 5s to 1m and it works.

I guess it's a limits thing, but the odd thing is that it never does this for a similar timechart that has a limit=28 on the same data, but is just doing the same timechart, but instead of perc90, is doing

count by request

[samjenk_2]

I'm having a similar issue on a search head running 6.6.4, but not on 6.4.7. Can you report your version of Splunk you're using?

When searching over large data sets (2.2 million events) and using a | timechart count by <fieldname> clause, 6.4.7 returns a table and chart of the top ten values of <fieldname> plus the rest aggregated as OTHER. This is the expected behavior.

Running the same search, Splunk 6.6.4 proceeds normally, graphing the top 10 + OTHER, until just before the search finalizes. When the search finalizes, all of the columns are aggregated into OTHER in both the Statistics and Visualization tabs. This is not expected, and is repeatable on searches returning more than 2.2 million events. Both of these search heads are searching over the same data. When timechart is invoked on 6.6.4 with useother=f, the search returns no results. Fore smaller data sets (fewer than 2.2 million events), 6.6.4 behaves as 6.4.7 does.

[cmerriman]

70 minutes * 5 seconds is only 350 data points per line, so that shouldn't be an issue.
http://docs.splunk.com/Documentation/Splunk/6.6.3/Viz/ChartDisplayissues

have you tried to set useother=f

[bowesmana]

Not tried useother, but I want other. Odd thing is that it can display a 75 minute rt dashboard and when redrawing the dashboard as last 70, it shows the count chart correctly with 28 items at 5s, but not the percentile chat with 20 items at 5s

[DalJeanis]

@cmerriman - there are 12 5-second intervals in a minute. 70*12=840 >800. Limit for 5 seconds would be 4000 seconds, about 66 minutes.

[cmerriman]

good catch @DalJeanis, multiplied when i should have divided....i thought that sounded wrong, but couldn't put my finger on it....