There are 1500 events and 1624343 statistics point when I have given a time span of 1msec.In the visualisation entire time period is not there.How can i include all data points?
This kind of question arises from time to time.
Math may be the problem with the answer you would like. Let me try a more thorough explanation of what I think mmodestino and Iradics are trying to say:
You say 1500 events - this wouldn't be an awful number of data points to plot on your X axis on a decent monitor, if you were plotting one or a few items per event. I think even at 1500 events you might be better off with ... | timechart span=1s avg(VAL) AS Average, min(VAL) as Minimum, max(VAL) as Maximum
to show this, but either way works.
Unfortunately you have 1.6 million data points, or something like 1083 actual data points per event. So even if those values were all unique, to plot them would still require more vertical data points than a 1080p monitor has. They would fit on a 4k monitor, but it wouldn't be comfortable and at its best it would be a semi-solid block of pixels covering half the screen.
Together, that means that even a scatter chart won't do this well. Even if you crank down the data points to individual pixels and up the limits a long ways (be sure to upvote Iradic's comment above if that's the case!), I don't think you'll be happy with the results. And to make sense of that, you'd have to have some relationship between the data points that you could rely on to help group things.
So, let's step back - what is it about this information you are trying to show? What are these values?
Would there be a way to summarize them in a meaningful manner? (As per mmodestino's answer above - if that's the case be SURE to upvote his comment, or I can even convert it to an answer for you and you can accept it instead since he got there first!)
Can they be manipulated to identify outliers and only display those? NOTE: detecting outliers isn't with the outliers
command, but with anomalies
. outliers
removes them instead of finding them.
Could they be analyzed for trends across individual series and those plotted?
Can they be compared to their own history and only certain sized variants displayed?
Could you use the commands in the Machine Learning Tool Kit and its Custom tools and visualizations to display this?
So, unelss I'm missing some critical piece of information or if you got an extra few digits in the number of measurements, the answer to your question isn't a simple "Sure, just do this and that and BAM you have 1.6 million dots on a graph that make sense." You can possibly get them there, they just may not make sense.
cp_christy,
Can we help you with some of the alternative methods of displaying this? Did the explanation make sense as to why this can't work at full resolution?
If this did indeed explain adequately, could you mark the question as accepted to help other folks who may have a similar question?
Thanks,
Rich
Is it an issue of data truncation? I've had this page of the documentation bookmarked for a while for data truncation issues:
https://docs.splunk.com/Documentation/Splunk/6.6.1/Viz/ChartDisplayissues
That's exactly what it is. Unless you are displaying a timechart in Times Square, no monitor is going to draw 1 Million data points..also I question what you are trying to communicate to your audience.
Generally, the answer is aggregation via statistical commands, like | timechart span=1s avg(dataPoints) AS dataPoints by host
. No human can consume that many data points on a screen....What are these data points trying to communicate? Can you share a sample so we can assist you further?
http://docs.splunk.com/Documentation/Splunk/6.6.1/SearchReference/CommonStatsFunctions