Dashboards & Visualizations

I have 1500 events and 1624343 statistics values and but all the data points are not available in the visualization(line chart).How can I include all the data points?

cp_christy
New Member

There are 1500 events and 1624343 statistics point when I have given a time span of 1msec.In the visualisation entire time period is not there.How can i include all data points?

Tags (1)
0 Karma

Richfez
SplunkTrust
SplunkTrust

This kind of question arises from time to time.

Math may be the problem with the answer you would like. Let me try a more thorough explanation of what I think mmodestino and Iradics are trying to say:

You say 1500 events - this wouldn't be an awful number of data points to plot on your X axis on a decent monitor, if you were plotting one or a few items per event. I think even at 1500 events you might be better off with ... | timechart span=1s avg(VAL) AS Average, min(VAL) as Minimum, max(VAL) as Maximum to show this, but either way works.

Unfortunately you have 1.6 million data points, or something like 1083 actual data points per event. So even if those values were all unique, to plot them would still require more vertical data points than a 1080p monitor has. They would fit on a 4k monitor, but it wouldn't be comfortable and at its best it would be a semi-solid block of pixels covering half the screen.

Together, that means that even a scatter chart won't do this well. Even if you crank down the data points to individual pixels and up the limits a long ways (be sure to upvote Iradic's comment above if that's the case!), I don't think you'll be happy with the results. And to make sense of that, you'd have to have some relationship between the data points that you could rely on to help group things.

So, let's step back - what is it about this information you are trying to show? What are these values?

Would there be a way to summarize them in a meaningful manner? (As per mmodestino's answer above - if that's the case be SURE to upvote his comment, or I can even convert it to an answer for you and you can accept it instead since he got there first!)

Can they be manipulated to identify outliers and only display those? NOTE: detecting outliers isn't with the outliers command, but with anomalies. outliers removes them instead of finding them.

Could they be analyzed for trends across individual series and those plotted?

Can they be compared to their own history and only certain sized variants displayed?

Could you use the commands in the Machine Learning Tool Kit and its Custom tools and visualizations to display this?

So, unelss I'm missing some critical piece of information or if you got an extra few digits in the number of measurements, the answer to your question isn't a simple "Sure, just do this and that and BAM you have 1.6 million dots on a graph that make sense." You can possibly get them there, they just may not make sense.

0 Karma

Richfez
SplunkTrust
SplunkTrust

cp_christy,

Can we help you with some of the alternative methods of displaying this? Did the explanation make sense as to why this can't work at full resolution?

If this did indeed explain adequately, could you mark the question as accepted to help other folks who may have a similar question?

Thanks,
Rich

0 Karma

lradics
Path Finder

Is it an issue of data truncation? I've had this page of the documentation bookmarked for a while for data truncation issues:
https://docs.splunk.com/Documentation/Splunk/6.6.1/Viz/ChartDisplayissues

0 Karma

mattymo
Splunk Employee
Splunk Employee

That's exactly what it is. Unless you are displaying a timechart in Times Square, no monitor is going to draw 1 Million data points..also I question what you are trying to communicate to your audience.

- MattyMo
0 Karma

mattymo
Splunk Employee
Splunk Employee

Generally, the answer is aggregation via statistical commands, like | timechart span=1s avg(dataPoints) AS dataPoints by host. No human can consume that many data points on a screen....What are these data points trying to communicate? Can you share a sample so we can assist you further?

http://docs.splunk.com/Documentation/Splunk/6.6.1/SearchReference/CommonStatsFunctions

- MattyMo
0 Karma
Get Updates on the Splunk Community!

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...

Security Highlights | January 2023 Newsletter

January 2023 Splunk Security Essentials (SSE) 3.7.0 ReleaseThe free Splunk Security Essentials (SSE) 3.7.0 app ...

Platform Highlights | January 2023 Newsletter

 January 2023Peace on Earth and Peace of Mind With Business ResilienceAll organizations can start the new year ...