Chart of multiple data series

[lukeh]

Hi,

I am trying to craft a search to chart bandwidth utilization across multiple switches and multiple interfaces, however I have run into a few issues:

1/ I can chart one switch with multiple interfaces, as per the example below... however if I want to add some additional fields to chart (eg. capacity, engineering_limit, & augmentation_limit) they are plotted multiple times per metric 😞
ie. those additional fields should only be plotted once, not for each interface (aka metric).

2/ Following on from above, I actually want to chart multiple switches with multiple interfaces, however timechart cannot accept multiple split-by-clause's 😞
I have tried using xyseries (as per http://docs.splunk.com/Documentation/Splunk/latest/Search/Chartmultipledataseries ) but I can't figure out how to use it with streamstats and at the same time plot the delta for the interface usage with a per_second calculation.

3/ The interfaces should actually be stacked, but this doesn't seem to work when using timechart with a split-by-clause 😞

I am running on Splunk 5.0.1. Any help is greatly appreciated 🙂

index=mediacap ( hostname="cht-cdn6506-1" ) ( metric="ifInOctets_port-channel2" OR metric="ifInOctets_port-channel3" ) 
| streamstats current=t global=f window=2 earliest(value) as curr latest(value) as next by metric 
| eval delta=next-curr 
| eval inkilobits=(delta*8/1000) 
| eval capacity=(( 47.3 )*1024*1024) 
| eval eng=(capacity*(( 90 )/100)) 
| eval aug=(capacity*(( 70 )/100)) 
| timechart span=5m per_second(inkilobits) as in_kbps
max(capacity) as capacity
max(eng) as eng
max(aug) as aug
by metric

Thanks in advance,

Luke 🙂

[jonuwz · Accepted solution]

For the 1st part of the question, move this :

| eval capacity=(( 47.3 )*1024*1024) 
| eval eng=(capacity*(( 90 )/100)) 
| eval aug=(capacity*(( 70 )/100)) 

After the timechart command, and remove theis from the timechart command :

max(capacity) as capacity
max(eng) as eng
max(aug) as aug

How exactly do you want the graph to look in 2 + 3 ?

For each time slice, have multiple columns representing the hosts, with each column having stacked interfaces ?

Not sure thats possible out the box.

Also, You'd need 2 Y axis on your graph to plot the 'capacity','aug' fields seperately to the stacked columns. See here

A quick way to have multiple switch/interfaces would be to start the search like this :

index=mediacap (metric="ifInOctets_port-channel2" OR metric="ifInOctets_port-channel3" ) | eval metric=hostname.":".metric | ...

[lukeh]

I was able to use 'addtotals' to get the combined total 🙂

Thanks again for your help Jon! You taught me a couple of things and put me on the right track to Ninjaville 😛

[lukeh]

Thanks for your help Jon! Moving the evals for capacity, eng, & aug worked a treat... and combining the hostname & metric field into one is brilliant! We are almost there now 🙂

However, we are still using timechart with a split-by-clause on the new metric field... so can we stack them all together to show overall bandwidth usage for the port-channels across all relevant hostnames?

Here is the current screenshot: https://dl.dropbox.com/u/1193777/splunk-media-poc3.png

What we need is all of the cht* and ken* metrics stacked to show overall bandwidth usage 🙂

Thanks in advance,

Luke.