Dashboards & Visualizations

Chart Overlay and Different Graph Type

aaronkorn
Splunk Employee
Splunk Employee

Hello,

We are working on a dashboard panel right now that is going to show overall CPU utilization for user/system but want to overlay 2 processes that we want to monitor for utilization as well. The data is coming back fine but ideally we would like the User/System graph to be one graph ie. stacked area and the process utilization to be in the form of a line chart. Any ideas?

Here is the search we are using:
index=os source=cpu host=x38dev10 | multikv fields | timechart span=60s max(pctUser) as User, avg(pctSystem) as System by host | appendcols [search index="os" sourcetype="ps" host=x38dev10 | multikv fields pctCPU, COMMAND, USER | eval process=USER."/".COMMAND | search USER=tomcat OR redis | timechart span=60s avg(pctCPU) by process useother="f"]

jonuwz
Influencer

Here's an example :

<?xml version="1.0" encoding="utf-8"?>
<dashboard>
  <label>SuperChart</label>
  <row>
    <chart>
      <searchName>SuperChartSearch</searchName>
      <title>SuperChart/title>

      <!-- set up alternative palatte for stacked chart, transparancy makes the line chart more visible -->
      <option name="charting.transparentBrushPalette">solidFill</option>
      <option name="charting.transparentBrushPalette.colorPalette">@colorPalette</option>
      <option name="charting.transparentBrushPalette.alpha">0.7</option>

      <!-- set chart to use columns 0,1,2,3 where @data is the original "cube", and 0 is the time-->
      <option name="charting.data1">view</option>
      <option name="charting.data1.table">@data</option>
      <option name="charting.data1.columns">[0,1,2,3]</option>
      <option name="charting.chart.data">@data1</option>

      <!-- set chart2 to use columns 0,4,5, where 0 is time, 4 + 5 are data -->
      <option name="charting.data2">view</option>
      <option name="charting.data2.table">@data</option>
      <option name="charting.data2.columns">[0,4,5]</option>
      <option name="charting.chart2.data">@data2</option>

      <!-- splunk doesnt suppory 2 Y axis, so we have to clone -->
      <option name="charting.axisY2">#axisY</option>                 <!-- clone the Yaxis into axisY2 -->
      <option name="charting.axisLabelsY2">#axisLabelsY</option>     <!-- clone the Yaxis Labels -->
      <option name="charting.axisLabelsY2.axis">@axisY2</option>     <!-- associate the new Y2 axis-labels with the Y2 axis -->

      <!--create the 1st chart -->
      <option name="charting.chart">column</option>
      <option name="charting.chart.columnBrushPalette">@transparentBrushPalette</option>
      <option name="charting.chart.stackMode">stacked</option>
      <option name="charting.chart.useAbsoluteSpacing">True</option>
      <option name="charting.chart.columnSpacing">20</option>
      <option name="charting.chart.columnAlignment">.5</option>
      <option name="charting.axisTitleY.text">CPU</option>
      <option name="charting.axisLabelsY.placement">left</option>

      <!--create the 2nd chart -->
      <option name="charting.chart2">line</option>
      <option name="charting.chart2.nullValueMode">zero</option>
      <option name="charting.chart2.axisY">@axisY2</option>          <!-- the Yaxis of chart 2 is a link to charting.axisY2 -->
      <option name="charting.axisTitleY2">axisTitle</option>
      <option name="charting.axisTitleY2.text">Memory</option>
      <option name="charting.axisLabelsY2.placement">right</option>

      <!-- throw the structure to layout -->
      <option name="charting.layout.charts">[@chart,@chart2]</option>
      <option name="charting.layout.axisLabels">[@axisLabelsX,@axisLabelsY,@axisLabelsY2]</option>
      <option name="charting.layout.axisTitles">[@axisTitleX,@axisTitleY,@axisTitleY2]</option>
    </chart>
  </row>
</dashboard>

This assumes saved search called "SuperChartSearch" returns data like this :

... | bin _time span=1h | stats avg(User) as User avg(Sys) as Sys avg(Wait) as Wait avg(mem_free) as mem_free avg(swap_free) as swap by _time

alt text

And will display like this :

alt text

Update

If you want a realtime chart, replace the <searchName>.....</searchName>

with (for example)

  <searchString>index=_internal | bin _time span=1m | stats min(eps) avg(eps) max(eps) sum(eps) perc95(eps) by _time</searchString>
    <earliestTime>rt-15m</earliestTime>
    <latestTime>rt</latestTime>

lguinn2
Legend

This is documented as a migration issue in moving from Splunk 5 to Splunk 6. In Splunk 6, simple XML uses JavaScript, not Flash. The technique used here requires Flash.

http://docs.splunk.com/Documentation/Splunk/6.0.1/AdvancedDev/Migration#Flash_charting_not_available...

spammenot66
Contributor

@iguinn are you saying this won't work anymore in the latest 6.3? If so how would we replicate this in 6.3? I know this is still possible because I've seen a couple folks at the conf use a similiar chart

0 Karma

watsm10
Communicator

This works brilliantly. Only problem I have is that when I try to generate a PDF, it doesn't render correctly. Anyone else having this issue?

0 Karma

jonuwz
Influencer

that is surprising : )

feel free to mark the answer as accepted if its working for you.

0 Karma

aaronkorn
Splunk Employee
Splunk Employee

worked like a charm. Thanks!

0 Karma

jonuwz
Influencer

try something like this - I have no idea if it works - I dont have the app for *nix :

index=os host=x38dev10 source=cpu OR sourcetype=ps
| multikv
| search USER=tomcat OR USER=redis OR source=cpu
| eval metric=(if(sourcetype=="ps",USER."/".COMMAND,"CPU"))
| timechart max(pctUser) as User, avg(pctSystem) as System avg(pctCPU) as proc by metric

0 Karma

aaronkorn
Splunk Employee
Splunk Employee

Thanks for the update. For some reason, I think how I have the subsearch setup with appendcols does not like realtime searches. Do I have the search setup properly for this or is there a better way to go about pulling in both the searches?

Thanks!

0 Karma

jonuwz
Influencer

updated answer

0 Karma

aaronkorn
Splunk Employee
Splunk Employee

This is excellent! Thank you very much for the help!

Last question though.. How would I make this into a realtime search without having to use a saved search? How would I specify in the XML to make it a realtime search if I use this as the search?

index=os source=cpu host=x38dev10 | multikv fields | timechart span=60s max(pctUser) as User, avg(pctSystem) as System by host | appendcols [search index="os" sourcetype="ps" host=x38dev10 | multikv fields pctCPU, COMMAND, USER | eval process=USER."/".COMMAND | search USER=tomcat OR redis | timechart span=60s avg(pctCPU) by process useother="f"]

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...