Multiple line charts representing a single field for multiple hosts

[AshimaE]

I have the log files of several hosts and wish to represent a single field CPU usage for each of them as a separate line chart.
Presently I have been able to create a chart for a single host as follows

index=abc sourcetype=def host=ghi earliest=-6h | table _time cpuUse

How could I do the same for all the hosts on a single/separate graph?
Thanks in advance.

[yuanliu]

Though most of the time it is desirable to have multiple hosts on the same chart, I do see occasional urge for individual charts - on a dashboard. (Especially when you have many hosts. Same goes with some other metrics.) In fact I had proposed this for my previous group who wanted such a separation-aggregation. I didn't implement this but here's my proposed solution: Compose the XML elements for each of the charts you want to produce using tstats, then incorporate the output into your dashboard source. Obviously this will have severe limitations if the number of hosts grows very fast and you'll have to update your dashboard constantly. But for many environments, this should suffice.

| tstats count by host where index=abc sourcetype=def
| eval Panel="  <row>
    <panel>
      <chart>
        <title>Chart for " + host + ": avg(cpuUse)</title>
        <search>
          <query>
            index=abc sourcetype=def host=" + host + " earliest=-6h
            |timechart avg(cpuUse)
          </query>
        </search>
      </chart>
    </panel>
  </row>"
| table Panel

Output would look like these:

<row> <panel> <chart> <title>Chart for host1: avg(cpuUse)</title> <search> <query> index=abc sourcetype=def host=host1 earliest=-6h |timechart avg(cpuUse) </query> </search> </chart> </panel> </row>
<row> <panel> <chart> <title>Chart for host2: avg(cpuUse)</title> <search> <query> index=abc sourcetype=def host=host2 earliest=-6h |timechart avg(cpuUse) </query> </search> </chart> </panel> </row>
<row> <panel> <chart> <title>Chart for host3: avg(cpuUse)</title> <search> <query> index=abc sourcetype=def host=host3 earliest=-6h |timechart avg(cpuUse) </query> </search> </chart> </panel> </row>

Note the output is "flattened" in terms of line breaks and indentations so you may want to employ some pretty-print for readability in the final dashboard source. (You can still use the output as is, just not as pretty.)

[woodcock]

I am not sure that I catch your drift but, to show all hosts in the same graph, do this:

index=abc sourcetype=def earliest=-6h | timechart avg(cpuUse) BY host

Or, maybe you mean to show each host in his own graph and have the graph panels allocate dynamically, then you can do this:

https://docs.splunk.com/Documentation/Splunk/6.6.1/Viz/VisualizationTrellis
But this feature is new to v6.6.1 which not everybody is running; if you need to do this in earlier versions, look at the multiplexer capability of SideView Utils:
http://sideviewapps.com/apps/sideview-utils/

[cmerriman]

something like this, perhaps:

index=abc sourcetype=def host=* earliest=-6h | table _time host cpuUse

or even to bucket per hour (or other span if wished):

index=abc sourcetype=def host=* earliest=-6h | timechart  limit=0 span=1h avg(cpuUse) as avgcpuUse by host