The first log entry in your example now has the correct format, whereas the second log entry is incorrect. If you were just providing the second log entry as a comparison then continue on, otherwise you have to delete the old template in your nagios configuration.
The nagios performance data is listed in the 'result' key/value pair, but as you are not using the CPU Nagios Plugin supported by Splunk for Nagios (ie. Check CPU Performance: http://exchange.nagios.org/directory/Plugins/System-Metrics/CPU-Usage-and-Load/Check-CPU-Performance/details ) the graph is not displaying.
You can change the relevant search in the dashboard (eg. NagiosLinuxPerformanceGraphs.xml) to make the graph display correctly:
From:
<param name="search">timechart span=5m avg(CpuSystem) as System,avg(CpuUser) as User,avg(CpuNice) as Nice,avg(CpuIowait) as Wait</param>
To:
<param name="search">search name="CPU Load" | rex field=_raw "5s=(? .*)%" | eval total=100 | timechart span=5m avg(CpuLoad) as CpuLoad, max(total) as Total</param>
Then, remove the following lines from the same "CPU Usage" group:
<param name="charting.chart.stackMode">stacked</param>
<param name="charting.seriesColors">[0xFF0000,0x0000FF,0x00FF00,0xFF00FF]</param>
<param name="charting.legend.masterLegend"></param>
Essentially, the new search extracts a new field called 'CpuLoad' and adds a new field called 'total' which is always 100. ie.
CpuLoad=43 -> created by the rex command which matches 43 from 5s=43%
After you update the xml in the dashboard, click on the Splunk logo at the top left in Splunk for Nagios to reload the dashboards, then click on the relevant dashboard and the graph should display correctly 🙂
All the best,
Luke 🙂
... View more