Splunk for JMX: Operating System -> CPU Usage returns 0 results

[aware74]

I tried to diagnose the search, and found that the following search:

index=jmx sourcetype=jmx mbean_domain="java.lang" mbean_property_type="OperatingSystem" OR "Runtime" host="*" jvmDescription="*"  | stats  max(jvmUptime) as uptime max(cpuTime) as cputime max(processors) as cpus by jvmDescription,_time  | streamstats  current=f global=f window=1 first(uptime) as next_uptime first(cputime) as next_cputime by jvmDescription  | eval  d_uptime=uptime-next_uptime  | eval  d_cputime=cputime-next_cputime

Returned the following kind of gaps:

     _time                 jvmDescription        uptime     cputime         cpus     next_cputime     next_uptime
11/13/12 6:38:19.850 PM    qa2-zk-01-n1        1033382707
11/13/12 6:38:19.856 PM    qa2-zk-01-n1                     440610000000     2                        1033382707
11/13/12 6:39:19.822 PM    qa2-zk-01-n1        1033442680                             440610000000
11/13/12 6:39:19.828 PM    qa2-zk-01-n1                     440640000000     2                        1033442680
11/13/12 6:40:19.824 PM    qa2-zk-01-n1        1033502685                             440640000000
11/13/12 6:40:19.834 PM    qa2-zk-01-n1                     440690000000     2                        1033502685
11/13/12 6:41:19.797 PM    qa2-zk-01-n1        1033562654                             440690000000
11/13/12 6:41:19.802 PM    qa2-zk-01-n1                     440730000000     2                        1033562654
11/13/12 6:42:19.807 PM    qa2-zk-01-n1        1033622669                             440730000000
11/13/12 6:42:19.813 PM    qa2-zk-01-n1                     440760000000     2                        1033622669

Is this because I am using the prependDate attribute on my formatter?

[Damien_Dallimor · Accepted solution]

That is most likely the cause.
Try it without the prependDate setting , that optional property is there should you require a custom date format for you MBean output for whatever reason.

..or, take off the milliseconds pattern from the format string ("SSS")

If you do require to use the prependDate option , then you can have a seperate JMX config file for these specific instances.

[aware74]

Thank you. I removed that attribute, restarted the Splunk universal forwarder on the server with the target JVM, and then the CPU Usage dashboard panel started to return results.