Monitoring of Java Virtual Machines with JMX: Dump all attributes vs Dump specific attributes (Kafka)

[aarontimko]

Hi!

I'm using Kafka 0.9.
The Monitoring of Java Virtual Machines with JMX add-on is working well from a Universal Forwarder on a Kafka broker, but I can't figure out the syntax to grab more specific metrics.

Method 1: This works when I grab attributes for a mbean domain:

<mbean domain="kafka.server" properties="*" dumpAllAttributes="true"/>
<mbean domain="kafka.controller" properties="*" dumpAllAttributes="true"/>

Results of Method 1:

host=smdvsyskafka01,jvmDescription="kafka",pid="4714",mbean_domain=kafka.server,mbean_property_name=IsrShrinksPerSec,mbean_property_type=ReplicaManager,RateUnit=SECONDS,OneMinuteRate=0.0,EventType=shrinks,Count=0,FifteenMinuteRate=0.0,FiveMinuteRate=0.0,MeanRate=0.0

host=smdvsyskafka01,jvmDescription="kafka",pid="4714",mbean_domain=kafka.controller,mbean_property_name=ActiveControllerCount,mbean_property_type=KafkaController,Value=1

Method 2: grab specific attributes under specific types for a mbean domain:

<mbean domain="kafka.server" properties="type=ReplicaManager">
<attribute name="UnderReplicatedPartitions" outputname="UnderReplicatedPartitions"/>
<attribute name="IsrShrinksPerSec" outputname="IsrShrinksPerSec"/>
<attribute name="IsrExpandsPerSec" outputname="IsrExpandsPerSec"/>
</mbean>

<mbean domain="kafka.controller" properties="type=KafkaController">
<attribute name="ActiveControllerCount" outputname="ActiveControllerCount"/>
</mbean>

I get no results for Method 2; events continue to get retrieved and delivered for the java.lang metrics I also have in my config.xml, but I no longer get anything for the kafka.server and kafka.controller metrics.

And there are no parsing or stanza errors from jmx.py in splunkd.log on the Universal Forwarder.

Is there a simple syntax issue I'm missing?

[Damien_Dallimor · Accepted solution]

You missed the "name" attribute from the properties and didn't declare the attributes correctly:

 <mbean domain="kafka.server" properties="type=ReplicaManager,name=IsrShrinksPerSec">
 <attribute name="RateUnit" outputname="rate_unit"/>
 </mbean>


 <mbean domain="kafka.controller" properties="type=KafkaController,name=ActiveControllerCount">
 <attribute name="Value" outputname="value"/>
 </mbean>

[aarontimko]

Thanks!!

With this syntax, I also tested I can get all properties under a particular property type as well.
This is so helpful.

<mbean domain="kafka.server" properties="type=ReplicaManager,name=*" dumpAllAttributes="true"/>
<mbean domain="kafka.server" properties="type=BrokerTopicMetrics,name=*" dumpAllAttributes="true"/>
<mbean domain="kafka.controller" properties="type=KafkaController,name=*" dumpAllAttributes="true"/>