All Apps and Add-ons

Monitoring of Java Virtual Machines with JMX: Dump all attributes vs Dump specific attributes (Kafka)

aarontimko
Path Finder

Hi!

I'm using Kafka 0.9.
The Monitoring of Java Virtual Machines with JMX add-on is working well from a Universal Forwarder on a Kafka broker, but I can't figure out the syntax to grab more specific metrics.

Method 1: This works when I grab attributes for a mbean domain:

<mbean domain="kafka.server" properties="*" dumpAllAttributes="true"/>
<mbean domain="kafka.controller" properties="*" dumpAllAttributes="true"/>

Results of Method 1:

host=smdvsyskafka01,jvmDescription="kafka",pid="4714",mbean_domain=kafka.server,mbean_property_name=IsrShrinksPerSec,mbean_property_type=ReplicaManager,RateUnit=SECONDS,OneMinuteRate=0.0,EventType=shrinks,Count=0,FifteenMinuteRate=0.0,FiveMinuteRate=0.0,MeanRate=0.0

host=smdvsyskafka01,jvmDescription="kafka",pid="4714",mbean_domain=kafka.controller,mbean_property_name=ActiveControllerCount,mbean_property_type=KafkaController,Value=1

Method 2: grab specific attributes under specific types for a mbean domain:

<mbean domain="kafka.server" properties="type=ReplicaManager">
<attribute name="UnderReplicatedPartitions" outputname="UnderReplicatedPartitions"/>
<attribute name="IsrShrinksPerSec" outputname="IsrShrinksPerSec"/>
<attribute name="IsrExpandsPerSec" outputname="IsrExpandsPerSec"/>
</mbean>

<mbean domain="kafka.controller" properties="type=KafkaController">
<attribute name="ActiveControllerCount" outputname="ActiveControllerCount"/>
</mbean>

I get no results for Method 2; events continue to get retrieved and delivered for the java.lang metrics I also have in my config.xml, but I no longer get anything for the kafka.server and kafka.controller metrics.

And there are no parsing or stanza errors from jmx.py in splunkd.log on the Universal Forwarder.

Is there a simple syntax issue I'm missing?

0 Karma
1 Solution

Damien_Dallimor
Ultra Champion

You missed the "name" attribute from the properties and didn't declare the attributes correctly:

 <mbean domain="kafka.server" properties="type=ReplicaManager,name=IsrShrinksPerSec">
 <attribute name="RateUnit" outputname="rate_unit"/>
 </mbean>


 <mbean domain="kafka.controller" properties="type=KafkaController,name=ActiveControllerCount">
 <attribute name="Value" outputname="value"/>
 </mbean>

View solution in original post

Damien_Dallimor
Ultra Champion

You missed the "name" attribute from the properties and didn't declare the attributes correctly:

 <mbean domain="kafka.server" properties="type=ReplicaManager,name=IsrShrinksPerSec">
 <attribute name="RateUnit" outputname="rate_unit"/>
 </mbean>


 <mbean domain="kafka.controller" properties="type=KafkaController,name=ActiveControllerCount">
 <attribute name="Value" outputname="value"/>
 </mbean>

aarontimko
Path Finder

Thanks!!

With this syntax, I also tested I can get all properties under a particular property type as well.
This is so helpful.

<mbean domain="kafka.server" properties="type=ReplicaManager,name=*" dumpAllAttributes="true"/>
<mbean domain="kafka.server" properties="type=BrokerTopicMetrics,name=*" dumpAllAttributes="true"/>
<mbean domain="kafka.controller" properties="type=KafkaController,name=*" dumpAllAttributes="true"/>
0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...