All Apps and Add-ons

How to send metrics data from kafka to Splunk metrics index

daniel_splunk
Splunk Employee
Splunk Employee

I'm able to send metrics data from kafka to Splunk event index, any idea how to send metrics data to metrics index as well?

0 Karma

daniel_splunk
Splunk Employee
Splunk Employee

For Kafka connect to ingest metrics data - we need to use collectd as documented here:

https://www.splunk.com/blog/2018/04/25/unleashing-data-ingestion-from-apache-kafka.html

Once collectd is installed, below is an example of a connector to send collectd metrics to a Splunk metrics index
The Splunk metrics index is optimized for ingesting and retrieving metrics. For more information, see the Metrics manual.

curl http://localhost:8083/connectors -X POST -H "Content-Type: application/json" -d '{
"name": "metrics-via-collectd",
"config": {
  "connector.class": "com.splunk.kafka.connect.SplunkSinkConnector",
  "tasks.max": "1",
  "topics":"collectd",
  "splunk.sourcetypes": "collectd_http",
  "splunk.indexes": "kafka_metrics",
  "splunk.hec.uri": "https://localhost:8088",
  "splunk.hec.token": "727ac06e-5150-4f0f-a85d-c7b070176a2c",
  "splunk.hec.ack.enabled" : "false",
  "splunk.hec.ack.poll.interval" : "20",
  "splunk.hec.ack.poll.threads" : "2",
  "splunk.hec.event.timeout" : "120",
  "splunk.hec.ssl.validate.certs": "false",
  "splunk.hec.raw" : "true",
  "splunk.hec.raw.line.breaker" : "####"
  }
}'

Then, you can use below sample data to send to kafka.

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic collectd
[{"values":[164.9196798931339196],"dstypes":["derive"],"dsnames":["value"],"time":1505356687.894,"interval":10.000,"host":"collectd","plugin":"protocols","plugin_instance":"IpExt","type":"protocol_counter","type_instance":"InOctets"}]

Then, check from Splunk metrics index by using below command to confirm metrics data is indexed correctly.

| mcatalog values(metric_name) WHERE index=kafka_metrics AND metric_name=protocols.protocol_counter.InOctets.value
0 Karma
Get Updates on the Splunk Community!

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Announcing the General Availability of Splunk Enterprise Security 8.1!

We are pleased to announce the general availability of Splunk Enterprise Security 8.1. Splunk becomes the only ...

Developer Spotlight with William Searle

The Splunk Guy: A Developer’s Path from Web to Cloud William is a Splunk Professional Services Consultant with ...