All Apps and Add-ons

Splunk Connect for Kafka: Previously stored configurations reload when Kafka Connect is restarted.

yhu_splunk
Splunk Employee
Splunk Employee

Hello, I need help on Kafka Connect.

I am using Kafka _2.12-1.1.1 and Splunk Connect for Kafka version 1.20. It's distributed mode but there is only 1 Kafka connect node.
I'm having a problem with managing configurations.

When I DELETE a connector, and then restart Kafka Connect, the old connectors might be reloaded.
And sometimes create a new connector with curl localhost:8083/connectors -X POST -H…. will failed (no error message, but not added successfully).

It seems the old connector configurations were stored somewhere and sometimes Kafka Connect will looking for those when restarted.

I am thinking if we can delete old configurations permanently before creating new connectors then the problem might be solved.

Labels (2)
0 Karma

ilya_resh
Engager

Not sure if answer is relevant anymore or is feasible in your environment, but anyway:
* Try deleting the topics that sc4k has created using kafka-topics.sh command

* try changing the topics that sc4k is using (in the connector-distributed.proprties)

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...