All Apps and Add-ons

Splunk Connect for Kafka: Previously stored configurations reload when Kafka Connect is restarted.

yhu_splunk
Splunk Employee
Splunk Employee

Hello, I need help on Kafka Connect.

I am using Kafka _2.12-1.1.1 and Splunk Connect for Kafka version 1.20. It's distributed mode but there is only 1 Kafka connect node.
I'm having a problem with managing configurations.

When I DELETE a connector, and then restart Kafka Connect, the old connectors might be reloaded.
And sometimes create a new connector with curl localhost:8083/connectors -X POST -H…. will failed (no error message, but not added successfully).

It seems the old connector configurations were stored somewhere and sometimes Kafka Connect will looking for those when restarted.

I am thinking if we can delete old configurations permanently before creating new connectors then the problem might be solved.

Labels (2)
0 Karma

ilya_resh
Engager

Not sure if answer is relevant anymore or is feasible in your environment, but anyway:
* Try deleting the topics that sc4k has created using kafka-topics.sh command

* try changing the topics that sc4k is using (in the connector-distributed.proprties)

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...