I have been using kafka for almost 2 years now. I never had an issue. Recently, I believe the volume of data increased and I started having the erroor beloow
com.splunk.hecclient.HecException: All channels have back pressure
at com.splunk.hecclient.LoadBalancer.send(LoadBalancer.java:62)
at com.splunk.hecclient.Hec.send(Hec.java:233)
at com.splunk.kafka.connect.SplunkSinkTask.send(SplunkSinkTask.java:257)
at com.splunk.kafka.connect.SplunkSinkTask.handleFailedBatches(SplunkSinkTask.java:127)
at com.splunk.kafka.connect.SplunkSinkTask.put(SplunkSinkTask.java:62)
at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:524)
at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:302)
at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:205)
at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:173)
at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:170)
at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:214)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
My config:
curl -s localhost:8083/connectors -X POST -H "Content-Type: application/json" -d '{
"name": "forwarder",
"config": {
"connector.class": "com.splunk.kafka.connect.SplunkSinkConnector",
"tasks.max": "5",
"splunk.indexes": "splunk--index",
"splunk.sources": "http:kafka_sip",
"topics":"topic",
"splunk.hec.uri": "splunkforwarder:8083",
"splunk.hec.token": "token",
"splunk.hec.raw": "false",
"splunk.hec.ack.enabled":"false",
"splunk.hec.ssl.validate.certs": "false",
"splunk.hec.max.batch.size": "100",
"splunk.hec.json.event.formatted": "true"
}
}'
Would you please advise why this started happening now?
