How to consume messages from kafka topic using splunk installed in gateway nodes?

[govindaraop]

Hi,

I am looking for help here.
I want to consume messages from kafka topics from the universal forwarder installed in my gateway nodes. Please help, how can I achieve this? I don't want Splunk to directly connect to my kafka brokers and consume the messages.

[fman82]

You could consume data from Kafka via a Splunk search without indexing the data by proxying through Lenses.io (It's a secure access layer for Apache Kafka which avoids you have to point directly to Kafka brokers and managing ACLs). See: https://lenses.io/blog/2020/05/new-kafka-governance-approval-flows-app-topology/

[hgehrts_splunk]

Hi! There is a blog entry on how to ingest data from kafka at scale:
https://www.splunk.com/blog/2016/10/31/splunking-kafka-at-scale.html
It users the Splunk HTTP Event Collector. Is this maybe an option you are looking for?