All Apps and Add-ons

How to consume messages from kafka topic using splunk installed in gateway nodes?

govindaraop
New Member

Hi,

I am looking for help here.
I want to consume messages from kafka topics from the universal forwarder installed in my gateway nodes. Please help, how can I achieve this? I don't want Splunk to directly connect to my kafka brokers and consume the messages.

Labels (1)
Tags (1)
0 Karma

fman82
Explorer

You could consume data from Kafka via a Splunk search without indexing the data by proxying through Lenses.io (It's a secure access layer for Apache Kafka which avoids you have to point directly to Kafka brokers and managing ACLs). See: https://lenses.io/blog/2020/05/new-kafka-governance-approval-flows-app-topology/

alt text

0 Karma

hgehrts_splunk
Splunk Employee
Splunk Employee

Hi! There is a blog entry on how to ingest data from kafka at scale:
https://www.splunk.com/blog/2016/10/31/splunking-kafka-at-scale.html
It users the Splunk HTTP Event Collector. Is this maybe an option you are looking for?

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!