Getting Data In

How to index data which is forwarded to DNS?

saifuddin9122
Path Finder

Hello

I am using DNS lists for load balancing. I am pointing my forwarders to send data to my DNS, but I was wondering how can an indexer listen for data which is being forwarded to DNS?

I searched for document, but I could not find it, so can anyone please let me know how I can solve it?

Is it simply by enabling the listening port on my indexers?
or do I need any connection between my DNS and indexers?

thanks in advance

0 Karma

Richfez
SplunkTrust
SplunkTrust

It may be possible that what you are looking for is the Splunk App for Stream. Stream uses the underlying packet capturing mechanisms on the various platforms to capture data off the wire and send it in via the Forwarder.

The topic is a bit lengthy to get into the solution here, I'd recommend reading and following the extensive documentation in the app and its areas themselves.

The Splunk App for Stream.
How to Install the Splunk App for Stream.
One of the many pages on how to Easily Set Up the Splunk App for Stream. "Easily" being relative, you know.

0 Karma

ddrillic
Ultra Champion

Please also keep in mind that Splunk app for Stream is a Splunk-supported free app - supported and free!

0 Karma

Richfez
SplunkTrust
SplunkTrust

Huh. Somehow I didn't see any of the other comments against the original question. This may not be your solution. I'll leave it for now just in case, but no worries if it isn't.

0 Karma

somesoni2
Revered Legend

You need to setup Indexer to receive data onto a port (e.g. 9997). Then you need to configure your DNS LB to forward oncoming traffic to that port (9997) on the Indexers, So here is how it'll look like

                To DNS LB                       Forward to 
Forwarders------------------> DNS LB --------------------> Indexers (receiving on 9997)
               on some port                Indexers on 
                say 9997                          port 9997
0 Karma

saifuddin9122
Path Finder

Hello

i have followed the way u specified but i could not see any events in indexer.
on my forwarder i am getting a message as Forwarding to indexer group default-autolb-group blocked for 100 seconds.

here is my outputs.conf

[tcpout]
defaultGroup = default-autolb-group

[tcpout:default-autolb-group]
server = splunk-dns-test.XXXX:9997

[tcpout-server://splunk-dns-test.test-XXXX:9997]

and on my indexer i enabled listening

can you please help me

thanks

0 Karma

Richfez
SplunkTrust
SplunkTrust

Can you confirm with tcpdump (or wireshark depending on your OS) that the packets are being seen on your interface on the Splunk server?

0 Karma

ShaneNewman
Motivator

This is not recommended. This was being done at a previous company I consulted for and resulted in terrible performance, upon talking to a few at Splunk we discovered that the load balancers were breaking up the stream from the UF. It is much better to allow the SUFs to load balance for you - as they were designed.

0 Karma

saifuddin9122
Path Finder

SUFs do you mean static list load balancing??

can you please provide me some clear point what your are talking about LBS.

0 Karma

ShaneNewman
Motivator

Yes, the static list found in the outputs.conf.

Load balancing

0 Karma

saifuddin9122
Path Finder

ok thanks for clarification !!!

working on both as a part of testing

0 Karma
Get Updates on the Splunk Community!

Splunk Education - Fast Start Program!

Welcome to Splunk Education! Splunk training programs are designed to enable you to get started quickly and ...

Five Subtly Different Ways of Adding Manual Instrumentation in Java

You can find the code of this example on GitHub here. Please feel free to star the repository to keep in ...

New Splunk APM Enhancements Help Troubleshoot Your MySQL and NoSQL Databases Faster

Splunk Observability has two new enhancements to make it quicker and easier to troubleshoot slow or frequently ...