Deployment Architecture

How to check Splunk components connectivity in a distributed environment?

namrithadeepak
Path Finder

Hi,

How do I check the connectivity between:

(All of them are unix)

  • Search Head -> Indexer
  • Indexer -> Forwarder
  • Deployment server -> Deployment Client

What string should I look for in logs?
Since they are all unix machines, can I check the connectivity using a command or any other way?

Thanks,
Deepak

0 Karma
1 Solution

mattymo
Splunk Employee
Splunk Employee

There are a few ways to check connectivity between these elements.

Search Head to Indexer:

Telnet on port 8089 between the SH <> IDX

You can also check the Search Peer status from the Search Head by navigating to Settings > Distributed Search > Search Peers. It will depict the IDX statuses.

Forwarder to Indexer:

Telnet from fwd to IDX on port 9997 (or whatever forward port you set in outputs.conf)

From the forwarder you can also grep $SPLUNK_HOME/var/log/splunk/splunkd.log for TcpOutputProc ( tail -100 splunkd.log | grep TcpOutputProcto check if you are connecting to the indexers

From the indexer perspective you can search index=_internal sourcetype=splunkd tcpin_connectionsand confirm you see your forwarder

DS to Deployment Client:

Telnet on port 8089 from DS to Client and vice versa.
Check _internal logs for index=_internal sourcetype=splunkd deploymentClient to confirm phone home.

View solution in original post

mattymo
Splunk Employee
Splunk Employee

There are a few ways to check connectivity between these elements.

Search Head to Indexer:

Telnet on port 8089 between the SH <> IDX

You can also check the Search Peer status from the Search Head by navigating to Settings > Distributed Search > Search Peers. It will depict the IDX statuses.

Forwarder to Indexer:

Telnet from fwd to IDX on port 9997 (or whatever forward port you set in outputs.conf)

From the forwarder you can also grep $SPLUNK_HOME/var/log/splunk/splunkd.log for TcpOutputProc ( tail -100 splunkd.log | grep TcpOutputProcto check if you are connecting to the indexers

From the indexer perspective you can search index=_internal sourcetype=splunkd tcpin_connectionsand confirm you see your forwarder

DS to Deployment Client:

Telnet on port 8089 from DS to Client and vice versa.
Check _internal logs for index=_internal sourcetype=splunkd deploymentClient to confirm phone home.

View solution in original post

.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!