Hi All,
I am setting up splunk cluster environment. IN which i have 1 deployer and cluster master and 4 indexer and 3 search head.
after setting up cluster now i am setting monitoring console on deployer . Unfortunately i am not able to see the search head mambers in destributed search. i was able to see all 4 indexers but not search heads.
Can you please suggest what was the issue.?
Please let me know any input required
Did you add the search heads as peers to the MC? On the MC/Deployer, go to Settings->Distributed Search and click the Add New link. Enter the information for a SH. Repeat for the other SHs.
Hi @btshivanand,
There are two checks:
Ciao.
Giuseppe
Thanks for your reply... Can you please tell me how i need forward my internal logs to indexer...
i builded search head cluster and then i joined them master with below command.
./splunk edit cluster-config -mode searchhead -master_uri https://8089 -secret XXXX
Hi @btshivanand,
each Splunk server (not the Indexers obviously) should send its internal logs to the Indexers,
The way is the one described by @richgalloway.
Ciao.
Giuseppe
Thanks for you kind answer.I was trying set up the one of search head which is not clsuster to send logs to indexer.i was not at succesfull.i created app and i defined output.conf as below..Can you susggest me is there any suggestion
# Turn off indexing on the search head
[indexAndForward]
index = false
[tcpout]
defaultGroup = my_search_peers
forwardedindex.filter.disable = true
indexAndForward = false
[tcpout:my_search_peers]
server = X:9997, X:9997, X:9997, X:9997
Hi @btshivanand,
you can do the same thing by UI [Settings -- Forwarding and Receiving -- Forward].
Ciao.
Giuseppe
I am getting below error after adding all the indexers
The TCP output processor has paused the data flow. Forwarding to host_dest= inside output group default-autolb-group from host_src=XXXX has been blocked for blocked_seconds=10. This can stall the data flow towards indexing and other network outputs. Review the receiving system's health in the Splunk Monitoring Console. It is probably not accepting data. Learn more.
Hi @btshivanand,
you have to forward the logs of all Splunk Servers except Indexers because they are already indexed.
Ciao.
Giuseppe
Thanks.. issue resolved...port 9997 was blocked from zone2 where our search head is running..
Thanks alot for the help
i See one more message.
Now skipping indexing of internal audit events, because the downstream queue is not accepting data. Will keep dropping events until data flow resumes. Review system health: ensure downstream indexing and/or forwarding are operating correctly.
Make the SHs forwarder their logs by adding an outputs.conf file to their configuration, like you would do for a universal forwarder.
Did you add the search heads as peers to the MC? On the MC/Deployer, go to Settings->Distributed Search and click the Add New link. Enter the information for a SH. Repeat for the other SHs.
Thanks alot i was able to add search head into the monitoring console