Deployment Architecture

Waiting for requisite number of peers to join the cluster - https://127.0.0.1:8089. Cluster has only 0 peers (waiting for 3 peers to join the cluster)

ashishlal82
Explorer

I am receiving this error on my CM. The peer nodes are not able to join CM

0 Karma

micahkemp
Champion

What is preventing them from joining the cluster? Are there network or host based firewalls blocking the traffic? Are the peer nodes properly configured?

We need more information to help.

0 Karma

ashishlal82
Explorer

On the indexer boxes I am receving 1069 error that splunkweb services are not running. You think that could be the issue?

0 Karma

micahkemp
Champion

splunkweb doesn't need to be running on the indexers, so that isn't necessarily pointing to the issue. Is splunkd running on the indexers?

0 Karma

ashishlal82
Explorer

yes Splunkd is running on the indexers. But splunk webservice is not.

0 Karma

micahkemp
Champion

Are the indexers configured to be a part of the cluster? What do you get for this command when run on an indexer:

splunk btool server list clustering --debug
0 Karma

ashishlal82
Explorer

[general]
serverName = Server-name
pass4SymmKey = $1$tVLM068URVKj

[queue]
maxSize = 0

[queue=indexQueue]
maxSize = 1GB

[queue=parsingQueue]
maxSize = 100MB

[queue=aggQueue]
maxSize = 100MB

SHOULD_LINEMERGE = false

[queue=auditQueue]
maxSize = 100MB

[queue=typingQueue]
maxSize = 10GB

[queue=tcpout_my_indexers]
maxSize = 100MB

[sslConfig]
sslPassword = $1$4h6Yj+pSDgCj
allowSslCompression=false

[lmpool:auto_generated_pool_download-trial]
description = auto_generated_pool_download-trial
quota = MAX
slaves = *
stack_id = download-trial

[lmpool:auto_generated_pool_forwarder]
description = auto_generated_pool_forwarder
quota = MAX
slaves = *
stack_id = forwarder

[lmpool:auto_generated_pool_free]
description = auto_generated_pool_free
quota = MAX
slaves = *
stack_id = free

[license]
master_uri = https://10.55.3.23:8089

[diskUsage]
minFreeSpace = 500
Looks ok to me

0 Karma

nickhills
Ultra Champion

Unless your post has been cropped, you have no cluster config there at all.
You should have a section like:

[clustering]
master_uri = xxxxx:xxx
pass4SymmKey = xxxx
...
...

How did you configure the peers, through the UI, or via the cli with something like
splunk edit cluster-config -mode slave -master_uri https://10.160.31.200:8089 -replication_port 9887 -secret your_key?
https://docs.splunk.com/Documentation/Splunk/7.0.1/Indexer/ConfigurepeerswithCLI

If my comment helps, please give it a thumbs up!

schose
Builder

search have a look at peer nodes

cat splunkd.log |grep -i error

or provide log

0 Karma

ashishlal82
Explorer

its a windows box

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...