topic Can you help me with an Issue building a Splunk cluster? in Deployment Architecture

Can you help me with an Issue building a Splunk cluster?

a238574 — Thu, 28 Feb 2019 19:46:35 GMT

I manage a couple of small Splunk clusters, and for the 1st time, I need to build one form scratch. I am testing in our sandbox environment, but when I bring the cluster up, I end up with index issues that can't seem to be resolved.

cannot fix up search factor as bucket is not serviceable
Cannot fix search count as the bucket hasn't rolled yet.

The above messages show up for every bucket in the _audit and _internal indexes. The build is a fairly simple one: 2 indexer peers, 1 master, and 1 dedicated search head

1- It's RHEL based
2- Install the rpm, 7.0.3 is the version I am playing with
3 - Set the firewall rules to allow the traffic
4 - /opt/splunk/bin/splunk enable boot-start -user root --accept-license
5 - /opt/splunk/bin/splunk start --accept-license

For the master I run - /opt/splunk/bin/splunk edit cluster-config -mode master -replication_factor 2 -search_factor 2 -secret xxx -cluster_label test

For the indexer peers - /opt/splunk/bin/splunk edit cluster-config -mode slave -master_uri https://xx.xxx.xx.xx:8089 -replication_port 9887 -secret xxx

For the Search head - /opt/splunk/bin/splunk edit cluster-config -mode searchhead -master_uri https://xx.x.xxx.xx:8089 -secret xxx

Restart Splunk on the master, then configure the other nodes and restart Splunk.

Not sure what I am missing or doing wrong.

Re: Can you help me with an Issue building a Splunk cluster?

MuS — Thu, 28 Feb 2019 22:52:21 GMT

You are building an index cluster with 1 node and configure replication_factor=2 , this cannot work. You would need at least 2 nodes to make it work.

cheers, MuS

Re: Can you help me with an Issue building a Splunk cluster?

a238574 — Fri, 01 Mar 2019 13:07:47 GMT

Its got 4 nodes 1 for each function 2 indexer peers, 1 master, and 1 dedicated search head

Re: Can you help me with an Issue building a Splunk cluster?

a238574 — Fri, 01 Mar 2019 13:10:46 GMT

The steps are the same for each node except for the cluster config command

Re: Can you help me with an Issue building a Splunk cluster?

a238574 — Thu, 07 Mar 2019 16:17:43 GMT

Found the issue. I had copied the rhel firewall rules from another splunk env that had modified the replication port from 9887 to 8080. Once I fixed the rules the index issues were automatically fixed.