Deployment Architecture
Provide Splunk Cloud feedback in this confidential UX survey by June 17
for a chance to win a $200 Amazon gift card!

Add second index cluster to search head

trevor_dunstan8
Explorer

Hi all,

Ill try and keep it short and to the point. 

We have a standalone search head that is currently connected to an index cluster with 4 peers. We would now like to connect a second 3 peer index cluster that is hosted in AWS.

When I add the AWS cluster master to the search head via Settings -> Indexer Clustering it actually fails to connect due to the below error:

Master has multisite enabled but the search head is missing the 'multisite' attribute'

but if I configure in the server.conf file and reboot, the AWS cluster master connects fine but the 3 peers do not appear as per below screenshot and I am not able to search the indexes.

   Peers.PNG

If I manually add the index peers under Settings -> Distributed Search -> New Search Peer, the peers add fine and I am able to search indexes in AWS as required. 

I need the peers to be discovered automatically by the search head via the cluster master as the AWS indexers are rebuilt on a regular basis.

Below is the server.conf on our search head

server.conf.PNGand I have been informed that autodiscovery is enabled on the AWS Cluster master.

I have logged a case with Splunk but thought I would try here as well.

Any information would be appreciated

Thanks

 

Trev

Labels (1)
0 Karma

trevor_dunstan8
Explorer

Issue turned out to be a DNS issue and our search head was not able to resolve DNS names for the indexers in AWS. As an interim solution we have updated the hosts file on the search head with the AWS pool of IP addresses and hostnames for the AWS indexers. Not elegant by any means but is temporary until DNS forwarders can be set up.

0 Karma

soutamo
SplunkTrust
SplunkTrust

Hi

we are using DNS names on all configurations and updated those when creating new server / after termination , when server brings up with different IP. This has done on our ansible scripts by calling r53 services. Is this suitable option for you?

r. Ismo

0 Karma

soutamo
SplunkTrust
SplunkTrust
Hi
Otherwise it seems to be correct, but can you add multisite = false to onperm-master stanza?
r. Ismo
0 Karma

trevor_dunstan8
Explorer

I should have also mentioned that FW rules appear to be in place as I am able to SSH directly to the AWS cluster master and AWS indexers from our search head over port 8089

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!