Changing peer IP address ?

[fde]

Hello,

I've got a cluster with 2 peers, 1 seach head and 1 CM. All of them with a single network.

Due to network change, the server are going to have an additionnal card with a new network address.

I'll like to know if it's possible to swap the IP address used for réplication between peer member and SH communication while keeping the old one for forwarder communication 

Initialy:

peer 1 => 10.254.x.1, peer 2 => 10.254.x.2

After changes :

Peer 1 => forwarder communication, 10.254.x.1, réplication/SH comm=> 10.254.y.1

Peer 2 => forwarder communication, 10.254.x.2, réplication/SH comm=> 10.254.y.2

I've try to use register_replication_address and register_search_address parameter in server.conf with the new address 10.254.y. but the peer and the CM, complain of duplicate guid/member.

Do you have any advice on how to do this, if it's possible ?

Thanks 

Frédéric 

[fde · Accepted solution]

Hi,

My problem is solved.

With support guidance, I change IP address used for replication.

Here is the steps used to:

1°) stop indexer

2°) change etc/system/local/server.conf and  add register_replication_address parameter with the new IP

3°) rename etc/instance.conf to etc/instance.conf.bkp

4°) restart indexer

• CM =  Index clustering after few seconds => OK
• MC reconfigured the Indexer => OK
• CM and SH  =  Distributed Search =>  2 entry for the same instance name with 2 different Peer URI, the 2 entry appeared down and Sick

5°) I deleted one entry, the one with the old Peer URI => After 5min and a refresh, no more entry Down and Sick.

And now, all seems to be ok. IP address for peer URI is now showing the new IP address.

[gcusello]

Hi @fde ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉

[gcusello]

Hi @fde ,

I'm not sure that's possible to use two IP addresses for each server: in Splunk every hostname has one IP address.

Ciao.

Giuseppe

[fde]

Hi,

I've seen in server.conf specification under clustering stanza, different parameter (https://docs.splunk.com/Documentation/Splunk/9.0.7/Admin/Serverconf#High_availability_clustering_con...)

register_search_address: "This is the address that advertises the peer to search heads. This is useful in the cases where a splunk host machine has multiple interfaces and only one of them can be reached by another splunkd instance."

register_forwarder_address: "This is the address on which a peer is available for accepting data from forwarder.This is useful in the cases where a splunk host machine has multiple interfaces and only one of them can be reached by another splunkd instance."

register_replication_address: "This is the address on which a peer is available for accepting replication data. This is useful in the cases where a peer host machine has multiple interfaces and only one of them can be reached by another splunkd instance."

So It's seems to be possible to use multiple IP, but I'm wordering how to convert peer with single IP to peer with multiple IP using this parameter.

Frédéric

[fde]

Here is the message logged on peer side, when during starting after adding register_replication_address

05-02-2024 11:26:38.046 +0200 WARN CMSlave [1355571 indexerPipe] - Failed to register with cluster master reason: failed method=POST path=/services/cluster/master/peers/?output_mode=json manager=CM:8089 rv=0 gotConnectionError=0 gotUnexpectedStatusCode=1 actual_response_code=500 expected_response_code=2xx status_line="Internal Server Error" socket_error="No error" remote_error=Cannot add peer=NEW-IP mgmtport=8089 (reason: Peer with guid=F00C07FD-F8A6-4C15-91D3-8A6CDCF28C96 is already registered and UP). [ event=addPeer status=retrying AddPeerRequest: { active_bundle_id=054CB061CFCD038A4B98FCDB01CE3F2F add_type=Initial-Add base_generation_id=0 batch_serialno=1 batch_size=26 forwarderdata_rcv_port=9997 forwarderdata_use_ssl=0 guid=F00C07FD-F8A6-4C15-91D3-8A6CDCF28C96 last_complete_generation_id=0 latest_bundle_id=054CB061CFCD038A4B98FCDB01CE3F2F mgmt_port=8089 register_forwarder_address= register_replication_address=NEW-IP register_search_address=NEW-IP replication_port=8087 replication_use_ssl=0 replications= server_name=PEER-NAME site=site1 splunk_version=9.0.7 splunkd_build_number=b985591d12fd status=Up } Batch 1/26 ].