Hey gang. This is somewhat urgent.
Moved the indexer to a physical box from a virtual. In our situation we use an alias 'splunkprodindex' to point to the indexer. We did this knowing we would have to move the indexer off virtual. This was to theoretically make it easier to do this.
Shut everything down. rsynced the data. Changed the alias to point to the new indexer in DNS. Updated serverclass to whitelist the indexer in the deployment server. Updated forwarder outputs with a comment field as it contained the DNS alias and we wanted to make sure the deployment server had a conversation with the forwarders immediately
I have 55 missing forwarders in the deployment monitor screen. When we check a number of these, we see in the log they are still trying to talk to the old server name. Active Directory DNS was flushed. Name resolution on the box (Windows AD servers) was confirmed as to be pointing to the right IP address. We removed the old indexer from the indexer role, and started the new indexer and deployment server and we could see that serverclass and forwarder outputs had a recent timestamp that was after the changes made in the deployment server.
We've had several things report in with no trouble. We have stopped and restated forwarders. We have in one case uninstalled and reinstalled the forwarder on one Active Directory server. When we installed it, we only supplied the deployment server name, no indexer name specified.
One AD server in a different domain has since checked in to the new server. We made no changes to it.
What do we do next? We're going to lose AD data as the logs roll over shortly if we have not already.
