Setting up a CAS server for the Exchange app. There are two NICs on the machine (2008 R2), the deployment server is seeing the private IP for the CAS array. Yet the binding order for the NICs are to use the public interface before the private one. Not sure why Splunk appears to be using the private IP to communicate with the deploy server. How do I tell Splunk to use the public interface?
I've restarted the Splunk forwarding service, and reloaded the deploy-server.
So, your "private" and "public" IP addresses are on the same subnet? In that case, you'll need to add a static route that points to your Splunk server and specifies which interface to use:
netsh interface ip show interface
to get your interface indexes (Idx column). Then, do a:
netsh interface ip add route 188.8.131.52/32 interface=1 nexthop=184.108.40.206
Where 220.127.116.11 is the IP address of your Splunk deployment server, 1 is the interface index you got from the show interface command, and 18.104.22.168 is the gateway for the 22.214.171.124/24 subnet. Modify appropriately with your real world values.
You'll probably want to set up a static route to point to your indexer too.
You can to a
netsh interface ip add route /? to get help for the command.
For the one that works, my guess is that you just got lucky as it had a 50/50 chance of picking the one you wanted. It's probably too late for this, but it may have behooved you to have your private and public IPs on separate subnets.
It does indeed look like a routing problem. The servers, including the deployment server, are on 126.96.36.199. For a CAS server that's properly identifying itself, the routing looks like:
188.8.131.52 255.255.255.0 On-link 184.108.40.206 266 220.127.116.11 255.255.255.0 On-link 18.104.22.168 266
For the problemantic CAS server, it has this:
22.214.171.124 255.255.255.0 On-link 126.96.36.199 266 188.8.131.52 255.255.255.0 On-link 184.108.40.206 266
I assume the issue is that the private IP comes first on the problematic CAS. Any idea how to fix that? (aware it's not a splunk issue). I'm unable to reboot this server without 3 days of Change Management notification to users, so a fix with no impact to production users would be good.
I'm not sure exactly how you'd have to fix it, but in Windows you can specifiy cost/priorities for different interfaces, which will affect the order in which routing is chosen. The CLI
route add command can let you change it, but you should probably actually set it via the TCP/IP Networking Control Panel.
Is this a binding question, or more of an addressing/routing one? Unless configured with something like
SPLUNK_BINDIP, Splunk relies on the OS to decide the appropriate interface to use based on the destination address and the route table.
I would confirm the address/DNS name for the deployment server in
deploymentclient.conf, and check the routing table to see which interface should be selected for that address/name. If the same name DNS maps to multiple IP addresses, this might contribute to the issue.
Of course, if you are using
SPLUNK_BINDIP then all the above is moot.