[Veritas-ha] SUMMARY: Virtual IP question

[Frank_DeMontier@ssga.com]

Many thanks and kudos to:

Bayard Bell
Neil Bliss
James Mello
Jim Senicka

All say basically the same thing. Following is the responses from the
gurus:

You may need to IfConfigTwice to get the router(s) servicing the virtual
IP to update their ARP cache and related data.  Particularly for
high-performance routers and routing switches, MAC and IP address are
linked together with VLAN and/or interface information in cache and then
linked to structures like the ARP cache to reduce the CPU load on the
router.  If you're dealing with stale cache data, you could easily
forward the packet out the wrong interface and promptly have it
discarded by the network stack.  Unless you've got a mutant DMZ layout,
this problem should not, however, be specific to the outside network.

Have you sniff'ed, tcpdump'ed, or snort'ed what's on the wire?  Very
important is whether the router or routing switch actually forwards the
packet to the correct interface?

> The question....is there a way to "mask" and/or force the external
network
> to recognize the virtual IP..?? I also believe this is more "Sun" related
> than VCS,(??) but thought I'd ask...

The external network only knows the IP, as there is no layer 2
connectivity between the outside world and your DMZ or internal network
(right?).  The MAC translation of the IP is therefore only known on the
local segment.
##########################################################################

That's why I suggested that you check to see if your router is accepting
gratuitous arps :) A lot of times, the router is configured to cache that
particular data for a long period of time without regard to new arp's
being sent out. In very security concious environs, these addresses are
sometimes hardcoded at the router.

Based on the site security policy as well as the type of equipment you
see, you'll need to try to balance your security versus speed of MAC
address updates.... We ran into this a *ton* when using F5 BigIP machines
(same problem, different equipment).....

#######################################################################################
The cluster is using IP aliasing to failover.
At time of failover, a gratuitous ARP request is sent
out advising all concerned that the IP address new equals a new
MAC address. Your external net is coming through a router with
a connection to the VCS hosts. This router is configured to
not accept gratuitous ARP updates on the interface facing
the cluster. Reconfigure the router to accept the ARP updates.

###########################################################################################
You'd really need to go into more detail about how it is that this failover
pair is connected to the outside world.  So after a failover, the virtual
IP is
brought up on the second box, but traffic from the second box (with it's
spiffy
new IP address) can no longer get to the outside world.  Can other machines
ping the virtual IP?  If traffic can get into the box, but not back out,
I'd
suspect a routing issue on the box itself.  If traffic can't get in or out,
I'd
start taking a look at ARP caches, especially on any routers or switches
that
these systems are connected to.

Buddy DeMontier
State Street Global Advisors
Infrastructure Technical Services
2 International Place
Boston Ma 02110
617-664-6141