bonding, link aggregation, and switch config

no_at_spam.org
Date: 12/10/04 

Date: Fri, 10 Dec 2004 13:37:33 GMT

Hi,

I have a few questions regarding Linux bonding for high availability.
I'm trying to setup bonding with 2.4.20 kernel and have a few questions.
 
Scenario 1: Creating a bond between two servers (wired back-to-back)
using dual gigabit ports on each side. (Example 1 in the bonding.txt file)
 
I set it up just like advised in the bonding.txt file, i.e round robin
mode with twice the speed and failover. There are few problems:

  1. Link aggregration does not work. I dump a huge iso across the bonded
     pipe using scp and it took about 30 seconds. I took down the bond
     and dumped the same file over individual NICs and it also took about
     30 seconds. I ran the same tests again with ttcp over a couple minute
     interval (instead of dumping the iso) and the speed was similar to
     dumping the iso. And that speed is around what it should be for a
     gigabit port (~ 900MBit/sec)

  2. Failover using MII. To simulate a fault, I first ifdown the NIC, and
     the sending server still sends packets out in round robin fashion. That
     is understandable because even though the interface is ifdown, mii-tool
     still show the link ok msg. I simulate a faul by pulling the cable
     and it worked fine. So is miimon the right mode to use to failover
     if an interface is ifdown.

  3. Link aggregration with ARP interval: I ran #1 again using ARP
     insteading of miimon and it still didn't work.
     link aggregration still didn't work.

  4. If I understand correctly from the bonding.txt file and basic networking
     knowledge, if a host is in the ARP table, the node should not ARP again.
     So I did a ping test across the bond and watch the traffic. ARP
     request/reply msgs were still flying by though the ARP table still lists
     the host I'm pinging. How do you explain that?

  5. Using arp interval with round-robin doesn't seem to work. I sniffed
     both physical interfaces and the packets are only sent out on one
     interface though I use mode 0 (default, round robin)

  6. If I want to run the dual gigaE ports up to one or two cisco switch(es)
     for failover and/or link aggregration. Would you use miimon or
     arp_interval?

     a) According to bonding.txt, if both NICs on host go to a single switch,
     setup a trunk on both switch ports for link aggregation. Is that right?
     How does the switch handle the same MAC on both ports? If it blocks
     the port with STP, you would not get "twice the speed" would you?
     Should I setup a {Fast,Giga}EtherChannel on the switch instead of
     a trunk?

  7. Does this kernel support 802.3ad or LACP by default? Is it more desired
     when connecting to a cisco switch if the switch supports it?

Scenario 2: Connecting the NICs to more than one switch to eliminate SPoF.
The bonding.txt file says "This mode is more problematic...", have you tried
it successfully? If so, what's your config?

Please advise. Thanks.

Henry

Relevant Pages

  • Re: Restart: VLAN question...
    ... How do I troubleshoot this setup to try to track dow what is happening..? ... - See if you also see the ARP response. ... If both the above look OK, sniff the traffic between the ... client switch and the "Internet connection switch" and see ...
    (comp.dcom.lans.ethernet)
  • RE: ARP issues with dual ethernet link and CE 6.0 R2
    ... Isn't there some way, when you swap from port 1 to port 2 on the switch, to ... internal 10/100 Mbit ethernet MAC) and running Microsoft Windows ... traffic between the two we can see that the device sends an ARP ...
    (microsoft.public.windowsce.platbuilder)
  • Re: ARP issues with dual ethernet link and CE 6.0 R2
    ... Isn't there some way, when you swap from port 1 to port 2 on the switch, to ... internal 10/100 Mbit ethernet MAC) and running Microsoft Windows ... traffic between the two we can see that the device sends an ARP ...
    (microsoft.public.windowsce.platbuilder)
  • Re: Aging implementation
    ... ARP timers are usually adjustable by the admin with an arp ... VLAN you are running HSRP. ... If switch A is the HSRP router, ... mac-address-table, ensuring that the mac-address is known and keeping ...
    (comp.dcom.sys.cisco)
  • Re: Aging implementation
    ... you need to change the default ARP timeout to 300 seconds to ... VLAN you are running HSRP. ...  If switch A is the HSRP router, ...
    (comp.dcom.sys.cisco)