Bonding / Link Aggregation must be configured on the computer itself (this example shows Linux) and at the switch.
First configure the computer side for bonding (see section 2), then configure the switch. The switch is shown first here as the switch is the primary objective for this article.
1) On the Mellanox switch.
1.1) IP connection note.
Web browser to the switch IP and login, default credential is: Account: admin, Password: admin.
If only using just the IP for the 1st time, you will get site can't be reached error: ERR_CONNECTION_REFUSED
1st time connecting on a particular machine must also use https://, you will get a message of connection is not private NET::ERR_CERT_AUTHORITY_INVALID error, then proceed in unsafe mode
1.2) LAG (Link Aggregation Group) creation.
ETH Mgmt > Link Aggregation. Check the LACP Enable box.
Create LAG interfaces - In the Create LAG section, fill-in LAG Interface Number, 1 is the 1st and Apply. Name will be created as Po1, Po2....
Click on the "Ports" tab, the LAG Group shows up as "Port channels" as shown below, 1 = Po1, 2 = Po2, etc.....
Select one of the "Port channel:" (this is NOT a physical port # but the LAG interface, Emllanox calls this "Port channel" and it is confusing!
Scroll down to the Port Configuration and adjust. See Po1 (Port channel) example below.
- The MTU to match the port's MTU (9216).
- FlowControl Mode to Global.
- Click Apply
At the "Ports" tab and select the port to assign the bond to, under the Port Configuration section. See Port 1 example below.
- Set MTU to 9216
- Set FlowControl Mode to Global.
- Set LAG/MLAG to the LAG interface name created above (Po1 or Po2...) where this port should be bonded to.
- Set LAG Mode to LACP Active.
- Click Apply
Repeat this for each port and bond them to the appropriate LAG interface.
If setup properly these ports turns green:
ETH Mgmt will show the LAG Members:
Port 1 example:
Po1 (Port channel) example:
2) On the Linux computers:
These files typically exists in
bond0 is the "master" of the bond (create if done not exists):
# cat ifcfg-bond0 DEVICE=bond0 NAME=bond0 BOOTPROTO=none MTU=9000 ONBOOT=yes IPADDR=10.241.243.74 NETMASK=255.255.255.192 GATEWAY=10.241.243.65 NM_CONTROLLED=no BONDING_OPTS="mode=802.3ad miimon=100 lacp_rate=fast xmit_hash_policy=layer3+4"
ens1f0 is one of the two physical port as a "slave" to bond0:
# cat ifcfg-ens1f0 DEVICE=ens1f0 NAME=ens1f0 BOOTPROTO=none MTU=9000 ONBOOT=yes SLAVE=yes MASTER=bond0 NM_CONTROLLED=no
ens1f1 is the other physical port also as a "slave" to bond0:
# cat ifcfg-ens1f1 DEVICE=ens1f1 NAME=ens1f1 BOOTPROTO=none
MTU=9000 ONBOOT=yes SLAVE=yes MASTER=bond0 NM_CONTROLLED=no