technology, networking and IP telephony
HP NIC Teaming with Nortel Switches
I recently needed to look at HP’s NIC Teaming feature within their Proliant server product line to provide “ultra” high-availability to a new Windows 2008 Cluster which was running Microsoft Exchange 2007. It seems that NIC teaming has come a long ways from the original NIC fault tolerance where one NIC would simply act as a standby should the first loose LINK with the network switch.
HP’s NIC Teaming now supports 802.3ad (LACP) which provides a load balanced and fault tolerant solution. I’ve posted several documents to my website for anyone that might be looking for additional information.
Here is a very good document that outlines NIC teaming from head to toe, along with additional documents that outline the different capabilities and what operating systems are supported.
In short you need to setup an LACP group on the Nortel switch for any ports that you want to be in the LAG (Link Aggregation Group). You can leave the HP NIC teaming options set to Automatic and it should negotiate as a “802.3ad Dynamic with Fault Tolerance” when you have the Nortel switch configuration properly.
5520-48T-PWR> enable 5520-48T-PWR# config term 5520-48T-PWR(config)# interface FastEthernet 1/1-2 5520-48T-PWR(config-if)#lacp aggregation port 1/1-2 enable 5520-48T-PWR(config-if)#lacp mode port 1/1-2 active 5520-48T-PWR(config-if)#lacp key port 1/1-2 101
Cheers!
Related posts:
- #3 written by lou 3 years ago
i wonder if servers-connected ERS switch (not sure if it’s 8600 or stacking family) has a feature that can shutdown access ports when uplink connection fails. it’s not SMLT, but more like black-hole prevention.
e.g.:- ERS switches below is a server farm switch
- if ERS-A detects failure in uplink ports to L2-switch A, to prevent blackholing traffic coming from servers, the ERS shuts down the access ports (where servers are attached to).
- so, automatically, servers will switchover to the other NIC to use ERS-B.++++++++++
| L2-switch A |
++++++++++
||
|| uplink
||
+————–+
| |
| ERS A |
| |
| |
| access |
+————–+
| | | |
| | | |
| | | |
server farms (dual home/NIC-team?)
| | | |
| | | |
| | | |
+————–+
| |
| ERS B |
| |
| |
| access |
+————–+
||
|| uplink
||
++++++++++
| L2-switch |
| |
++++++++++thanks.
Hi Lou,
You bring up some really good stuff. I believe you’re making reference to UFD (Uplink Failure Detection) which is present in some of the HP GbE2 switches and HP ProCurve switches.
You might also want to look at VLACP (Virtual Link Aggregation Control Protocol). It’s a proprietary extension of LACP that Nortel uses to provide a “hearbeat” if you will between two switches. It’s primarily used when connecting to end points over an intermediate network (such as a carriers Ethernet backbone). However it plays a very important role in how redundant networks detect and recover from a network failures.
So NIC teaming with LACP should provide you that hearbeat between the servers and the access switch. You would then run VLACP across an SMLT uplink to two core ERS 8600 switches. There is also an advanced license key available from HP that enables SMLT link functionality on the HP NIC teaming side of things although I’ve never investigated that.
You can find additional information on VLACP from this post.
On another note I’ve never run a LAG (802.3ad) in an SMLT configuration. We primarily use Nortel’s MultiLink trunking (MLT) which is proprietary to Nortel equipment but I’m guessing it’s probably supported.
Thanks for the comment!
- #5 written by Bill McDonald 3 years ago
HP Discontinued the Advanced / Inteligent Networking Pack option late 2008, so you no longer have the Dual Channel options.
We currently have some of our Servers configured in SLB mode, and using Dual 8300 Nortel cores (SMLT/IST/etc) We have it split accross the cores using S-SMLT (Single Port SMLT)
You also need to turn off Spanning tree completly on these ports in order for it to work.This did not work on the 3.0.3 Codebase, and we upgraded to 4.1.1.0 to fix alot of other issues too. (MS- NLB Support, etc)
It provides us redundant cores, full speed, and active failover. It works great.
- #7 written by aaron 3 years ago
Hello Michael;
I have searched high/low for the exact steps to utilize LACP on a Nortel 8310 using 8348GTX cards, and am unsuccessful thus far. Below is what I have.* server with dual nics
* Admin wants us to config LACP on our Nortel ERS 8310 chassis running 4.1.1 code.
* I will utilize 2 different 8348GTX modules.Here is what I have configured so far, but it doesn’t appear to want to play nicely.
2 PORTS on 8310 configured with:
* lacp ENABLED
* key 5
* aggregation TRUE
* mode ACTIVE
* member of server vlanMLT 5
*created mlt 5
* issued mlt 5 lacp enable
* issued mlt 5 to use key 5
* LACP enabled.
* aggregatable.
* added the appropriate VLAN the server should use.
**UNABLE to add port members w/o getting the following error:
“rcMLTPortMemebers.5:This MLT is used for link aggregation.”This is my first time setting one of these up, I must be doing something wrong.
Any thoughts/guidance would be greatly appreciated.
Thanks in advance,
aaron
Hi Aaron,
You’ve actually done too much configuration. You don’t need to configure a Multi-Link trunk, you only need to enable LACP on the specific ports and set them to the same key value. A LAG (Link Aggregation Group) is the same thing as an MLT (Multi-Link Trunk) only first is dynamic while the second is statically configured. I don’t have specific experience on the ERS 8300 but I would assume that it works just the same as the ERS 5500 or ERS 8600 switches.
I’ve posted the Link Aggregation Control Protocol(LACP) 802.3ad and VLACP for ES and ERS Technical Configuration Guide on my website. I would suggest you review the document if you have any questions about LACP and/or VLACP.
Please let us know if that solves your problem.
Good Luck!- #9 written by aaron 3 years ago
Michael,
thanks for the quick response, and for sharing the information you did. (I read through the documentation, and made some changes to my original config.) We didn’t get a chance to test this configuration against a server yet, but plugged an ERS 5520 (configured for LACP) into the 8310 LACP enabled ports and did notice port assignments/vlan under the MLT tab dynamically populate, so it appears to be working.Again, many thanks.
aaorn
- #11 written by Zubair 2 years ago
Dear MIchael,
I am trying to use LACP for some HITACHI NAS servers they support link aggregation 802.3ad dynamic. What is the signifance of Key? as on my servers i dont have any option to put this key. The only option i get is link aggregation add servers NIC to it and assign IP to them. Can i use default key?
Thanks
ZUbairHi Zubair,
You won’t have any option on the client side of things (Hitachi NAS). You will have options on the network switch. The admin key helps the switch to identify which ports should be grouped (trunked) together. If the ports you are using in connecting the NAS have different key values then the network switch will not know it should trunk them together.
Good Luck!
- #13 written by Nicholas 2 years ago
Hi Michael,
I am having problems trying to SMLT to Linux-based devices. All of them support 802.3ad link aggregation / bonding. I think the problem might be on my Nortel end. I have 2x 8310 chassi and I need to allocate 1 port from each chassi and connect it to a Linux router (Vyatta, etc). I have the bonding configured on the Linux box. I was wondering what would be the configuration on the Nortel end.
I tried going into the port and set SLT, but apparently it didnt work. I was only able to Ping one of the switches when using single port SMLT. I tried LACP and it didnt work either. I must be doing something wrong. Can you show me/correct me in terms of the Nortel configuration? Someone mentioned about using LACP via SMLTSystemID (basically LACP on SMLT) but I am kinda confused as SMLT ports cannot be used for LACP (the JVM does not permit it).
Thanks in advance,
Nicholas
- #14 written by Sem 2 years ago
- #16 written by siraj 2 years ago
Hi Michael,
i am trying to configure lacp btw cisco 3020 blade enclosure switch and ERS8600. there are 4 links in the trunk with auto-negotiate enabled. one link is negotiating at 100mbps and not becoming part od lacp. on cisco side he port is showing as suspended.
8600 config,
# MLT CONFIGURATION
#
mlt 24 create
mlt 24 ntstg disable
mlt 24 lacp enable
mlt 24 lacp key 3# LACP CONFIGURATIO
#
config lacp enable# PORT CONFIGURATION – PHASE I
#
ethernet 2/19 perform-tagging enable
ethernet 2/20 perform-tagging enable
ethernet 2/21 perform-tagging enable
ethernet 2/22 perform-tagging enable# STG CONFIGURATION
#
stg 1 remove ports 2/23
stg 1 add ports 2/20-2/22,2/37-2/39,4/41-4/48# PORT CONFIGURATION – PHASE II
#
ethernet 2/19 default-vlan-id 501
ethernet 2/19 stg 1 stp disable
ethernet 2/19 lacp key 3
ethernet 2/19 lacp aggregation true
ethernet 2/19 lacp timeout short
ethernet 2/19 lacp enable
ethernet 2/20 default-vlan-id 501
ethernet 2/20 stg 1 stp disable
ethernet 2/20 lacp key 3
ethernet 2/20 lacp aggregation true
ethernet 2/20 lacp timeout short
ethernet 2/20 lacp enable
ethernet 2/21 default-vlan-id 501
ethernet 2/21 stg 1 stp disable
ethernet 2/21 lacp key 3
ethernet 2/21 lacp aggregation true
ethernet 2/21 lacp timeout short
ethernet 2/21 lacp enable
ethernet 2/22 default-vlan-id 501
ethernet 2/22 stg 1 stp disable
ethernet 2/22 lacp key 3
ethernet 2/22 lacp aggregation true
ethernet 2/22 lacp timeout short
ethernet 2/22 lacp enable - #18 written by siraj 2 years ago
only one port 2/21(nortel side)–> 0/19(cisco side) got suspended.
!
spanning-tree mode pvst
spanning-tree extend system-idinterface Port-channel10
switchport trunk encapsulation dot1q
switchport trunk native vlan 501
switchport mode trunk!
interface GigabitEthernet0/17
switchport trunk encapsulation dot1q
switchport trunk native vlan 501
switchport mode trunk
media-type rj45
speed 100
duplex full
channel-protocol lacp
channel-group 10 mode active
!
interface GigabitEthernet0/18
switchport trunk encapsulation dot1q
switchport trunk native vlan 501
switchport mode trunk
media-type rj45
speed 100
duplex full
channel-protocol lacp
channel-group 10 mode active
!
interface GigabitEthernet0/19
switchport trunk encapsulation dot1q
switchport trunk native vlan 501
switchport mode trunk
media-type rj45
speed 100
duplex full
channel-protocol lacp
channel-group 10 mode active
!
interface GigabitEthernet0/20
switchport trunk encapsulation dot1q
switchport trunk native vlan 501
switchport mode trunk
media-type rj45
speed 100
duplex full
channel-protocol lacp
channel-group 10 mode active
! - #19 written by siraj 2 years ago
Hi Matt,
suspecting something related to port 21(nortel side) harsdware, i tried changing the port to 18(nortel side) but no luck. this time port 20(nortel side) gone out.
below are the show cmd outputs,
DataCenter_Core1:5# sh mlt info
================================================================================
Mlt Info
================================================================================
PORT SVLAN MLT MLT PORT VLAN
MLTID IFINDEX NAME TYPE TYPE ADMIN CURRENT MEMBERS IDS
——————————————————————————–
24 4119 MLT-24 trunk normal norm norm 2/18-2/19,2/22 501 506
28 4123 Hitachi_NAS access normal norm norm 2/37-2/39 503
29 4124 SMLT-MC-Srv access normal smlt norm 4/41 501
30 4125 SMLT-DC-DIST access normal smlt smlt 4/42-4/45 13
32 4127 DC-SW-IST trunk normal ist ist 4/46-4/48 12 13 50
1 502 503 504 505 506MULTICAST DESIGNATED LACP LACP
MLTID IFINDEX DISTRIBUTION NT-STG PORTS ADMIN OPER
——————————————————————————–
24 4119 disable disable 2/18 enable up
28 4123 disable enable 2/37 enable up
29 4124 disable enable null disable down
30 4125 disable enable 4/42 disable down
32 4127 disable enable 4/46 disable downDataCenter_Core1:5# sh mlt lacp info 4119
================================================================================
LACP Aggrgator Information
================================================================================
MAC COLLECTOR AGGR PORT
MLTID IFINDEX ADDR MAXDELAY ORINDI MEMBERS
——————————————————————————–
24 4119 00:1c:9c:1f:90:51 32768 aggr 2/18-2/19,2/22——————————————————————————–
OPER OPERLAST
MLTID IFINDEX STATE CHANGE
——————————————————————————–
24 4119 up 304 day(s), 14:16:07——————————————————————————–
ACTOR ACTOR ACTOR ACTOR
MLTID IFINDEX SYSPRIO SYSID ADMINKEY OPERKEY
——————————————————————————–
24 4119 32768 00:1c:9c:1f:90:00 3 3——————————————————————————–
PARTNER PARTNER PARTNER
MLTID IFINDEX SYSPRIO SYSID OPERKEY
——————————————————————————–
24 4119 32768 00:24:c3:93:ee:80 10DataCenter_Core1:5# sh port info lacp all port 2/18,2/19,2/20,2/22
================================================================================
Actor Admin
================================================================================
INDEX SYS SYS KEY PORT PORT STATE
PRIO ID PRIO
——————————————————————————–
2/18 32768 00:1c:9c:1f:90:00 3 0×91 32768 act long aggr
2/19 32768 00:1c:9c:1f:90:00 3 0×92 32768 act long aggr
2/20 32768 00:1c:9c:1f:90:00 3 0×93 32768 act long aggr
2/22 32768 00:1c:9c:1f:90:00 3 0×95 32768 act long aggr================================================================================
Actor Oper
================================================================================
INDEX KEY SELECTED ATTACHED AGGR STATE
AGGR ID AGGR ID
——————————————————————————–
2/18 3 4119 4119 true act long aggr sync col dis
2/19 3 4119 4119 true act long aggr sync col dis
2/20 1171 147 147 true act long indi sync def
2/22 3 4119 4119 true act long aggr sync col dis================================================================================
Partner Admin
================================================================================
INDEX SYS SYS KEY PORT PORT STATE
PRIO ID PRIO
——————————————————————————–
2/18 0 00:00:00:00:00:00 0 0×0 0 pas long indi
2/19 0 00:00:00:00:00:00 0 0×0 0 pas long indi
2/20 0 00:00:00:00:00:00 0 0×0 0 pas long indi
2/22 0 00:00:00:00:00:00 0 0×0 0 pas long indi================================================================================
Partner Operational
================================================================================
INDEX SYS SYS KEY PORT PORT STATE
PRIO ID PRIO
——————————————————————————–
2/18 32768 00:24:c3:93:ee:80 10 0×11 32768 act long aggr sync col
dis
2/19 32768 00:24:c3:93:ee:80 10 0×12 32768 act long aggr sync col
dis
2/20 0 00:00:00:00:00:00 0 0×0 0 pas long indi2/22 32768 00:24:c3:93:ee:80 10 0×14 32768 act long aggr sync col
dis================================================================================
LACP Extention
================================================================================
INDEX ADMIN OPER FAST SLOW AGGRWAIT TIMEOUT FAST SLOW AGGRWAIT TIME
OUT
ENABLED ENABLED TIME TIME TIME SCALE TIME TIME TIME SCALEADMIN ADMIN ADMIN ADMIN OPER OPER OPER OPER
——————————————————————————–
2/18 true true 1000 30000 2000 3 1000 30000 2000 32/19 true true 1000 30000 2000 3 1000 30000 2000 3
2/20 true false 1000 30000 2000 3 1000 30000 2000 3
2/22 true true 1000 30000 2000 3 1000 30000 2000 3
- #20 written by siraj 2 years ago
HI Siraj,
Are you using a cross-over cable or does the HP 3120 Interconnect provide MDI-X features? What card are you using in the Nortel ERS 8600? 8648GTR, 8648GTRS?
As you already know your configuration is right on the money… very interesting that you have duplex mis-match issues with that new equipment.
Thanks for the comment!
PS: I’m not sure that Cisco supports short timers with LACP?
- #23 written by DaveTheRave 2 years ago
I think we have been fed some duff advice on HP NIC teaming! We use HP proliants plugged into two 8600′s with high availability. We were told to create an SMLT (across the two 8600′s)and then manualy configure the HP NIC teaming software to use “Switch Assisted Load Balancing and Fault tolerance” on “TCP”
Is this wrong?
Hi Dave,
I believe this is the old school (static) way of configuring a trunk using Nortel’s proprietary MLT feature. While this configuration will obviously work there are some drawbacks. The biggest draw back that stands out to me is the lack of any failure detection beyond link loss. Using the new LACP 802.3ad configuration allows both sides to dynamically add/remove links and allows both sides the ability to poll the far end providing a greater deal of failure detection beyond just link loss.
The LACP feature has really only matured in the past few years so prior to that you had Nortel’s MLT and Cisco’s Etherchannel.
Thanks for the comment!
- #26 written by Raj 2 years ago
Hi Raj,
I would suggest you post your issues in the discussion forums
You’ll probably want to provide some additional information for anyone to offer any advice and/or help.
Good Luck!
- #28 written by Parvesh katoch 1 year ago
- #29 written by Jeff O'Hara 10 months ago
Micheal,
Thanks for all the information it is very helpful. I need to connect a server to a pair of 8600 chassis’s that are connected together with IST’s. The question I have is would this be a simple MLT with LACP configuration or do I need to configure it as SMLT?The configuration I have compiled for MLT with LACP is below.
ERS8600-A
config ethernet 1/44
config vlan 1 ports remove 1/44
config vlan 2 ports add 1/44ERS8600-B
config ethernet 1/44
config vlan 1 ports remove 1/44
config vlan 2 ports add 1/44ERS8600-A
config ethernet 1/44 lacp key 2
config ethernet 1/44 lacp aggregation true
config ethernet 1/44 lacp timeout short
config ethernet 1/44 lacp enableERS8600-B
config ethernet 1/44 lacp key 2
config ethernet 1/44 lacp aggregation true
config ethernet 1/44 lacp timeout short
config ethernet 1/44 lacp enableERS8600-A
config mlt 2 create
config mlt 2 name “MLT_LACP_1″
config mlt 2 lacp key 2
config mlt 2 lacp enableERS8600-B
config mlt 2 create
config mlt 2 name “MLT_LACP_1″
config mlt 2 lacp key 2
config mlt 2 lacp enable - #31 written by Jeff O'Hara 10 months ago
Micheal,
Thanks again for the assistance. I found an Avaya document – Data Center Server Access Solution Guide
NN48500-577, Date: July 2010, Version: 1.1 – They also state that it needs to be SMLT, however in the configuration they present they don’t use LACP, the configuration example is the same for both 8600′s:config mlt 2 create
config mlt 2 perform-tagging enable
config mlt 2 add port 1/44
config vlan 2 add-mlt 2
config mlt 2 smlt create smlt-id 2Choose switch assisted server load balancing with failover protection on the HP server.
Do you agree with this or should I stick with the MLT\LACP configuration as presented earlier and just add SMLT as you recommended. The Avaya configuration looks simplier on the surface, however that doesn’t always mean it will work better.
Thanks
JeffHi Jeff,
If your equipment supports LACP (802.3ad) you want to use that over just a static port channel, Ether channel, Multi-Link trunk, etc. LACP will provide a heartbeat mechanism and several other features that regular static grouping/bonding doesn’t support. It will also facilitate better control over the load-balancing mechanism that’s employed to distribute the traffic across the multiple paths.
Why don’t you test it yourself ahead of time?
I have run several similar tests using an HP DL360 G5 creating an HP NIC team leaving the NIC teaming software configured as “Automatic” (default). The server will look for the LACP keepalives and will automatically configured itself for LACP/802.3ad mode if it sees those LACP packets. If it doesn’t see any LACP packets it will fall back to NFT (Network Fault Tolerant), perhaps even NFT with TX load-balancing if your NIC software supports that.
Good Luck!
- #33 written by Jeff O'Hara 10 months ago
It’s no problem at all… you might want to browse the forums;
http://forums.networkinfrastructure.info/
You better yet post your findings so that others following in your footsteps can benefit.
Cheers!
- Comment Feed for this Post
The link http://www.michaelfmcnmara.com/files/TeamingWP.pdf
is broken…