LACP Configuration Examples (Part 3)

In part 3 of this series I’ll provide a relatively simple example of a LACP LAG between a HP GbE2c L2/L3 switch and two Nortel switches, we’ll terminate two different LAGs on the two ERS 8600 switches using Nortel’s proprietary SMLT (Split MultiLink Trunking) technology.

Example 2 – Ethernet Routing Switch 8600 to a set of HP GbE2c L2/L3 switches using LACP trunks with SMLT

As I said before a picture is worth a thousand words and can be very helpful in designing any network topology.

lacp-example3

I’m going to skip the configuration of the two Nortel Ethernet Routing Switch 8600s since you can refer to the earlier post for an example of how to configure them. In this design we need to disable the virtual cross connect that exists between the A and B sides of the two HP GbE2c switches. Please note that I’m working with the HP GbE2c (C-Class enclosure) not the GbE2 (P-Class enclosure). There are some slight differences between the two. The virtual trunk ports between the A and B sides are on ports 17 and 18 so those ports need to be disabled in order to prevent a loop.

HP-GbE2c-A / HP-GbE2c-B
/c/port 17/dis
/c/port 18/dis

With the virtual trunk cross connects disabled we can now wire each switch independently to the upstream switch(s) which in this case happens to be two ERS 8600s. As is usual for me I’ll create a network management VLAN and place the IP interface of each GbE2c switch in that VLAN (VLAN 200).

HP-GbE2c-A / HP-GbE2c-B
/c/l2/vlan 200
/c/l2/vlan 200/ena
/c/l2/vlan 200/name "10-101-255-0/24"

Let’s add VLAN 200 to the two ports, 21 and 22, that we’ll be using to uplink to the 8600 switches. We haven’t yet enabled tagging so the switch will ask you if you’d like to change the PVID from VLAN 1 (default) to VLAN 200, you can safely answer yes to this question.

HP-GbE2c-A / HP-GbE2c-B
/c/l2/vlan 200/add 21
/c/l2/vlan 200/add 22

Let’s enable tagging on both uplink ports along with RMON and set the PVID just to be safe;

HP-GbE2c-A / HP-GbE2c-B
/c/port 21/tag ena
/c/port 21/pvid 200
/c/port 21/rmon e
/c/port 22/tag ena
/c/port 22/pvid 200
/c/port 22/rmon e

Let’s turn off Spanning Tree on the uplinks, we only want Spanning Tree local to the switch since SMLT will take care of providing the loop free topology.

HP-GbE2c-A / HP-GbE2c-B
/c/l2/stp 1/port 21/off
/c/l2/stp 1/port 22/off

Now it’s time to configure LACP and create the LAG (Link Aggregation Group). We’ll using LACP key 50 but you could use any admin key (number) so long as both ports are configured with the same admin key.

HP-GbE2c-A / HP-GbE2c-B
/c/l2/lacp/port 21/mode active
/c/l2/lacp/port 21/adminkey 50
/c/l2/lacp/port 22/mode active
/c/l2/lacp/port 22/adminkey 50

Here’s the special sauce that will work in combination with the NIC teaming software to fail over in the event of an upstream switch problem or an uplink problem where the GbE2c continues to function but there’s a problem upstream. This configuration will cause the GbE2c switch to disable (admin-down) the server switch ports in the event that the LACP group goes down. This will cause the NIC teaming configuration on the servers to fail-over to the standby NIC.

HP-GbE2c-A / HP-GbE2c-B
/c/ufd/on
/c/ufd/fdp/ltm/addkey 50
/c/ufd/fdp/ltd/addport  1
/c/ufd/fdp/ltd/addport  2
/c/ufd/fdp/ltd/addport  3
/c/ufd/fdp/ltd/addport  4
/c/ufd/fdp/ltd/addport  5
/c/ufd/fdp/ltd/addport  6
/c/ufd/fdp/ltd/addport  7
/c/ufd/fdp/ltd/addport  8
/c/ufd/fdp/ltd/addport  9
/c/ufd/fdp/ltd/addport 10
/c/ufd/fdp/ltd/addport 11
/c/ufd/fdp/ltd/addport 12
/c/ufd/fdp/ltd/addport 13
/c/ufd/fdp/ltd/addport 14
/c/ufd/fdp/ltd/addport 15
/c/ufd/fdp/ltd/addport 16

If you haven’t already let’s configure an IP address (for management) on VLAN 200;

HP-GbE2c-A
/c/l3/if 1/ena
/c/l3/if 1/addr 10.1.255.128
/c/l3/if 1/mask 255.255.255.0
/c/l3/if 1/broad 10.1.255.255
/c/l3/if 1/vlan 200

We need to use a different IP address for the B side switch on VLAN 200;

HP-GbE2c-B
/c/l3/if 1/ena
/c/l3/if 1/addr 10.1.255.129
/c/l3/if 1/mask 255.255.255.0
/c/l3/if 1/broad 10.1.255.255
/c/l3/if 1/vlan 200

As mentioned by a few other folks on this blog and in the forums this solution only provides an active/passive solution in terms of the NIC teaming configuration. This is because the GbE2c L2/L3 switches don’t support IST/SMLT technology. While this will only provide 1Gbps of bandwidth (2Gbps if you count full duplex) between the blade server and the network it will provide significant level of redundancy and high-availability. In this design the network is protected from a GbE2c switch failure, a Nortel Ethernet Routing Switch 8600 failure, and multiple uplink/downlink failures.

Please feel free to post comments and questions here about this post. Questions regarding specific configurations can be posted in the forums; http://forums.networkinfrastructure.info/nortel-ethernet-switching/

Cheers!

  • LACP Configuration Examples (Part 4) November 25, 2013, 5:10 pm

    […] 470, an Avaya Ethernet Routing Switch 8600 Cluster to an Avaya Ethernet Switch 470 via SMLT, an Avaya Ethernet Routing Switch 8600 Cluster to a HP GbE2c(Blade Technologies) via SMLT, and even an Avaya Ethernet Routing Switch 5520 to Cisco Catalyst […]

{ 16 comments… add one }

  • Rajeev April 21, 2011, 12:10 pm

    Hi, Thanks for the wonderful post. Its very helpful. Got a small question

    - could you let me know how to assign gateway for the management interface

    Appreciate your help!

    Thanks!

    • Michael McNamara April 23, 2011, 1:25 pm

      Hi Rajeev,

      This depends on whether it’s a GbE2c or GbE2 but here you are;

      /cfg/l3
      gw 1
      addr 1.1.1.1
      ena

      Good Luck!

  • David Barahona December 2, 2011, 6:20 pm

    Hola como estas Michael McNamara muy bueno tu blog no se si me puedes ayudar ya que tengo poco de trabajar con este tipo de swiches y tengo que hacer una implementacion de los mismos con smlt y lacp a que correo te puedo contactar para que me des una idea ?? y me des tus observaciones mi correo es

    saludos

  • Luís Nascimento February 4, 2012, 1:16 pm

    Hi Michael,

    I tried to configure an LACP port-channel from a multilink trunk interface (Etherchannel) to a Cisco Cat 6513 but the LACP shows that it’s down.

    >> LACP# dump
    port mode adminkey operkey selected prio aggr trunk status
    21 active 50 50 no 32768 — — down
    22 active 50 50 no 32768 — — down

    Current LACP system ID: 00:17:ef:7e:f4:00
    Current LACP system Priority: 32768
    Current LACP timeout scale: long

    Current LACP params for 21: active, Priority 32768, Admin Key 50

    Current LACP params for 22: active, Priority 32768, Admin Key 50

    On the Cisco CAT 6513:

    !
    interface GigabitEthernet9/37
    switchport
    switchport trunk encapsulation dot1q
    switchport trunk allowed vlan 76,77,550
    switchport mode trunk
    channel-group 15 mode on
    end

    !
    interface GigabitEthernet9/38
    switchport
    switchport trunk encapsulation dot1q
    switchport trunk allowed vlan 76,77,550
    switchport mode trunk
    channel-group 15 mode on
    end
    interface Port-channel15
    switchport
    switchport trunk encapsulation dot1q
    switchport trunk allowed vlan 76,77,550
    switchport mode trunk
    end

    I have tried channel-group 15 mode active on the Cat 6513 but with no results.

    Any idea what’s keeping the LACP down?

    Thanks,
    Luís Nascimento

    • Michael McNamara February 4, 2012, 1:34 pm

      Hi Luis,

      What switch are you connecting to your Catalyst 6513?

      Do you have the configuration to post?

      Cheers!

  • Luis Nascimentro February 4, 2012, 2:09 pm

    I forgot to say that’s a HP GbE2c blade switch, the configuration:

    >> Configuration# dump
    script start “GbE2c Ethernet Blade Switch for HP c-Class BladeSystem” 5 /**** DO NOT EDIT THIS LINE!
    /* Configuration dump taken 19:00:34 Sat Feb 4, 2012
    /* Version 5.1.3, Base MAC address 00:17:ef:7e:f4:00
    /* Slot: 1
    /c/sys/access/https/access e
    /c/sys/access
    tport 54322
    /c/sys/ssnmp
    rcomm
    wcomm “SIM”
    /c/port 1
    tag ena
    pvid 77
    tagpvid dis
    /c/port 2
    tag ena
    pvid 77
    tagpvid dis
    /c/port 3
    tag ena
    pvid 77
    tagpvid dis
    /c/port 4
    tag ena
    pvid 77
    tagpvid dis
    /c/port 5
    tag ena
    tagpvid dis
    /c/port 6
    tag ena
    tagpvid dis
    /c/port 7
    tag ena
    pvid 550
    tagpvid dis
    /c/port 8
    tag ena
    pvid 550
    tagpvid dis
    /c/port 9
    tag ena
    pvid 77
    tagpvid dis
    /c/port 10
    tag ena
    pvid 77
    tagpvid dis
    /c/port 11
    tag ena
    pvid 77
    tagpvid dis
    /c/port 12
    tag ena
    pvid 77
    tagpvid dis
    /c/port 13
    tag ena
    pvid 77
    tagpvid dis
    /c/port 14
    tag ena
    pvid 76
    tagpvid dis
    /c/port 15
    name “IM”
    tag ena
    tagpvid dis
    /c/port 16
    name “SPS20″
    tag ena
    pvid 77
    tagpvid dis
    /c/port 17
    tag ena
    tagpvid dis
    /c/port 18
    tag ena
    tagpvid dis
    /c/port 20
    tagpvid dis
    /c/port 21
    tag ena
    tagpvid dis
    /c/port 21/gig
    mode full
    /c/port 22
    tag ena
    tagpvid dis
    /c/port 22/gig
    mode full
    /c/port 23
    pvid 201
    tagpvid dis
    /c/port 24
    tag ena
    tagpvid dis
    /c/l2/vlan 1
    def 1 2 3 4 5 6 7 9 10 11 12 13 14 15 16 17 18 20 21 22 24
    /c/l2/vlan 76
    ena
    name “VLAN 76″
    def 5 6 14 21 22
    /c/l2/vlan 77
    ena
    name “VLAN 77″
    def 1 2 3 4 7 8 9 10 11 12 13 16 21 22
    /c/l2/vlan 201
    ena
    name “DCR”
    def 8 15 23
    /c/l2/vlan 500
    name “VLAN 500″
    def 8
    /c/l2/vlan 550
    ena
    name “VLAN-Datacenter”
    def 7 8 21 22
    /c/l2/stp 1/clear
    /c/l2/stp 1/brg/prior 65535
    /c/l2/stp 1/add 1 201 500 550
    /c/l2/stp 1/port 3/on
    /c/l2/stp 1/port 4/on
    /c/l2/stp 1/port 5/on
    /c/l2/stp 1/port 6/on
    /c/l2/stp 1/port 7/on
    /c/l2/stp 1/port 8/on
    /c/l2/stp 1/port 9/on
    /c/l2/stp 1/port 10/on
    /c/l2/stp 1/port 11/on
    /c/l2/stp 1/port 12/on
    /c/l2/stp 1/port 13/on
    /c/l2/stp 1/port 14/on
    /c/l2/stp 1/port 15/on
    /c/l2/stp 1/port 16/on
    /c/l2/stp 1/port 21/off
    /c/l2/stp 1/port 22/off
    /c/l2/stp 2/clear
    /c/l2/stp 2/brg/prior 65535
    /c/l2/stp 2/add 76
    /c/l2/stp 3/clear
    /c/l2/stp 3/brg/prior 65535
    /c/l2/stp 3/add 77
    /c/l2/trunk 2
    add 21
    add 22
    /c/sys/syslog
    host 192.168.32.63
    /c/l2/lacp/port 21
    mode active
    adminkey 50
    /c/l2/lacp/port 22
    mode active
    adminkey 50
    /c/ufd/on
    /c/ufd/fdp/ltd
    addport 1
    addport 2
    addport 3
    addport 4
    addport 5
    addport 6
    addport 9
    addport 10
    addport 11
    addport 12
    addport 13
    addport 14
    addport 16
    /c/l3/if 1
    ena
    addr 192.168.10.64
    mask 255.255.255.0
    broad 192.168.10.255
    vlan 77
    /c/l3/gw 1
    ena
    addr 192.168.10.254
    arp enabled
    /c/sys/ntp
    on
    prisrv
    /
    script end /**** DO NOT EDIT THIS LINE!

    >> Configuration#

    • Michael McNamara February 4, 2012, 2:45 pm

      What’s the output from the following command?

      /stats/l2/lacp 21
      /stats/l2/lacp 21

      Cheers!

  • Luis Nascimento February 4, 2012, 2:57 pm

    Here’s the output:

    ——————————————————————
    LACP statistics for port 21:
    Valid LACPDUs received : 0
    Valid Marker PDUs received : 0
    Valid Marker Rsp PDUs received : 0
    Unknown version/TLV type : 0
    Illegal subtype received : 0
    LACPDUs transmitted : 849
    Marker PDUs transmitted : 0
    Marker Rsp PDUs transmitted : 0

    The port 22 it’s shutown to prevent a loop.

    Thanks!

  • Luis Nascimentro February 4, 2012, 3:22 pm

    Michael,

    I found the issue. To change the Port-channel to LACP so, from “channel-group 15 mode on” to “channel-group 15 mode active” we need to take the interface out of the port-channel and then add it again with the “channel-group 15 mode active” to the interface.

    this was in the log of the CAT6513:
    Feb 4 20:06:14.175: %EC-5-ERRPROT2: Command rejected: the interface Gi9/37 is already part of a channel with a different type of protocol enabled

    Thanks for all the help!

    Best Regards,
    Luis Nascimento

  • Sim June 12, 2013, 11:47 am

    Hi ,

    I am configuring same swtiches for LAG with remote switches which is configured with MC-LAG and I am facing difficulty in it.

    Does LAG LACP supported in multi chassic environement.

    Regards,
    Sim

    • Michael McNamara June 15, 2013, 8:59 am

      There is no industry standard “mutli-chassis” LAG or LACP.

      Avaya (formerly Nortel) utilizes IST/SMLT (InterSwitch Trunking/Split MultiLink Trunking) while Cisco utilizes vPC (Virtual Port Channels).

      Several vendors support LAG/LACP across switches in the same stack but any legacy true multi-chassis Layer 2 pathing is going to be proprietary.

      I’m obviously excluding new protocols such as SPBm and FabricPath.

      Cheers!

  • Ryan August 23, 2013, 12:20 pm

    Maybe I’m missing something but if I’m not mistaken, If you configure your MSTP instances properly on the intra switch link and setup lacp between your downstream switches and upstream blade system chassis switches MSTP should just block the intra switch link and allow full use of all 5 gigabit ports available on the switches allowing for proper fail over in the event of a network problem.

    • Michael McNamara August 25, 2013, 1:34 pm

      Hi Ryan,

      IST/SMLT technology is not compatible with STP/RSTP/MSTP.

      Now with that said there have been some changes in 7.x software for the ERS 8600/8800 which I believe now allows some RSTP/MSTP interaction.

      Good Luck!

      • Ryan August 25, 2013, 2:29 pm

        Nice i hadn’t realized that but good to know!

Leave a Comment