If your interested in some of the back story behind SMLT you might find the conversation interesting.

Cheers!

Let me start with some definitions and then we’ll move on from there;

• MLT (MultiLink Trunk) a proprietary bonding protocol to bond two or more physical links into a single virtual link between two switches.
• DMLT (Distributed MultiLink Trunk) a proprietary bonding protocol to bond two or more physical links into a single virtual link across multiple cards or switches (in a stack configuration) between two switches.
• SMLT (Split MultiLink Trunk) a proprietary bonding protocol to bond two or more physical links into a single virtual link between two core cluster switches and a single edge/distribution switch.
• SLT (Single Port Split MultiLink Trunk – formerly S-SMLT for Single Split MultiLink Trunk) a proprietary bonding protocol to bond two physical links into a single virtual link between two core switches and a single edge/distribution switch. This is just an SMLT with only two ports maximum – one from each core/distribution switch.
• IST (InterSwitch Trunk) a proprietary bonding protocol between two core cluster switches that allows them to deploy SMLT or SLT topologies to edge/distribution switches. This is just an MLT which is used to bridge the VLANs between the two cluster switches. The IST also provides a transport for the two cluster switches to exchange ARP and FDB/MAC table information.

You can use MLT or DMLT between two switches in what I would refer to as a traditional trunking application. A Distributed MultiLink Trunk provides additional redundancy by spreading the links out across multiple line cards or switches in a stack depending on the switch model/configuration. An MLT/DMLT is Avaya’s proprietary equivalent to Cisco’s EtherChannel or PortChannel feature. An Avaya MLT or DMLT configuration can interoperate with Cisco’s EtherChannel or PortChannel configuration.

It’s important to point out that Avaya switches will (by default) only send out BPDU frames on the lowest number if index of a MLT or DMLT trunk. This can be overridden in newer software releases with the command “mlt 1 bpdu all-ports“. Cisco switches will send out BPDU frames on all ports in an EtherChannel or PortChannel configuration.

You can use SMLT or SLT between two cluster switches and a single edge/distribution switch or stack essentially creating a triangle topology without the need for Spanning Tree. Both links between the core and edge/distribution are actively forwarding traffic.  An SMLT/SLT is Avaya’s proprietary equivalent to Cisco’s Virtual PortChannel feature. When do you use one over the other, SMLT or SLT? The later software releases only allowed between 32 and 64 MLTs per switch. If you had more than 31 edge switches you would run out of available MLTs, so Avaya came up with SLT – you can have as many SLTs as you have ports in the switch. SMLT will allow you to bond between 2 and 8 ports into a single virtual trunk on each cluster switch while SLT is designed to allow two ports max (one per cluster switch).

It’s important to note that you can utilize LACP with MLT, DMLT or SMLT ports just a you can with PortChannel groups.

The majority of closets I deploy utilizing SLT in the cluster core although there are a few closets that require more than 2 x 1Gbps uplinks so for those we utilize a SMLT configuration allowing up to 16 x 1Gbps links between the core cluster switches and edge/distribution switches.

Spanning Tree Protocol and IST/SMLT

Avaya has not extended the functionality of the Spanning Tree Protocol to run over an IST/SMLT topology. You can’t run STP between your core cluster switches and your edge switch/stack. That doesn’t mean that we can abandon STP altogether. It’s critical that Spanning Tree be utilized on all the edge ports in FastStart (PortFast) mode to eliminate the possibility of anyone accidentally creating a loop between any two ports in the edge switch. I also recommend that BPDU filtering be enabled on all edge ports along with Broadcast and Multicast rate-limiting.

Virtual Link Aggregation Control Protocol (VLACP)

In an Avaya network there is a special secret sauce that helps to bring everything together providing timely failure detection and recovery in an MLT, DMLT, IST, SMLT and SLT topology. VLACP is a lightweight heartbeat protocol utilized between two Avaya switches to detect Layer 2 connectivity issues between two endpoints. The trick these days isn’t detecting a failure but knowing when to restore a failed path taking into account the time it takes to rebuild routing and forward tables. VLACP is an Avaya proprietary protocol so it will only work between two Avaya switches.

If you’d like to know more about VLACP or how to configure it you can read my article entitled, Is VLACP right for me?

Cheers!

References: Switch Clustering using Split Multi-Link Trunking (SMLT) with VSP 9000, ERS 8600/8800, 8300, and 5000 Technical Configuration Guide

Let’s assume that these switches are already setup in an IST pair (future post?) and that we want to add a new edge/closet switch to the network. We’ll utilize port 1/7 on both ERS 8600 switches to connect to ports 1/47 and 1/48 on the edge switch. The edge switch should be setup as an MLT. You can refer to this post for additional details regarding how to configure the edge switch.

Here’s a diagram of our example topology…

Step 1.

Let’s start configuring the ERS8600-A switch;

config ethernet 1/7 perform-tagging enable
config ethernet 1/7 untagged-frames-discard enable
config ethernet 1/7 default-vlan-id 200
config ethernet 1/7 cp-limit enable multicast-limit 7500 broadcast-limit 5000
config ethernet 1/7 enable-diffserv true
config ethernet 1/7 slpp packet-rx enable
config ethernet 1/7 slpp packet-rx-threshold 5
config ethernet 1/7 mstp cist forceportstate disable
config ethernet 1/7 mstp msti 1 forceportstate disable
config ethernet 1/7 smlt 107 create
config ethernet 1/7 vlacp  enable
config ethernet 1/7 vlacp  fast-periodic-time 500
config ethernet 1/7 vlacp  timeout short
config ethernet 1/7 vlacp  timeout-scale 5

Let’s break down those commands and review each;

• config ethernet 1/7 perform-tagging enable

This command will enable tagging to make the port an 802.1q trunk port. This will enable us to trunk multiple VLANs over the single interface, it will also preserve an Layer 2 QoS/CoS information.

• config ethernet 1/7 untagged-frames-discard enable

This command will discard any non 802.1q tagged frames that are received on the port. This can be a valuable defense measure in protecting your network. What would happen if the edge switch was accidentally factory reset with both uplinks still connected? A loop would result, however, with this feature all frames from the edge switch will be discarded until the switch is reconfigured.

• config ethernet 1/7 default-vlan-id 200

This command will set the PVID to our management VLAN. This value will only be considered if the port receives a frame which doesn’t have an 802.1q header and hence is missing the VLAN ID. The command “untagged-frames-discard enable” essentially negates this command but we set it anyway so we’re consistent in our configurations.

• config ethernet 1/7 cp-limit enable multicast-limit 7500 broadcast-limit 5000

This command will enable CP-Limit to protect the core network from too many Multicast or broadcast packets flooding the link. CP-Limit will shutdown the link to try and protect the core network. This is just one of many defense mechanisms available to help protect your network.

• config ethernet 1/7 enable-diffserv true

This command will enable DiffServ (Layer 3 QoS) on the switch port and set it for Trusted, so the switch will honor all DiffServ marked packets and give those packets the appropriate priority and queuing.

• config ethernet 1/7 slpp packet-rx enable
• config ethernet 1/7 slpp packet-rx-threshold 5

These commands will enable Simple Loop Protection Protocol (SLPP) to help detect any misconfiguration of the MultiLink trunks on the edge/closet switch.

• config ethernet 1/7 mstp cist forceportstate disable
• config ethernet 1/7 mstp msti 1 forceportstate disable

These commands will disable Multiple Spanning Tree Protocol (MSTP) no the switch ports. Spanning Tree is not compatible with Avaya’s Split Trunking Protocol since we are quite literally creating a loop in the physical topology. If this switch was running STP the command would like so, ethernet 1/7 stg 1 stp disable.

• config ethernet 1/7 smlt 107 create

Here’s the command that you’ve been waiting for … this command essentially creates a S-SMLT or Single Link Trunk (SLT). The  ID used in the connection needs to match the peer ERS 8600 switch.

Design note – in my networks I use numbers to denote the different IDFs or ICRs. I usually add 100 to those numbers for the SMLT ID and VLAN IDs. Since this is IDF #7 (or ICR #7) the SMLT ID is 100 + 7 = 107 and the VLAN for this closet will eventually be 107. If I was still using VRRP the VRRP ID would also be 107. You can use whatever number you’d like but they must match on the two ERS 8600s!

• config ethernet 1/7 vlacp  enable
• config ethernet 1/7 vlacp  fast-periodic-time 500
• config ethernet 1/7 vlacp  timeout short
• config ethernet 1/7 vlacp  timeout-scale 5

These commands enable VLACP on the port and utilize the recommended values from Avaya.

You should repeat the commands above in Step 1 on both Avaya Ethernet Routing Switch 8600s, substitute the appropriate port numbers and SMLT ID.

Design note – in my networks the edge/closet switches are still Layer 2 only so I perform all the routing in the core switches. I will usually have a “default” VLAN per edge/closet switch although I do have multiple VLANs that span multiple edge/closet switches.

Step 2.

With the port configured now we’ll build the VLAN that we’ll associate with most ports on the edge switch.

config vlan 107 create byport-mstprstp 1 name "10-1-112-0/23"
config vlan 107 add-mlt 1
config vlan 107 ports add 1/7 member portmember
config vlan 107 fdb-entry aging-time 21601
config vlan 107 ip create 10.1.112.1.1/255.255.254.0 mac_offset 0
config vlan 107 ip igmp proxy-snoop enable
config vlan 107 ip igmp snoop enable
config vlan 107 ip dhcp-relay enable
config vlan 107 ip ospf interface-type passive
config vlan 107 ip ospf enable
config vlan 107 ip rsmlt enable
config vlan 107 ip rsmlt holdup-timer 9999

Let’s break down those commands and review each;

• config vlan 107 create byport-mstprstp 1 name “10-1-112-0/23”

This command will create VLAN 107 and make it a port based VLAN with the name “10-1-112-0/23”. You might be asking what the mstprstp is… this specific switch I’m working with has been deployed with MSTP enabled. If you have a switch still using STP (default) then the command would look like so config vlan 107 create byport 1 name “10-1-112-0/23”

• config vlan 107 add-mlt 1

This command will add VLAN 107 to our IST which in this case happens to be MLT ID 1.

• config vlan 107 ports add 1/7 member portmember

This command will add VLAN 107 to port 1/7 which we are using to connect our edge/closet switch.

• config vlan 107 fdb-entry aging-time 21601

This command will set the default FDB aging time for all MAC information learned in this VLAN to 6 hours and 1 second. This is a best practice recommendation by Avaya to help reduce the ARP broadcast storms that can result when the FDB table expires a large number of entries which then in turn causes them to be removed from the ARP table causing the switch to re-ARP for them.

• config vlan 107 ip create 10.1.112.1.1/255.255.254.0 mac_offset 0

This command will configure a Layer 3 interface on VLAN 107 with the IP address of 10.1.112.1/23. Your mac_offset will differ depending on how many IP interfaces you already have deployed on your switch.

• config vlan 107 ip igmp proxy-snoop enable
• config vlan 107 ip igmp snoop enable

This command will enable IGMP snooping and proxy on the VLAN.

• config vlan 107 ip dhcp-relay enable
• config vlan 107 ip dhcp-relay create-fwd-path server 10.1.1.100
• config vlan 107 ip dhcp-relay enable-fwd-path server 10.1.1.100

These commands will enable DHCP relay on the VLAN, and forward all DHCP requests to 10.1.1.100.

• config vlan 107 ip ospf interface-type passive
• config vlan 107 ip ospf enable

This command will enable OSPF on the VLAN and will set it to passive (best practice for edge/closet VLANs).

• config vlan 107 ip rsmlt enable
• config vlan 107 ip rsmlt holdup-timer 9999

This command will enable RSMLT which replaces the VRRP functionality. We set the holdup-timer to infinity, we don’t want the ERS 8600 to stop accepting packets for it’s peer at anytime.

You should repeat the commands above in Step 2 on both Avaya Ethernet Routing Switch 8600s, substitute the appropriate IP address and ports.

Step 3.

There are a few items that we still need to take care of to round out the configuration.

We need to enable SLPP for VLAN 107;

• config slpp operation enable
• config slpp add 107

These commands will enable SLPP globally and will also enable SLPP in VLAN 107.

Step 4.

Here are some commands you can use to verify the configuration and operation.

You can check the SMLT table and verify that the trunk is configured as SMLT and operating as SMLT;

ERS-8610-A:5# show smlt info
================================================================================
Mlt SMLT Info
================================================================================
MLT   SMLT     ADMIN    CURRENT
ID    ID       TYPE     TYPE
--------------------------------------------------------------------------------
4     4        smlt     smlt
10    10       smlt     norm
15    15       smlt     norm

================================================================================
Port SMLT Info
================================================================================
PORT  SMLT     ADMIN    CURRENT
NUM   ID       TYPE     TYPE
--------------------------------------------------------------------------------
1/7   3        smlt     smlt
4/4   6        smlt     smlt

You can check the MLT table and verify that VLAN 107 is a member of MLT 1 (IST);

ERS-8610-A:5# show mlt info
================================================================================
Mlt Info
================================================================================
PORT    SVLAN  MLT   MLT        PORT         VLAN
MLTID IFINDEX NAME      TYPE    TYPE  ADMIN CURRENT    MEMBERS      IDS
--------------------------------------------------------------------------------
1   6144  MLT-IST      trunk   normal ist    ist      1/1,4/1,8/1       1 2 3 4 5 9 10 20 21 25 99 100 101 102 103 107 198 199 200

You can verify that the IST is up and operational between the two ERS 8600 switches;

ERS-8610-A:5# show mlt ist info
================================================================================
Mlt IST Info
================================================================================
MLT   IP                   VLAN     ENABLE   IST
ID    ADDRESS              ID       IST      STATUS
--------------------------------------------------------------------------------
1     10.1.100.2         100      true     up

You can check the state of VLACP on port 1/7 to confirm that VLACP is enable and up.

ERS-86010-A:5# show port info vlacp port 1/7
================================================================================
VLACP Information
================================================================================
INDEX ADMIN   OPER    PORT   FAST    SLOW    TIMEOUT TIMEOUT ETHER      MAC
ENABLED ENABLED STATE  TIME    TIME    TIME    SCALE   TYPE       ADDR
--------------------------------------------------------------------------------
1/7   true    true    UP    500     30000   short     5      0x8103    01:80:c2:00:11:00

You can check the SONMP topology tables to make sure you have the correct port(s).

ERS-8610-A:5# show sys topology
================================================================================
Topology Table
================================================================================
Local                                                                     Rem
Port  IpAddress       SegmentId MacAddress   ChassisType      BT LS  CS   Port
--------------------------------------------------------------------------------
0/0  10.1.1.1      0x000000  0004387xxxxx ERS8610          12 Yes HtBt  0/0
1/1  10.1.1.2      0x000101  000fcdfxxxxx ERS8610          12 Yes HtBt  1/1
1/7  10.1.255.20   0x00012f  0014c73xxxxx mBayStack5520-48T-PWR 12 Yes HtBt  1/47

Cheers!

This is really great document that helps to outline the best practices when deploying an IST/SMLT configuration within your network.

Here’s the summary of the document updates for March 2011;

March 2011 – Added recommendation to use MLT advance mode on edge Ayava stackable switches when used with ERS 8000 square/full mesh topologies for IP based traffic. Changed VSP CP Limit configuration from port level to MLT level. VRRP hold-down timers can be set in the ERS 5000 6.2 release.

It’s well worth the read in my opinion.

Cheers!

The document covers topics such as convergence between IP telephony and data networking, chassis versus stackable, Layer 2 versus Layer 3 at the edge, redundancy, high availability, clustering (IST/SMLT), two tier and three tier network designs, VLANs, Spanning Tree, Control Plane Rate Limit (cp-limit), Extended CP-Limit (ext-cp-limit), VLACP, SLPP, QoS, VRRP, RSMLT, ECMP, Multicast, EAPoL and the list goes on and on. And best of all they provide configuration examples for a large number of the scenarios which are always helpful.

A lot of the material I cover here in my blog is covered in this document. I’ll probably pull a few excerpts from this document over the next few months and make some posts out of it, expanding on some of the examples and filling in any unanswered blanks.

I’m impressed with effort that Nortel has made in trying to “get out the word”.  This is really a great tool for Nortel customers! Let’s hope that Avaya will allow these folks to continue with their success.

Oh behalf of all those Nortel customers out there let me say “Thanks!”

Cheers!

Example 2 – Ethernet Routing Switch 8600 to a set of HP GbE2c L2/L3 switches using LACP trunks with SMLT

As I said before a picture is worth a thousand words and can be very helpful in designing any network topology.

I’m going to skip the configuration of the two Nortel Ethernet Routing Switch 8600s since you can refer to the earlier post for an example of how to configure them. In this design we need to disable the virtual cross connect that exists between the A and B sides of the two HP GbE2c switches. Please note that I’m working with the HP GbE2c (C-Class enclosure) not the GbE2 (P-Class enclosure). There are some slight differences between the two. The virtual trunk ports between the A and B sides are on ports 17 and 18 so those ports need to be disabled in order to prevent a loop.

HP-GbE2c-A / HP-GbE2c-B
/c/port 17/dis
/c/port 18/dis

With the virtual trunk cross connects disabled we can now wire each switch independently to the upstream switch(s) which in this case happens to be two ERS 8600s. As is usual for me I’ll create a network management VLAN and place the IP interface of each GbE2c switch in that VLAN (VLAN 200).

HP-GbE2c-A / HP-GbE2c-B
/c/l2/vlan 200
/c/l2/vlan 200/ena
/c/l2/vlan 200/name "10-101-255-0/24"

Let’s add VLAN 200 to the two ports, 21 and 22, that we’ll be using to uplink to the 8600 switches. We haven’t yet enabled tagging so the switch will ask you if you’d like to change the PVID from VLAN 1 (default) to VLAN 200, you can safely answer yes to this question.

HP-GbE2c-A / HP-GbE2c-B
/c/l2/vlan 200/add 21
/c/l2/vlan 200/add 22

Let’s enable tagging on both uplink ports along with RMON and set the PVID just to be safe;

HP-GbE2c-A / HP-GbE2c-B
/c/port 21/tag ena
/c/port 21/pvid 200
/c/port 21/rmon e
/c/port 22/tag ena
/c/port 22/pvid 200
/c/port 22/rmon e

Let’s turn off Spanning Tree on the uplinks, we only want Spanning Tree local to the switch since SMLT will take care of providing the loop free topology.

HP-GbE2c-A / HP-GbE2c-B
/c/l2/stp 1/port 21/off
/c/l2/stp 1/port 22/off

Now it’s time to configure LACP and create the LAG (Link Aggregation Group). We’ll using LACP key 50 but you could use any admin key (number) so long as both ports are configured with the same admin key.

HP-GbE2c-A / HP-GbE2c-B
/c/l2/lacp/port 21/mode active
/c/l2/lacp/port 21/adminkey 50
/c/l2/lacp/port 22/mode active
/c/l2/lacp/port 22/adminkey 50

Here’s the special sauce that will work in combination with the NIC teaming software to fail over in the event of an upstream switch problem or an uplink problem where the GbE2c continues to function but there’s a problem upstream. This configuration will cause the GbE2c switch to disable (admin-down) the server switch ports in the event that the LACP group goes down. This will cause the NIC teaming configuration on the servers to fail-over to the standby NIC.

HP-GbE2c-A / HP-GbE2c-B
/c/ufd/on
/c/ufd/fdp/ltm/addkey 50
/c/ufd/fdp/ltd/addport  1
/c/ufd/fdp/ltd/addport  2
/c/ufd/fdp/ltd/addport  3
/c/ufd/fdp/ltd/addport  4
/c/ufd/fdp/ltd/addport  5
/c/ufd/fdp/ltd/addport  6
/c/ufd/fdp/ltd/addport  7
/c/ufd/fdp/ltd/addport  8
/c/ufd/fdp/ltd/addport  9
/c/ufd/fdp/ltd/addport 10
/c/ufd/fdp/ltd/addport 11
/c/ufd/fdp/ltd/addport 12
/c/ufd/fdp/ltd/addport 13
/c/ufd/fdp/ltd/addport 14
/c/ufd/fdp/ltd/addport 15
/c/ufd/fdp/ltd/addport 16

If you haven’t already let’s configure an IP address (for management) on VLAN 200;

HP-GbE2c-A
/c/l3/if 1/ena
/c/l3/if 1/addr 10.1.255.128
/c/l3/if 1/mask 255.255.255.0
/c/l3/if 1/broad 10.1.255.255
/c/l3/if 1/vlan 200

We need to use a different IP address for the B side switch on VLAN 200;

HP-GbE2c-B
/c/l3/if 1/ena
/c/l3/if 1/addr 10.1.255.129
/c/l3/if 1/mask 255.255.255.0
/c/l3/if 1/broad 10.1.255.255
/c/l3/if 1/vlan 200

As mentioned by a few other folks on this blog and in the forums this solution only provides an active/passive solution in terms of the NIC teaming configuration. This is because the GbE2c L2/L3 switches don’t support IST/SMLT technology. While this will only provide 1Gbps of bandwidth (2Gbps if you count full duplex) between the blade server and the network it will provide significant level of redundancy and high-availability. In this design the network is protected from a GbE2c switch failure, a Nortel Ethernet Routing Switch 8600 failure, and multiple uplink/downlink failures.

Please feel free to post comments and questions here about this post. Questions regarding specific configurations can be posted in the forums; http://forums.networkinfrastructure.info/nortel-ethernet-switching/

Cheers!

Example 2 – Ethernet Routing Switch 8600 to Ethernet Switch 5520 using LACP trunk with SMLT

As I said before a picture is worth a thousand words and can be very helpful in designing any network topology.

As with the previous example we’ll start with the Ethernet Routing Switch 8600s and then progress to the Ethernet Routing Switch 5520s. In this example we’ll need to configure two ERS 8600 switches, I’ll assume that you already have an IST (InnerSwitch Trunk) built and running properly.

Let’s start by configuring a MLT group the same way we did so in the previous example. The ERS8600-A switch first;

ERS8600-A
config mlt 15 create
config mlt 15 name "SMLT_LACP"
config mlt 15 lacp key 15
config mlt 15 lacp enable

Now the ERS8600-B switch;

ERS8600-B
config mlt 15 create
config mlt 15 name "SMLT_LACP"
config mlt 15 lacp key 15
config mlt 15 lacp enable

In this example I’ve chosen to connect the uplinks to port 2/17 on each switch. I’ve chosen to use the same ports on both switches only to make the configuration easier to understand for myself. I would use whatever ports I wanted on either switch so long as they are all running at the same speed. In this case the ports are both 10/100Mbps ports and will auto-negotiate to 100Mbps with the MDI-X feature of the ERS 5520 switch.

I’ll enable tagging (802.1q) just like I did in my previous example and I’ll remove VLAN 1 and add VLAN 99. Outside of this example you would just add whatever VLANs you’ll be extended to the edge switch.

ERS8600-A
config ethernet 2/17 perform-tagging enable
config vlan 1 ports remove 2/17
config vlan 99 ports add 2/17

Now the ERS8600-B switch;

ERS8600-B
config ethernet 2/17 perform-tagging enable
config vlan 1 ports remove 2/17
config vlan 99 ports add 2/17

Next we’ll enable LACP on the specific ports and group them using the same admin key;

ERS8600-A
config ethernet 2/17 lacp key 15
config ethernet 2/17 lacp aggregation true
config ethernet 2/17 lacp timeout short
config ethernet 2/17 lacp enable

Now the ERS8600-B switch;

ERS8600-B
config ethernet 2/17 lacp key 15
config ethernet 2/17 lacp aggregation true
config ethernet 2/17 lacp timeout short
config ethernet 2/17 lacp enable

Now because we’re going to be running in an SMLT configuration we need to make a few global changes. We need to enable LACP globally, but we also need to make sure that both switches use the same LACP identifier when communicating with the edge switch. This is necessary so the edge switch won’t know that it’s actually connected to two different switches upstream. If the LACP identifiers didn’t match between the two ERS8600 switches the edge switch would become confused.

ERS8600-A
config lacp smlt-sys-id 00:01:81:28:84:00
config lacp enable

Now the ERS8600-B switch;

ERS8600-B
config lacp smlt-sys-id 00:01:81:28:84:00
config lacp enable

We need to configure the MLT to operate in an SMLT configuration. We also need to make sure that any VLANs we are extending to the edge switch are also bridged across the IST between the two ERS 8600 switches. In this example I’m extending VLAN 99 so I need to add VLAN 99 to the IST which happens to be MLT 1.

ERS8600-A
config mlt 15 smlt create smlt-id 15
config vlan 99 add-mlt 1

Now the ERS8600-B switch;

ERS8600-B
config mlt 15 smlt create smlt-id 15
config vlan 99 add-mlt 1

That’s all the commands required for the two ERS8600 switches.

With that said there are some best practices that should be applied to all downlinks when utilizing SMLT.

While I left this out of the previous example these settings are applicable to both examples.

Let’s make sure that we enable CP-LIMIT which will shutdown the port if the switch receives too many broadcast or multicast frames per second. While some users don’t like this feature it’s better to cut off an offending closet than loose an entire network due to a loop or misconfigured switch. A word of warning here! You do not want CP-LIMIT enabled on any ports used in your IST, you also don’t want it enabled on the uplinks of any ERS8600 switches that reside at the edge as they might cut themselves off from the network. Instead enable it in the core on the downlinks to the edge switches and closet switches.

ERS8600-A
config ethernet 2/17 cp-limit enable multicast-limit 2500 broadcast-limit 2500

Now the ERS8600-B switch;

ERS8600-B
config ethernet 2/17 cp-limit enable multicast-limit 2500 broadcast-limit 2500

Another feature that helps protect the network is SLPP (Simple Loop Protection Protocol). In my opinion this feature is a must for any serious network. I can’t tell you how many times this feature has saved the networks I manage today. This feature will detect a misconfigured MLT/LACP at the edge switch and shutdown one of the downlink ports to preventing a loop. With SLPP you need to pay attention to the threshold setting. You want different thresholds between the two ERS8600 switches so that only one uplink gets shutdown.

ERS8600-A
config slpp add 99
config slpp operation enable
config ethernet 2/17 slpp packet-rx-threshold 50
config ethernet 2/17 slpp packet-rx enable

Now the ERS8600-B switch with a threshold of 5;

ERS8600-B
config slpp add 99
config slpp operation enable
config ethernet 2/17 slpp packet-rx-threshold 5
config ethernet 2/17 slpp packet-rx enable

That’s it for the two ERS8600 switches.

I’m literally going to cut and past the configuration of the ERS5520 from the previous example as it should be identical.

vlan ports 33,34 tagging tagAll

Let’s add VLAN 99 to the ports, I’ve already created the VLAN ahead of time.

vlan members add 99 33,34

Now we just need to configure the LACP parameters for each port and then enable LACP.

interface fastEthernet 33-34
lacp key 13
lacp mode active
lacp timeout-time short
lacp aggregation enable
exit

Hopefully that’s been helpful!

Cheers!

We’ve been using VRRP in conjunction with SMLT to make sure that either core ERS 8600 switch in a cluster could assume the default gateway for any specific VLAN should the other switch fail. While I’m not at liberty to get into specifics I can tell you that we recently ran straight into a wall with our VRRP configuration. We had about 60+ VLANs (port based) on an ERS 8600 switch cluster where the VRRP ID for each VLAN was set to “1”. While Nortel had previously said this configuration was supported, we ran into significant issues testing some new software code for the 8600.

In short Nortel is now advising that you MUST have unique VRRP IDs. Nortel recommends as best practice that you use the VLAN ID but that can be problematic since valid values for the VLAN ID are between 1..4096 and valid values for the VRRP ID are between 1..512. In my case the VLAN IDs were between 1 and 200 so I was able to match them up.

config vlan <VLAN ID> ip vrrp <VRRP ID> address <IP ADDRESS>
config vlan <VLAN ID> ip vrrp <VRRP ID> backup-master enable
config vlan <VLAN ID> ip vrrp <VRRP ID> enable

ERS8600:5# config vlan 1 ip vrrp 1 address 10.10.1.1
ERS8600:5# config vlan 1 ip vrrp 1 backup-master enable
ERS8600:5# config vlan 1 ip vrrp 1 enable
ERS8600:5# config vlan 2 ip vrrp 2 address 10.10.2.1
ERS8600:5# config vlan 2 ip vrrp 2 backup-master enable
ERS8600:5# config vlan 2 ip vrrp 2 enable
ERS8600:5# config vlan 9 ip vrrp 9 address 10.10.9.1
ERS8600:5# config vlan 9 ip vrrp 9 backup-master enable
ERS8600:5# config vlan 9 ip vrrp 9 enable

I believe Nortel is now recommending RSMLT in place of VRRP which we’ll be using going forward. If you’ve been using a VRRP ID of 1 in every VLAN you might want to consider changing your configuration. I hope to write a post about RSMLT in the near future detailing how to configure it and the advantages using RSMLT over VRRP.

A personal note of Thanks to Richard M. and Roger G. from Nortel for their help and assistance in troubleshooting this problem.

Cheers!

The Nortel Ethernet Routing Switch 8600 has a feature called ping snoop that can be used to determine the specific path that specific IP traffic takes over an MLT, DMLT or SMLT path. Ping snoop works by enabling a filter that copies the ICMP messages to the CPU. The CPU then monitors the ICMP stream and outputs messages on the console indicating what ports are being traversed by the IP traffic.

There are different commands depending on the type of IO modules that are involved.

With non-R modules;

config diag ping-snoop create src-ip 30.30.30.0/24 dst-ip 30.30.30.0/24
config diag ping-snoop add-ports 1/47,2/1
config diag ping-snoop enable true
config log screen on

With R modules;

config filter acl 4096 port add 1/2
config filter acl 4096 enable
config filter acl 4096 ace 1 create name echo_reply
config filter acl 4096 ace 1 ip src-ip eq 10.119.255.20/32
config filter acl 4096 ace 1 ip dst-ip eq 10.101.241.25/32
config filter acl 4096 ace 1 protocol icmp-msg-type eq echoreply
config filter acl 4096 ace 1 enable
config filter acl 4096 ace 2 create name echo_request
config filter acl 4096 ace 2 ip src-ip eq 10.101.241.25/32
config filter acl 4096 ace 2 ip dst-ip eq 10.119.255.20/32
config filter acl 4096 ace 2 protocol icmp-msg-type eq echo-request
config filter acl 4096 ace 2 enable
config log screen on

In the above examples you need to substitute the appropriate IP addresses and switch ports.

I’ve used the ping snoop feature on numerous occasions to isolate the specific uplink that a TCP/UDP conversation was utilizing when traversing two switches that have multiple uplinks between each other [configured as MLT/DMLT/SMLT uplink].

Here’s a sample output from a Nortel ERS 8600 v4.1.1 switch;

sw-ccr-8600:5# CPP Task=tMainTask CPU5 [12/11/07 07:36:25] CPU INFO ICMP Reply received on port 8/14 with Src=10.124.240.32 Dst=10.124.240.20
sw-ccr-8600:5# CPP Task=tMainTask CPU5 [12/11/07 07:36:26] CPU INFO ICMP Reply received on port 8/14 with Src=10.124.240.32 Dst=10.124.240.20
sw-ccr-8600:5# CPP Task=tMainTask CPU5 [12/11/07 07:36:27] CPU INFO ICMP Reply received on port 8/14 with Src=10.124.240.32 Dst=10.124.240.20
sw-ccr-8600:5# CPP Task=tMainTask CPU5 [12/11/07 07:36:28] CPU INFO ICMP Reply received on port 8/14 with Src=10.124.240.32 Dst=10.124.240.20

I might be wrong about this but I believe the ping snoop feature only works on ingress packets (packets that are ingressing into the IO module/port you have configured for ping snoop).

Cheers!

Simple Loop Prevention Protocol (SLPP) provides active protection against Layer 2 network loops on a per-VLAN basis. SLPP uses a lightweight hello packet mechanism to detect network loops. SLPP packets are sent using Layer 2 multicast and a switch will only look at its own SLPP packets or at its peer SLPP packets. It will ignore SLPP packets from other parts of the network. Sending hello packets on a per VLAN basis allows SLPP to detect VLAN based network loops for un-tagged as well as tagged IEEE 802.1Q VLAN link configurations. Once a loop is detected, the port is shutdown. The SLPP functionality is configured using the following criteria:

• SLPP TX Process – the network administrator decides on which VLANs a switch should send SLPP hello packets. The packets are then replicated out all ports which are members of the SLPP-enabled VLAN. It is recommended to enable SLPP on all VLANs.
• SLPP RX Process – the network administrator decides on which ports the switch should act when receiving an SLPP packet that is sent by the same switch or by its SMLT peer. You should enable this process only on Access SMLT/SLT ports and never on IST ports or Core SMLT/SLT ports in the case of a square/full mesh core design.
• SLPP Action – the action operationally disables the ports receiving the SLPP packet. The administrator can also tune the network failure behavior by choosing how many SLPP packets need to be received before a switch starts taking an action. These values need to be staggered to avoid edge switch isolation – see the recommendations at the end of this section.

Loops can be introduced into the network in many ways. One way is through the loss of an MLT configuration caused by user error or malfunctioning equipment. This scenario may not always introduce a broadcast storm, but because all MAC addresses are learned through the looping ports, does significantly impact Layer 2 MAC learning. Spanning Tree would not in all cases be able to detect such a configuration issue, whereas SLPP reacts and disables the malfunctioning links, limiting network impact to a minimum. The desire is to prevent a loop from causing network problems while also attempting to not totally isolate the edge where the loop was detected. Total edge closet isolation is the last resort in order to protect the rest of the network from the loop. With this in mind, the concept of an SLPP Primary switch and SLPP Secondary switch has been adopted. These are strictly design terms and are not configuration parameters. The Rx thresholds are staggered between the primary and secondary switch, therefore the primary switch will disable an uplink immediately upon a loop occurring. If this resolves the loop issue, the edge closet still has connectivity back through the SLPP secondary switch. If the loop is not resolved, the SLPP secondary switch will disable the uplink and isolate the closet to protect the rest of the network from the loop.

I’ve deployed SLPP at one site with with a two tier network design utilizing SMLT with an IST core. It’s very important to remember that SLPP operates per VLAN id so you need to take that into consideration. You also don’t want to overload your switch fabric (CPU) by enabling SLPP on every VLAN, especially if you have a large number of VLANs.

Here’s an example of how to deploy SLPP between two core ERS 8600s (switch cluster).

ERS 8600 Core Switch A

ERS-8610:5# config slpp add 200
ERS-8610:5# config slpp operation enable
ERS-8610:5# config ethernet 1/1-1/8 slpp packet-rx enable
ERS-8610:5# config ethernet 1/1-1/8 slpp packet-rx-threshold 5

ERS 8600 Core Switch B

ERS-8610:5# config slpp add 200
ERS-8610:5# config slpp operation enable
ERS-8610:5# config ethernet 1/1-1/8 slpp packet-rx enable
ERS-8610:5# config ethernet 1/1-1/8 slpp packet-rx-threshold 50

This will cause both core ERS 8600 switches to transmit SLPP PDUs on VLAN 200. They will watch for those PDUs to return on port 1/1-1/8. It’s important in the example above to point out the different thresholds. You don’t want both core ERS 8600 switches cutting off both uplinks to the edge closets. Hence the core A switch will admin-down any port where it receives 5 of it’s own SLPP PDU packets. The core B switch will admin-down any port where it receives 50 of it’s own SLPP PDU packets. This configuration will generally disable one of the uplinks from the switch cluster (removing the loop) but won’t leave the edge switch disconnected from both core ERS 8600 switches.

Cheers!