About the only problem we have had is some sort of bug with DHCP Guard which caused no end of problems. The solution was to disable DHCP Guard, but long-term, we need to figure this problem out.

But not all is lost. The latest ESM software release is very nice and fixed some nagging bugs. CS1000 6.0 looks very solid, and our SRGs are running with no problems at all. We are very committed to Nortel over the next 5-7 years, but we will continue to move with deliberation when we touch any of the core switches.

We were not experiencing actual LANE lockups on 81612XLRS with 5.0.1. The group of four 10gig ports in LANE one would stop transmitting packets, but was still receiving them. Links up. No errors. We were lucking VLACP logically brought the affected ports down (although they were physically still up) and the SMLT peer took over. Without VLACP the traffic would have been black holed.

Are you running SLPP. In 5.0.1 with SLPP enabled the cpu buffer (99%) filled up on one side of the cluster causing the peer CPU to spike at 100%. This caused instability in the IST. If you globally disable SLPP, the problem goes away. This was due to the SLPP MAC address being placed in the forwarding table twice.

We tested that this is fixed in 5.1.1.1 and 5.0.5

Was fixed in 4.1.8.3 How can you manage so many versions of code? We’re always on the same code base with clusters and even stand alones now.

We only use VRRP when we have to. We use RSMLT which helps when SMLT converges when an 8600 in the cluster is coming back up. The problem is you can only have 32 SMLTs per RSMLT instance. We found this out the hard way. If you go over 32 it cause instability in the IST and things go nuts. A couple of VLAN (like the managment one) have way more than 32 SMLT instances.

We do have unique VRRP IDS across the switch.

Hi Michael,

No, they are not runnig 5.1.1.1 code with 8612XLRS modules. No LANE lock-ups until now. They are running 5.1.1.1 code for more than 2 weeks without problems.

Concerning DHCP Relay Agents on a cluster running 5.1.1.1 code: This IP should not be the same as configured for Virtual IP Address (VRRP). It will start a broadcast storm (DHCP Offers packets) on the IST link!!

Best Regards!

Hi Mark,

I’m happy that another Nortel customer is running 5.1.1.1 software without any issues. Also happy to hear that you haven’t experienced any LANE lockups on 5.1.1.1 software.

That’s an interesting comment regarding the SuperMezz cards and SLPP. Do you use VRRP in your configuration? Do you have unique VRRP IDs across the switch cluster? I ask because there is a known issue that we discovered last May that may be related. In short you need to have unique VRRP IDs across the switch, you can’t use VRRP ID 1 for VLAN 1 and VRRP ID 1 for VLAN 2, VRRP ID 1 for VLAN 3, etc.

Thanks for the comment!

The Cisco VSS solution has worked excellently in our environment. Unlike the ERS8600 it operates 100% as one logical switch similar to a stack of Nortel 5510s or Cisco 3750s. We have redundant fiber to all of our closets and have them setup as Machine EtherChannels (MECs) which is similar to SMLT, but in my experience works a lot better. There is only one active supervisor between the two chassis and the active/active detection is done both by the VSS (SMLT) links as well as by a dedicated heartbeat link.

Hi Udo,

Thanks for the comment. It would seem that a lot of folks are happy with 5.1.1.1 software, however, I will warn you that I’ve heard about issues with the 8612XLRS modules. You might want to contact Nortel support to inquire about any known issues with the 8612XLRS modules if you haven’t already installed them.

Cheers!

Hi Silvermachine,

I’m curious if your customers are running configuration with any 8612XLRS modules? I’ve heard some not to pretty feedback from a very large Nortel customer that they won’t allow 5.1.1.1 software back into their organization because it caused so many issue similar to LANE lock-ups. That same customer is now running 5.0.5 and has been very pleased with that software release and the 8612XLRS modules.

That’s an interesting comment concerning DHCP relay agents and NIC teaming configurations.

Thanks for the comment!

Hi Ben,

How has the Cisco VSS solution worked out for you? I have a location with an existing Cisco Catalyst 6509 where I need to add a redundant core switch and VSS is an option.

The Nortel 8600s have worked very well, it’s just frustrating to hit the occasional operational snags every once and a while. The hardware has been very reliable but as I’ve mentioned the software releases ever since v4.1.5.4 have been terrible, especially if you are running in an IST/SMLT configuration. While we don’t run every protocol possible we do drive our cores pretty hard, 100+ VLANs, IST, SMLT, RSMLT, PIM-SM, OSPF, BGP, SYSLOG, etc.

I’ll soon be cutting my teeth on the Cisco’s Nexus 7000 switch so you might hear my screaming about them pretty soon.

Thanks for the comment!

Hi Gby,

I would generally agree with you concerning the reliability and stability of the platform, however, the recent software releases have left serious questions in the minds of customers like myself. I understand from Nortel that a majority of the code changes behind their IST/SMLT re-design were to accommodate larger networks and provide better scalability in larger configurations. I can understand their motivations but basic stability and availability is paramount for almost any network these days. I’m still happy with v4.1.8.3 but it seems there’s always some event that calls my faith into question.

Just as a note I’ve heard a few comments about LANE lockups on the 8612XLRS modules when running 5.1.1.1 software

Thanks for the comment

Hi Bylie,

Isn’t it frustrating how the switch can sometimes go half in the can. I would much rather prefer it went completely belly up, this way the redundancy could kick in. I must admit that I’m probably being a little harsh on Nortel. I manage a fairly large network so the law of averages dictates that I’m statistically inclined to experience more problems just because I have more equipment where problems can develop.

I have heard really good things about the v5.0.5 software release from a number of people now, so you might want to consider v5.0.5 as an option.

Thanks for the comment!

We went to the 5.x code stream just about a year ago in order to bumb our sever farm backhauls up to 10GigE with 5600 TOR and multiple 8612XLRS cards. Supermezz cards caused us many critical failures.

Our advice, if you are on 5.x code and you are seeing cpu lockups or IST issues, disable SLPP and the SuperMezz cards. To this day we have not re-enabled the supermezz cards.

Mark

we use 2 ERS86xx in core without using SMLT but using IST. Every Switch at the Campus have just one uplink (no SMLT are used yet).

Three Month ago we update from 4.1x to 5.11 because in future we have to use some 8612LXRS Cards.

In Software 5.11 we have big trouble with the Arp-Table.
Some Hosts were unable to connect; some were able (all in the same VLAN). That was Hell!
The only way to fix this bug, was to switch off the IST between the ERS8600 (a big Thanks to Nortel for this very fast Support!)

Last week we upgrade from 5.11 to 5.111.
Now it looks like the bugs are gone away (IST are running)

Udo

Then it seems that this release has some issues that others had not
Of course recommandation from Nortel is to go on 5.x to correct these bugs
If release 5.1.x, as you state, is going to be suppressed by Nortel from Website, I suspect a great problem, in code quality since 6months … Something related to their capability to provide on-going support while beeing bankrupted ?
but FYI I have just checked and release 5.1.1 is still available for download

Gby