Ethernet Routing Switch 8600 Software Release v7.1.3.2

Avaya has released software v7.1.3.2 for the Ethernet Routing Switch 8600/8800.

• Every 5 seconds, on a timer, the CPU sends the clock time to all the line cards. A timer was created each time a card came online, so when there are multiple IO cards, multiple messages were sent every 5 seconds to all cards. Eventually a lockup resulted, and when it was detected by the CPU, the chassis was reset. (wi00996291)

Please refer to the release notes for all the details.

Motorola RFS7000 WiNG v5.2.21 Software Release

You may also want to review software release 5.2.3 and 5.2.4 before deciding to check out software release 5.2.21 which was intended to resolve several MESH issues with the AP7131.

RFS Controllers with WiNG v5.2.21 can adopt and provision the following 802.11n and legacy Access Points:

Dependent Access Points:

• AP621
• AP650
• Legacy: AP300

Adaptive/ Independent Access Points:

• AP6511
• AP6521
• AP6532
• AP7131 (Including the D-mode SKUs)
• AP7161

You should check over the release notes for all the details.

Cheers!

This release hints at being the last release in the 5.1 software branch since 5.1 was MD’d in December 2011. The 5.1 software branch is officially supported until 2016.

Please refer to the release notes for all the details.

Cheers!

Definitions

Now before I get started lets define some basic terms;

• Access port is defined as a port belonging to a single VLAN

• Trunk port as defined in Wikipedia is a port designed to carry multiple VLANs through a single network link through the use of a “trunking protocol”. To allow for multiple VLANs on one link, frames from individual VLANs must be identified. The most common and preferred method, IEEE 802.1Q adds a tag to the Ethernet frame header, labeling it as belonging to a certain VLAN. Since 802.1Q is an open standard, it is the only option in an environment with multiple-vendor equipment.

So by it’s definition an access port can only belong to one VLAN while a trunk port can belong to multiple VLANs.

It’s important to distinguish that we’re talking about single ports. A trunk group or trunk port group is made up of multiple ports which are combined into a single virtual port. Protocols such as MultiLink Trunking (Avaya), EtherChannel (Cisco) and LACP provide the ability to combine multiple trunk ports into a single virtual interface providing redundancy and additional bandwidth.

Basic Examples

In general the majority of edge switch ports will be configured as access ports. Any port used to connect a personal computer, laptop, server, printer, etc will be configured as an access port. Any port that connects to another switch will be configured as a trunk port.

Complex Examples

With the advent of virtualization VMware servers are often configured and connected to trunk ports. Where as servers would have traditionally been connected to access ports they can also be connected to trunk ports depending on their configurations. The advent of Voice Over IP (VoIP) to the desktop has also had an impact on how edge switches are configured when the desktop or laptop is connected to the IP phone which is in turn connected to the edge switch. I’ll cover that topic in more detail later on.

Avaya Ethernet Routing Switches

Ethernet Routing Switch 2500, 4000, 5000 Series

The Avaya Ethernet Routing Switch 2500, 4000 and 5000 series switches currently offer the following options.

• tagAll – sets the port as a trunk port tagging all frames with an 802.1Q header as they egress the port.
• untagAll – sets the port as an access port stripping all 802.1Q headers as they egress the port.
• tagPvidOnly – sets the port as a trunk port but only adds 802.1Q headers for the PVID VLAN as they egress the port.
• untagPvidOnly – sets the port as a trunk port but only adds 802.1Q headers for every VLAN other than the PVID VLAN as they egress the port.

What is the PVID? The PVID is the Default VLAN ID configured for that specific port. In a typical configuration where the port is an access (untagAll) port the PVID will be set to that VLAN automatically by the switch. In a trunk port configuration the PVID will be used to determine which VLAN to bridge any received untagged frames to if DiscardUntaggedFrames is not enabled. It’s recommended to enable DiscardUntaggedFrames on any port configured as a trunk (tagAll) port to avoid any potential configuration issues which might lead to a loop and a network outage. It’s also a best practice to configure the PVID on all trunk (tagAll) ports with the VLAN ID of your management VLAN.

Ethernet Routing Switch 1600, 8600, 8800 Series

You’ll notice on the Ethernet Routing Switch 1600, 8600 and 8800 series that the options are slightly different but achieve the same outcome.

• PerformTagging (Checked) – sets the port as a trunk port tagging all frames with an 802.1Q header as they egress the port.
• PerformTagging (Unchecked) – sets the port as an access port stripping all 802.1Q headers as they egress the port.

Additional options include DiscardTaggedFrames, DiscardUntaggedFrames and UntagDefaultVlan. These options can be used to achieve the same results as with the Avaya Ethernet Routing Switch 2500, 4000 and 5000 series switches with the exception of tagPvidOnly.

Is the PVID equivalent to the native vlan command in Cisco switches? It is if untagPvidOnly/UntagDefaultVlan is enabled. The PVID (DefaultVlanId) by itself only acts on untagged received frames. The untagPvidOnly/UntagDefaultVlan option acts on transmitted frames and so the combination of the two equates to the “switchport trunk native vlan #” on a Cisco switch.

It’s also important to point out that Avaya only supports 802.1Q tagging. So while Cisco supports ISL and 802.1Q there is no Avaya command similar to “switchport trunk encapsulation dot1q” since this is the default behavior with Avaya switches.

IP Telephony

There are some special considerations when desktops and laptops are physically connected to the PC port on back of an IP phone and then the IP phone is cabled to the edge switch.  In this scenario the common approach is to tag the voice VLAN while leaving the data VLAN untagged. Why? It’s important that we separate the voice traffic from the data traffic so we utilize two different VLANs, one VLAN will carry the voice traffic while one VLAN will carry the data traffic destined to the desktop or laptop. The desktop or laptop probably won’t be configured for 802.1Q tagging so it won’t understand an 802.1Q tagged frame. We need to guarantee that any frames being delivered to the PC port on the back of the IP phone are untagged, if they aren’t the laptop or desktop will just discard the frame. The IP phone will tag the voice frames with an 802.1Q header so the switch will properly bridge those frames to the voice VLAN. In this scenario we need to utilize the untagPvidOnly option in combination with configuring the PVID (DefaultVlanId) as the data VLAN. This way the voice VLAN will be tagged with an 802.1Q header so the phone understands it and the data VLAN will be untagged so the desktop or laptop understands it. The IP phone will be configured with the Voice VLAN ID so it knows which ID to use when communicating with the Call Server and Media Gateways.

Cheers!

New Features;

• 8812XL SPF+ I/O Module
• SPBM IEEE 802.1aq standards compliance
• VLACP flag detection and damping
• 8895 SF/CPU compact flash compatibility with Windows PC
• Fabric Memory Full Handling
• HTTPS configurable port
• CFM enhancements
• Dynamic VLAN addition to LACP-enabled MLT (LAG)
• EDM enhancements
• EDM support for tri-speed copper SPF

Resolved Issues;

• wi00564787 Packets processed by the CP egress through modules with an IP checksum of 0xffff instead of 0x0000.
• wi00853466 Cannot enable the VLACP hold timer with EDM.
• wi00876718 No backtraces in SMP log file after the switch crashed.
• wi00936425 Remote Fault Indication (RFI) is not reporting the correct operational status on 8634XGRS 10 Gbps ports.
• wi00937554 RSMLT peer IP is not reachable when one leg of the SMLT is down in a square configuration.
• wi00940800 OSPF Hello traffic sourced from the CPU should be marked as qos=7.
• wi00940805 OSPF traffic sourced from CPU should be marked as qos=7.
• wi00940810 IPFIX traffic sourced from CPU should be marked as qos=3.
• wi00940880 Disabling Autonegotiation on Gigabit-only capable ports result in a boot loop.
• wi00941200 Static BFD sessions do not come up after HA failover.
• wi00941202 ACLI allows you to configure a BGP redistribution from one VRF to another VRF.
• wi00941225 Gig ports on 8634XGRS module are not sending Remote Line Fault (RLF) clear signal when the ERS is powered down and then powered back up.
• wi00941227 Gig ports on 8648GBRS module are not sending Remote Line Fault (RLF) clear signal when the ERS is powered down and then powered back up.
• wi00944334 When a link flaps, there is no log message to indicate which side is causing the link to flap or why.
• wi00946689 After configuration and a reboot, LACP interfaces come up operationally different than administratively configured.
• wi00947146 Entering configure ip more-specific-non-local-route enable on a PIM router prevents multicast flow from reaching the receiver.
• wi00949429 After disabling and enabling SMLT links, duplicate packets appearing in the LACP diamond SMLT.
• wi00950141 ICMP, DHCP, Telnet, SNMP, ARP Reply traffic sourced from the CPU should not be marked as qos=7.
• wi00950143 SSH traffic sourced from CPU should be marked as qos=3.
• wi00950871 CPU to take I/O modules offline when certain error thresholds are met.
• wi00951489 IPv6 DiffServ packets have corrupt flow label and payload length fields.
• wi00952222 Port speed (HighSpeed Value) is reported for the next port instead of the current port.
• wi00957781 IST drops when an R or RS module containing a designated port for the IST MLT is removed from the switch.
• wi00958033 Unable to create IPX protocol based VLANs.
• wi00959042 Access policy configuration is lost during upgrade from 5.x to 7.1.1.0 when running ACLI.
• wi00959365 CFM fails when SPB IP is enabled and an incorrect CLIP address is incorrectly entered.
• wi00959844 L2 Trace route for CFM – CMAC does not work when one of the SPBM core switches is restarted and the adjacency is down.
• wi00962669 8648GTRS modules displaying the following error message: CPU5 [07/30/11 17:10:05] KHI WARNING Slot 1 Right Lane is experiencing Ingress RSP Errors – CIF Parity Error, CIF Protocol Error
• wi00962958 OSPF MD5 authentication key not saved in Ospf_md5key.txt file for VLAN 3993.
• wi00968188 EDM allows you to copy the clilog.txt file.
• wi00974401 IS-IS IP redistribution to aggregate IP routes into a summarized network does not work.
• wi00974403 When there are two networks with the same network address but with different masks, the network address with least specific mask is not redistributed.

If you utilize a lot of LACP/LAGs you’ll be happy that you can finally add/remove VLANs without disabling LACP, Dynamic VLAN addition to LACP-enabled MLT (LAG).

Please refer to the release notes for all the details.

Cheers!

Correction: As pointed out by Masch below in the comments you do not need an Advanced or Premium License to run SMLT on the Ethernet Routing Switch 8600. You do need an Advanced License to run SMLT on the Ethernet Routing Switch 8300. I’m not really sure why I was under that impression or if I mixed the ERS8300 and ERS8600 up.

Once you purchase an Advanced License from Avaya (or a reseller) you’ll receive a certificate with a license authorization code. You need to take that code along with the MAC address of the Ethernet Routing Switch 8600 and go to the Avaya eLicensing Portal. The portal is a little different from most sites in that you don’t create an account but rather you create a license bank securing it with a password. You upload your license authorization code to your license bank and then from the license bank you generate a license file by entering the MAC address of the switch. You then download the license file that you just generated and upload that file to your switch.

You can place the license file on your TFTP server and then download it from there or you can upload it directly using FTP (assuming you have it enabled). In the example below I’ll place the file on my TFTP server and copy the license file down from there to the Ethernet Routing Switch 8600.

ERS-8610:5# copy l10.1.1.1:/sw8600r1-lic.lic /flash/license.lic
Device /flash has 15101952 bytes of free

With the license file uploaded let’s try to load the license into the switch;

ERS-8610:5# config load-license
License File does not exist
License File does not exist
License File does not exist

You need to make sure that the license file is located at /flash/license.dat. In this example I uploaded it to the wrong filename (license.lic) so I’ll rename it now with move command;

ERS-8610:5# mv /flash/license.lic /flash/license.dat

Let’s try that again;

ERS-8610:5# config load-license

Now we can check to see that the license is installed properly.

ERS-8610:5# show license all

License file name : license.dat
License Type : ADVANCED
MD5 of Key : 6d97e0c5 f74a9540 xxxxxxxx 570b7512
MD5 of File : 9ee16adc 97c3394b xxxxxxxx abf81e72
Generation Time : 2011/12/06 06:17:26
Expiration Time :
Base Mac Addr : 00:1d:42:xx:xx:xx
flags : 0x00000022 SITE MEMO
memo :
Advanced License

Now if you have multiple CPU/SFs you need to copy the license file to the standby CPU/SF. If you have savetostandby enabled in your boot.cfg file then all you need to-do is to save the configuration. If you don’t have savetostandby enabled you need to manually copy the license file to your standby CPU/SF (copy /flash/license.dat 127.0.0.6:/flash/license.dat if slot 6 was your standby CPU/SF).

ERS-8610:5# save config
Save config to file /flash/config.cfg successful.
Save license to file license.dat successful.
Save to slave file /flash/config.cfg successful.
Save license file license.dat to standby successful.

Cheers!

Please refer to the release notes for all the details.

Cheers!

PS: Thanks to @Telair for the heads-up!

Upon inserting the 8648GBRS module in slot 9 we received the following alarm from the console;

CPU5 [10/18/11 12:48:29] SNMP INFO Chassis running on low fan cooling.Please change the fan. Note that if the switch is booted with fan check enabled, then to power up,
1. R module requires a fan (regular or high speed) in running state.
2. RS module requires a high speed fan in running state.
CPU5 [10/18/11 12:48:29] HW ERROR Card in Slot=9, Type=8648GBRS, not powered on due to fan requirements.
CPU5 [10/18/11 12:48:48] HW INFO Card removed: Slot=9 Type=8648GBRS

I immediately knew what the problem was, I had overlooked the high-speed fan tray requirement for any RS modules. We have legacy chassis’s at this location so I knew that slots 1 and 10 couldn’t be populated by an R/RS module, I also knew the power supplies had already been upgraded to 8005PS’s so there was adequate power. However, I overlooked the cooling requirement so today I had to order a bunch of 8010CMHS fan trays.

Cheers!

• Broadcast and multicast traffic that should be flooded within a VLAN is no longer forwarded out the same MLT as it arrived if the ingress port is in the port range 41-48 on an 8648GTR or 8648GTRS card. (wi00867614)

I stumbled across this problem while reviewing the port utilization (MRTG) for a number of switch ports. The ifInOctets and IfOutOctets for the ports didn’t look right so after running a packet capture from the standby CPU/SF and reviewing the trace we could see Multicast frames that were being sourced from the closet being sent back out to the closet.

I’m still working through a high CPU utilization issue with Avaya on an ERS 8600 IST/SMLT cluster running 5.1.5.1 software although I’ll probably try to load 5.1.6.0 in our testlab and see how it runs. I’d love to hear comments back from anyone running 5.1.6.0 software.

Cheers!

sw-8600.acme.org:5# config diag fpga dpc update 1 /pcmcia/dpc194.xsvf
Found /pcmcia/dpc194.xsvf

WARNING: Starting the FPGA update process. DO NOT reset the card or  box during
this process. Please wait for FPGA UPDATE SUCCESS message. (Command prompt will
be returned but update process will take several minutes)
This Message will get appended to the log file upon completion of the update.
The command prompt will return after the upgrade is complete.                                                                       -

sw-8600.acme.org:5# CPU5 [04/27/11 05:59:15] COP-SW ERROR SyncTransportLayer: msgState=1, ltrStatus=LTR_SYNC_MSG_SLOT_INUSE, Slot=1

sw-8600.acme.org:5# CPU5 [04/27/11 05:59:15] COP-SW ERROR SyncTransportLayer: msgState=1, ltrStatus=LTR_SYNC_MSG_SLOT_INUSE, Slot=1

sw-8600.acme.org:5# CPU5 [04/27/11 05:59:15] COP-SW ERROR SyncTransportLayer: msgState=1, ltrStatus=LTR_SYNC_MSG_SLOT_INUSE, Slot=1

sw-8600.acme.mlhs.org:5# CPU5 [04/27/11 05:59:15] COP-SW ERROR SyncTransportLayer: msgState=1, ltrStatus=LTR_SYNC_MSG_SLOT_INUSE, Slot=1

sw-8600.acme.mlhs.org:5# CPU5 [04/27/11 05:59:15] COP-SW ERROR SyncTransportLayer: msgState=1, ltrStatus=LTR_SYNC_MSG_SLOT_INUSE, Slot=1

sw-8600.acme.mlhs.org:5# CPU5 [04/27/11 05:59:31] COP-SW INFO Slot 1: FPGA UPDATE SUCCESS - You will need to reboot the switch or re-insert/reset the slot for FPGA to take effect.

There’s reference to this issue in the 7.0 and 7.1 release notes.

wi00518565 - When upgrading FPGA firmware on R or RS modules, the following message can appear: Router-C:5#/CPU5 [03/08/10 15:04:15] COP-SW ERROR 27894800: LtrId = 152,LtrPrio=0,ltrStatus = 15 LTR_SYNC_MSG_SLOT_INUSE),msgId=53,msgState=1,Slot=3 This message has no negative effect on the FPGAupgrade. There are no specific FPGA upgrades required with release 7.0.

Cheers!

Cheers!

The most notable feature in this software release is the inclusion of Shortest Path Bridging  (SPB).

“Release 7.1 of the Ethernet Routing Switch 8800/8600 supports the IEEE 802.1aq standard of Shortest Path Bridging MACinMAC (SPBM). SPBM makes network virtualization much easier to deploy within the Enterprise environment, reducing the complexity of the network while at the same time providing greater scalability.

SPBM eliminates the need for multiple overlay protocols in the core of the network by reducing the core control plane to a single protocol which can provide virtualization services for both layer 2 and layer 3, on a common Ethernet infrastructure using a pure Ethernet technology base. SPBM allows for layering the Ethernet network into edge and core domains with complete isolation between their MAC addresses. This technology provides all the features and benefits required by Carrier-grade deployments to the Enterprise market without the complexity of alternative technologies traditionally used in Carrier deployments (typically MPLS). SPBM integrates into a single control plane all the functions that MPLS requires multiple layers and protocols to support.

SPBM provides any-to-any connectivity in a network in an optimized, loop-free manner. It employs shortest-path trees to each destination, without the long convergence delays experienced with Spanning Tree Protocol. To do that, SPBM uses Intermediate System to Intermediate System (IS-IS) link state routing protocol to learn and distribute network information. IS-IS dynamically learns the topology of a network and uses its inherent knowledge to construct shortest path unicast and multicast trees from every node to every other node in the network. Also, unlike Spanning Tree Protocol, IS-IS does not block ports to provide a loop free topology, so bandwidth is not wasted.

The SPBM components introduced in this release are:

1. Shortest Path Bridging (IEEE 802.1aq) for simple and safe VLAN extensions across a network. SPBM does not use spanning tree, and all its links are active.
2. SMLT for dual-homing of non-SPBM switches to a pair of SPB/IST switches.
3. SPBM/IP for simple and safe VRF extensions across a network infrastructure without OSPF or BGP.
4. InterISID routing for routing of L2 VPNs in the SPB domain“

You can review the configuration guide around SPBM here.

You fine the release notes here along with an upgrade document here.

Cheers!

New Features in This Release

• VLACP HOLD Enhancement
• SLPP – Introduced New EtherType for SLPP PDU
• SLPP – Re-Arm per port SLPP PDU Receive Counter issue

I recently discussed (in the forums) the added benefit of VLACP when recovering from network failures. It seems that Avaya is continuing to build upon that goal with additional enhancements to VLACP to help aid in recovery after a network outage.  We also discussed the known issue with SLPP and how it can accidentally trip on occasion given that the counter/threshold is cumulative and never reset. Avaya has refined SLPP to reset the counter every 24 hours thereby eliminating any false positive conditions that might occur over a extended duration (uptime of the switch).

I would encourage everyone to review the release notes for all the resolved and known issues.

Cheers!

Avaya has released v7.0.0.2 software for the Ethernet Routing Switch 8600/8800. This maintenance release resolves the same issue that 5.1.3.1 and 4.1.8.5 was released to address;

• 8692 CP boards with the new PCMCIA controller could experience high cpu utilization and other systemic effects while accessing certain types of PCMCIA cards. The problem has now been addressed. (wi00734221 and wi00733439)

Please review the release notes for any additional information.

Cheers!

The new re-spin version module can be identified by the PCMCIA card slot now having front panel artwork that shows “PC Card” while older versions of the module show “PCMCIA”. Additionally, the new version of the hardware can be identified via the Hardware Revision 52.

Otherwise there doesn’t appear to be any need to upgrade your ERS8600 to this specific release.

Cheers!

Avaya has released v4.1.8.5 software for the Ethernet Routing Switch 8600. This maintenance release addresses the same issue that 5.1.3.1 was released to address;

• 8692 CP boards with the new PCMCIA controller could experience high cpu utilization and other systemic effects while accessing certain types of PCMCIA cards. The problem has now been addressed. (wi00564790)

Please review the release notes for any additional information.

Cheers!

• 8692 CP boards with the new PCMCIA controller could experience high cpu utilization and other systemic effects while accessing certain types of PCMCIA cards. The problem has now been addressed. (wi00564790)

Please review the release notes for any additional information.

Cheers!

Copyright (c) 2010 Avaya, Inc.
CPU Slot 5:    PPC 745 Map B
Version:       5.1.3.0/020
Creation Time: Aug  5 2010, 19:19:09
Hardware Time: SEP 13 2010, 15:51:09 UTC
Memory Size:   0x10000000
Start Type:    cold
SMART MODULAR TECH. HYA ATA

The /pcmcia device mounted successfully, but it appears
to have been formatted with pre-Release5.1 file system code.
Nortel recommends backing up the files from /pcmcia, and
executing dos-format /pcmcia to bring the file system on the
/pcmcia device to the latest ERS8600 baseline.
open_file:can't open "/pcmcia/pcmboot.cfg" 0x380003 _dosFsLib_FILE_NOT_FOUND
/flash/  - Volume is OK

Loaded boot configuration from file /flash/boot.cfg
Attaching network interface lo0... done.

Press  to stop auto-boot...
Loading /flash/p80a5130.img ... 12637154 to 43802740 (43802740)
Starting at 0x1000000...

SMART MODULAR TECH. HYA ATA
 Booting PMC280 Mezz HW. Please wait.......
The BootCode address is 0xe100100 3303
.
Mezz taking over console and modem.....
Mezz CPU Booted successfully

Initializing backplane net with anchor at 0x4100... done.
Backplane anchor at 0x4100... ..
Mounting /flash: .done.
CPU5 [09/13/10 11:52:36] SW INFO Found serial number <11:22:33:44:55:66> in file
CPU5 [09/13/10 11:52:36] SW INFO License Successfully Loaded From License Type -- ADVANCED

Ethernet Routing Switch 8600  System Software Release 5.1.3.0
Copyright (c) 1996-2010 Avaya, Inc.

CPU5 [09/13/10 11:52:36] SW INFO System boot
Warning: Please save your configuration after boot-up to avoid
CPU5 [09/13/10 11:52:36] SW INFO ERS System Software Release 5.1.3.0
CPU5 [09/13/10 11:52:40] MPLS INFO All MPLS components are up and active
CPU5 [09/13/10 11:52:40] HW INFO Card inserted: Slot=5 Type=8692SF
CPU5 [09/13/10 11:52:40] SW INFO R-Module inserted: Slot=1 Type=8630GBR, waiting to bootup...
CPU5 [09/13/10 11:52:40] SW INFO R-Module inserted: Slot=3 Type=8683XLR, waiting to bootup...
CPU5 [09/13/10 11:52:40] HW INFO Initializing 8692SF in slot #5 ...
CPU5 [09/13/10 11:52:49] SW INFO Slot  1: Loading /flash/p80j5130.dld
CPU5 [09/13/10 11:52:49] SW INFO Slot  3: Loading /flash/p80j5130.dld
CPU5 [09/13/10 11:53:28] SW INFO Slot  3: 8683XLR Initializing.  Do not remove board.
CPU5 [09/13/10 11:53:29] SW INFO Slot  1: 8630GBR Initializing.  Do not remove board.
CPU5 [09/13/10 11:53:33] SW INFO Slot  3: 8683XLR Initialization completed.
CPU5 [09/13/10 11:53:34] SW INFO Slot  3: Restart new image version 5.1.3.0
CPU5 [09/13/10 11:53:34] SW INFO Slot  1: 8630GBR Initialization completed.
CPU5 [09/13/10 11:53:34] SW INFO Slot  1: Restart new image version 5.1.3.0
CPU5 [09/13/10 11:53:43] SW INFO Slot  3: Loading /flash/p80j5130.dld
CPU5 [09/13/10 11:53:44] SW INFO Slot  1: Loading /flash/p80j5130.dld
CPU5 [09/13/10 11:54:04] SW INFO slot 1 found NP heartbeat - R-Module is online
CPU5 [09/13/10 11:54:04] SW WARNING Slot 1 is running older software version(s) of FPGA. Please upgrade to the latest version(s)
CPU5 [09/13/10 11:54:04] SW WARNING DPC FPGA current revision: 184. Upgrade to the latest revision: 194
CPU5 [09/13/10 11:54:08] SW INFO slot 3 found NP heartbeat - R-Module is online
CPU5 [09/13/10 11:54:08] HW INFO Initializing 8630GBR in slot #1 ...
CPU5 [09/13/10 11:54:08] SW WARNING Slot 3 is running older software version(s) of FPGA. Please upgrade to the latest version(s)
CPU5 [09/13/10 11:54:08] SW WARNING DPC FPGA current revision: 184. Upgrade to the latest revision: 194
CPU5 [09/13/10 11:54:09] HW INFO Initializing 8683XLR in slot #3 ...

*****************************************************************
* Copyright (c) 2010 Avaya, Inc.
* All Rights Reserved
* Ethernet Routing Switch 8006
* Software Release 5.1.3.0 (Released - General Availability)
*****************************************************************
Login: CPU5 [09/13/10 11:54:10] SW INFO Loading configuration from /flash/config.cfg
CPU5 [09/13/10 11:54:10] SW INFO NTP Enabled
CPU5 [09/13/10 11:54:10] SNMP INFO VRF name: GlobalRouter (VRF id 0): Max route warning set
CPU5 [09/13/10 11:54:10] SW INFO The system is ready
CPU5 [09/13/10 11:54:10] SNMP INFO Booted with PRIMARY boot image source - /flash/p80a5130.img
CPU5 [09/13/10 11:54:12] SW INFO PCMCIA card detected in Master CPU "ERS-8606" slot 5, Chassis S/N SS11111111
CPU5 [09/13/10 11:54:12] IP INFO the OSPF Md5 key file does not exist
CPU5 [09/13/10 11:54:12] SNMP INFO Chassis with Power Supply redundancy
CPU5 [09/13/10 11:54:12] SNMP INFO Fan Up(FanId=1, OperStatus=2)
CPU5 [09/13/10 11:55:00] SNMP INFO Sending Cold-Start Trap

I needed to copy down the software from the TFTP server to the local /flash member;

ERS-8606:5# copy 10.1.1.1:dpc194.xsvf /flash/dpc194.xsvf

With the software on the flash we can now start the upgrade;

ERS-8606:5# config diag fpga dpc update 1 /flash/dpc194.xsvf
Found /flash/dpc194.xsvf

WARNING: Starting the FPGA update process. DO NOT reset the card or  box during
this process. Please wait for FPGA UPDATE SUCCESS message. (Command prompt will
 be returned but update process will take several minutes)
This Message will get appended to the log file upon completion of the update.
The command prompt will return after the upgrade is complete.                                                

CPU5 [09/13/10 12:02:41] COP-SW INFO Slot 1: FPGA UPDATE SUCCESS - You will need to reboot the switch or re-insert/reset the slot for FPGA to take effect.

You’ll need to reboot the switch in order for the module(s) to load the new FPGA firmware.

Cheers!

There’s a lot of great technical information in this document including a lot of recommendations and best practices.

Cheers!

Please review the release notes for all the resolved issues, known issues and known limitations.

Switch management

• The command “show ports stats show-all” was previously not displaying all of the associated port statistics information. The ERS 8600 now will display the proper information. (Q02012952-01)
• The egress queue service rates configured in NNCLI mode were previously getting lost after a reboot. The egress queue service rates configured are now retained after a reboot. (Q02116358)
• In NNCLI mode “show qos egress-queue-set <queue-set-id>” command was showing no output. ERS 8600 now displays the proper output on executing this command. (Q02116618)
• Initiating a traceroute via JDM to an unknown destination while other ping or traceroute activities were being performed simultaneously from the same switch could potentially lead to system instability. This has been corrected. (Q02137566)

Platform

• In specific scenarios with filters and port mirroring being used simultaneously, the ERS8600 will no longer see an increase in CPU utilization due to packets being sent to the CP/SF incorrectly. (Q02141867)
• In some specific ACL configurations with a default-action specified, the ERS8600 previously blocked certain traffic patterns improperly. The issue has been addressed. (Q02068243)
• Previously links on an R-module could stay up for 60 seconds when the primary SF/CPU (of a dual non-HA SF/CPU configuration only) was not properly removed (module pulled out without prior master reset or switchover). The ERS 8600 now ensures that the links are brought down immediately when the primary SF/CPU is removed for this configuration. (Q02102654)
• The radius secret key will now be stored as encrypted in the shadov.txt file. Before upgrading to 5.1.3.0, it is now required of the user to delete the radius secret key and then re-add it after the up-grade is complete to avoid accessibility problem with RADIUS. This situation has already been addressed in 7.0.0.0 code release. (Q01881817-03)
• When multiple failovers are performed on an ERS 8600 in HA environments, some R module I/O modules could come back up off-line. This situation is now addressed in the 5.1.3.0 code release but also requires the use of the new updated DPC FPGA image (see File Names section). Associated with this 5.1.3.0 Release, the DPC firmware image must be upgraded for all R-module I/O modules to the dpc194.xsvf firmware image. If the firmware image is not upgraded, the user will receive a warning log message upon any re-boot of 5.1.3.0 code release, warning them that DPC FPGA firmware image is out of revision. (Q02053766-01)
• On E/M module cards, receiving 802.3x pause frames on gigabit Ethernet ports could previously result in port level resets. (Q02101087)
• For an ERS 8600 running with 8692 SF/CPU with SuperMezz, the SuperMezz physical LED will now display properly. (Q02102364)
• During boot-up, the potential for 8612XLRS 10Gig port flapping will no longer occur. (Q02138423)
• An error in loading configuration file previously resulted in all of the line cards being disabled. This behavior was added in 5.1 code. This behavior is now changed such that the rest of the correct config will be loaded and all the remaining line cards will not be disabled, but only the line card associated with the improper configuration will be disabled. This situation is different than an “invalid config file, with verify-config flag enabled”, in which case the system will not load the config and will bring up the system with all I/O modules disabled (versus loading the default config). This situation has already been addressed in 7.0.0.0 code release. (Q02056382-01)
• A power usage calculation error has been corrected for 8648TXE and 8691SF cards which previously lead to improper warnings being generated in some configurations indicating that the chassis was running on low power. (Q02072016)
• The value of the ingress records was showing improper values via the command show ip mroute-hw resource usage. This has been now been addressed and the proper values for these records are now displayed. (Q02120442)
• The software power tables embedded in the ERS 8600 were out of sync with the Power Supply Calculator posted via the web. This inconsistency has now been addressed. This issue has already been addressed in 7.0.0.0 code release. (Q02020261-01)

RSTP/MSTP

• An outage of 30sec has been observed in some RSTP setups when one of the root ports was disabled. This issue has now been addressed and the ERS 8600 re-converges within the proper time interval. (Q01984762-02)
• In RSTP mode, the log file transfer feature was not working properly. This issue has been addressed and the log file transfer feature is now working as expected when RSTP mode is enabled. (Q02101565-01)

IP Unicast

BGP

• In a specific aggregation scenario, when an ERS 8600 forms a neighbour relationship with Cisco, the BGP session will no longer go down due to a malformed AS path. (Q02085844-01)
• During some specific BGP transition scenarios, system instability was previously seen for the ERS 8600. This issue is now addressed. (Q02134841)
• BGP instabilities are no longer observed associated with a HA failover. (Q02135118)
• After an HA failover with a BGP route policy enabled, some BGP routes could be rejected and not re-advertised by the ERS 8600. This issue has been addressed. (Q02136224)

OSPF

• After an upgrade it is now ensured that the OSPF MD5 keys are no longer lost. (Q02127996)

VRRP

• Enabling and then disabling the IST protocol will no longer lead to improper values displayed in the VRRP records. (Q02106080-01)

IP Multicast

• ERS 8600 now ensures that when an IP Multicast sender and the receiver are connected to the same ERS 8600, that SPM (Source Path Messages – specific message type within PGM) packets are no longer dropped. This is independent of PGM being enabled or not, as PGM packet flows can function with PIM-SM enabled. (Q02119454)

MLT / SMLT

• ERS8600 will now deterministically (and properly) hash traffic flows for IPVPN Lite traffic on MLT links. (Q02142913)
• MLTs will now come up properly when the LACP Min Link feature is enabled. (Q02034692-02)
• In some rare network events, it is possible to learn the fdb-entry for an IP interface of an SMLT peer on the SMLT associated port instead of the IST MLT. In this scenario, the fdb-entry will now be properly updated afterward to correctly point to the IST MLT. (Q02142730)
• CPU high buffer utilization and associated IST instabilities will no longer be observed in specific high CPU and high traffic load conditions for the ERS 8600. (Q02109963)
• The 8632TXE module will now properly go offline in the scenario where both master and slave CPU’s are removed in systems running with smlt-on-single-cp enabled. (Q02123052)

VLACP

• If a mismatching VLACP configuration exists on two ends of a connection associated with a SLT, the SLT will no longer show as SMLT up improperly after a reboot. (Q02119095)

BFD

• On an ERS8600 running both BGP and BFD, when one tries to enable BGP first and then BFD on the BGP neighbour, the BFD related config can now be set properly. (Q02141063)

Cheers!

If you are looking for a great resource on BGP I would highly recommend O’Reilly’s book titled BGP.

If you are looking for Avaya/Nortel specific information concerning their BGP implement then you are in luck. Avaya has a technical configuration guide for the ERS 8600 that focuses on BGP. While this is an older document (November 2007) it still does a great job of providing a number of configuration examples and explaining the basics.

In the near future I might need to use an ERS 8606 as an Internet router. I’ll need to peer with the ISP since I’m multi-home to independent Internet Service Providers, although I’m not sure if the 8692SF can handle a full BGP routing table. Has anyone ever tried to feeding a full (or partial) BGP routing table from the Internet to an ERS 8800/8600 switch?

Cheers!

Here’s an example of a 1000BaseCWDM (40Km) SFP where the Rx path has been compromised by a squirrel. Yes, true story a squirrel chewed into the cable outside the building and damaged one of the fiber-optic strands in the cable.

ERS-8610:5# show sys pluggable-optical-modules info 1/15 detail

================================================================================
Pluggable Optical Module Info 1/15 Detail
================================================================================
Port: 1/15
Type: Gbic1470(40)
DDM Enabled : TRUE
Nortel PEC  : AA1419053-E6         CLEI       : IPUIAHGWAA
Vendor      : NORTEL               Vendor PN  : AA1419053-E6
Vendor REV  : A                    Vendor SN  : LUMNTW7DDT200XXX
Vendor Date : 12/12/07
Wavelength  : 1470.00 nm

Digital Diagnostic Interface Supported

DDM Status              : Ok
Calibration             : External
RX Power Measurement    : Average
Auxiliary 1 Monitoring  : Not Implemented
Auxiliary 2 Monitoring  : Not Implemented

--------------------------------------------------------------------------------
LOW_ALARM LOW_WARN    ACTUAL  HIGH_WARN HIGH_ALARM THRESHOLD
THRESHOLD THRESHOLD   VALUE   THRESHOLD THRESHOLD  STATUS
--------------------------------------------------------------------------------
Temp(C)       -24.9960  -24.9960   30.6367   99.9960   99.9960  Normal
Voltage(V)      2.8000    3.1000    3.3080    3.5000    3.8000  Normal
Bias(mA)        4.2500    4.5000   13.2720  100.0120  110.0140  Normal
TxPower(dBm)   -6.0      -5.0      -2.0       1.9000    2.9000  Normal
RxPower(dBm)  -26.1000  -25.1000  -40.0      -3.0      -2.1000  Normal

Cheers!

I’m curious if anyone out there is using Avaya’s IP Flow Manager and has any thoughts and/or comments to share.

I remember a few folks either here or on the forums commenting that they were using nTop to collect the IPFIX flow information. Anyone have any thoughts about nTop/nProbe?

Cheers!

We now know that any new switches purchased with the 8895SF/CPU will be labeled an ERS 8800 a opposed to an ERS 8600 switch. We also know that Avaya has released 7.0 software which will put pressure on their customers to upgrade to at least 5.x software. We also know that a lot of old hardware is coming MD and/or EoL. I just realized that the classic non-E cards (8648TX, 8608SX, 8608GB, 8608GT) have been EoL since 2006. I knew they were EoL but I didn’t think it was 4 years ago. So I went back and looked over the last few bulletins announcing (in some cases updating previous announcements) regarding the MD (Manufacture Discontinued) of various hardware modules.

Here are some notes that I came up with after looking threw the various bulletins;

• The Nortel Ethernet Routing Switch 8691 SF/CPU, 8616SXE, 8632TXE, 8616GTE, 8648TXE, 8608SXE, 8624FXE, 8608GBE, 8608GTE, 8632TXM, 8648TXM, 8608GBM, 8608GTM, 8672ATME, 8672ATMM, 8683POSM modules are to be declared Manufacturing Discontinue effective either Dec 31, 2009 or Apr 30, 2010. The official End of Life for these modules isn’t until April 30, 2015 (5 years).
• The Nortel Ethernet Routing Switch Web Switching Module (WSM) was declared Manufacturing Discontinue effective Feb 28, 2009. At this time for the Nortel Ethernet Routing Switch 8600 Security Modules (SDM 8660 versions) are also being declared Manufacturing Discontinue effective Dec 31, 2009. The official EoL for this module isn’t until March 31, 2014. Please note that the WSM includes the following products that were already MD in January 2009,  8600SM 8660FW, 8600SDM 8600TPS, 8600SDM 8600FW/TPS.
• The Avaya Data Solutions (ADS) [Nortel] Ethernet Routing Switch 8800/8600 R-Modules, namely 8630GBR (DS1404063-E5), 8648GTR (DS1404092-E5), 8683XLR (DS1404101-E5), and 8683XZR (DS1404064-E5) are to be declared Manufacturing Discontinue effective July 30, 2010 with an associated last order date of June 30, 2010. The official EoL for these modules isn’t until July 30, 2015 (5 years).

If the intention is to support the older hardware for 5 years isn’t it obvious that Avaya will also need to support the software for 5 years?

If you’re looking to deploy a new ERS 8600 you might really need to consider that the older cards and software will probably only be supported until 2015. However, if you deploy an ERS 8800 switch you’ll likely be supported for much longer than that… again 10+ years might not be unreasonable, although you might also consider a VSP9000 or similar high density 10Gbps switch.

Bulletins:

• Hardware Lifecycle Support Update for ERS 8800/8600 R-Modules Pre-Notification of Manufacture Discontinue
• Hardware Lifecycle Support Update for ERS 8691SF, and E/M Modules Final Notification of Manufacture Discontinue
• Hardware Lifecycle Support Update for ERS 8600 WSM and Security Modules Notification of Manufacture Discontinue

Cheers!

Ethernet Routing Switch 8800 release 7.0 software introduces support for the new 8895 Switch Fabric/CPU Module. When an 8000 Chassis is equipped with the 8895 SF/CPU, this system is known as an Ethernet Routing Switch 8800; conversely, when equipped with an 8692 SF/CPU module (with SuperMezz) the system is known as an Ethernet Routing Switch 8600. Ethernet Routing Switch 8800 release 7.0 software can only operate on an Ethernet Routing Switch 8600 system with appropriate hardware configurations.

With release 7.0 software, the Ethernet Routing Switch 8800 takes over as the go-forward solution for new customers seeking the most reliable and versatile Campus LAN Core Switch. Additionally, release 7.0 software ensures high levels of investment protection and continuity of service for returning Ethernet Routing Switch 8600 customers, as existing Ethernet Routing Switch 8600 deployments can be incrementally upgraded to take
advantage of new features.

In this document, use of the term Ethernet Routing Switch 8000 in relation to software and supported features indicates applicability to both 8600 systems and 8800 systems.

Here are some of the new features included in the 7.0 software release;

• 8895 SF/CPU
• 8003-R 3-slot chassis
• Enterprise Device Manager (EDM)
• Key Health Indicator (KHI) enhancements
• BPDU Filtering
• DHCP snooping
• Dynamic ARP Inspection
• IP Source Guard
• IGMP Layer 2 querier
• Multicast VLAN Registration (MVR)
• PIM-SSM with SMLT
• IP Multinetting
• Route Switch Processor Packet Tracing
• ERCD Records Dump
• IPv6 RSMLT
• IPv6 VRRP
• BGP+
• IPv6 RADIUS
• IPv6 DHCP Relay
• Singular Record Operations
• show debug generic

If you are serious about IPv6 then 7.0 should be a welcome upgrade. You shouldn’t have any issues deploying IPv6 with 7.0 software now that everything is supported in an IPv6 environment. Also added are some of the Ethernet Routing Switch 5000 series features such as BPDU filtering, DHCP snooping, IP source guard, IGMP v2 querier and dynamic ARP inspection.

In order to run 7.0 software you’ll need a 8692SF w/Mezz and all R/RS modules.

I would suggest you review the release notes and upgrade notes for all the details.

Cheers!

Avaya has released v5.1.2.0 software for the Ethernet Routing Switch 8600. I know there were quite a few users waiting for this release. Please feel free to post back here to let us all know how your making out with 5.1.2.0 if you happen to deploy it.

You can find a copy of the release notes here but you’ll obviously need to visit the Avaya/Nortel site to download the software.

Here are some of the bullet points from the release notes;

Switch management

• The SNMP trap for rcIpBgpPeerLastError will now be sent with a proper byte string length such that the last bye will no longer be lost. This could previously cause operational issues with some SNMP management stations. (Q02092718)
• ERS 8600 will no longer observe system instability associated with configuration changes to switch parameters involving SNMP settings. (Q02094258)
• Previously the ERS 8600 was applying a local Access Policy to IPv6 routed SSH packets. Now the system will route these packets and apply Access Policies to only local destination policy type (SSH, Telnet, HTTP) IPv6 packets. This will no longer cause inappropriate connection issues to remote hosts. (Q02070640-01)
• ERS8600 has been modified to now allow proper communication with NetQOS Management Device. (Q02049612-01)

Platform

• With both filtering and ingress mirroring enabled on the ERS8600, system instability could be seen under certain traffic conditions. This is now resolved. (Q02078239-01)
• For non-routed VLANs, SLPP will now use a source MAC address equal to the Base Mac Address of the ERS8600 plus the ID of the VLAN. This will ensure that received SLPP packets are processed against the correct non-routed VLAN when a loop is present in the network and avoid erroneous warning messages. (Q02081719)
• IP fix traffic from the switch to an external collector will no longer be sent with an improper QoS marking of QOS=7, but instead sent with QOS=0, now placing these packets into the proper default egress queue. Previously this traffic could potentially interfere with other system management traffic leading to the potential for system instability when IPFix was enabled. (Q02044640-01)
• High CPU utilization on an I/O line card co-processor will no longer result in a loss of messaging synchronization with the SSF CPU, which previously could have led to system instability. (Q02085085)
• ERS 8600 will no longer show instability in tLogger task while writing to the PCMCIA card with clilogging enabled. (Q02006689-01)
• ERS 8600 R and RS module card ports will now initialize multicast and broadcast bandwidth limiting values properly when these features are enabled. (Q02074960)
• ERS 8600 will now properly handle any broadcast destination MAC IPX packets of type RIP or SAP. Previously this could cause and issue for routing IPX for E/M modules (R/RS modules do not support IPX Routing). (Q01997486-04)
• Packet throughput performance for jumbo frames at line rate has been improved for the 8612XLRS modules. (Q02075673)
• Filter pattern definitions for HTTP packet streams will no longer impact other protocol traffic. (Q02089688)
• Users will now be able to connect to an ERS 8600 using Secure Copy (SCP) with access-level rwa when access-strict true is also configured. Previously SSH worked, but SCP did not. (Q01767930-01)
• ERS 8600 will no longer encounter link flapping upon reboot of an OM1400 edge device running SFFD when connected to 8630GBR ports. (02014236-01)
• ERS8600 will now properly forward DHCP packets with the DHCP-relay agent configured as the VRRP virtual IP when the DHCP request has the broadcast flag set. Best practice recommendation still continues to be to configure the DHCP-relay agent IP address as the VLAN physical address and not the VRRP IP address. (Q02059607-01)
• Reliability of R and RS series line card recovery after CPU resets (normally seen during switch software upgrades) has been improved due to enhancements in SSF CPU to I/O module co-processor message communication and synchronization. (Q02091485/ Q01997485)
• ERS 8600 will no longer silently drop packets when the number of ACEs with debug count enabled is such that system resources are at their maximum, but instead the filters will now all function properly. (Q02045086)

RSTP/MSTP

• Enhanced MSTP/RSTP logging information which was previously added in release 4.1.3.0 was not present in any 5.x code. This functionality has now been properly added. Q02053232)
• The VLAN interface on an ERS8600 in RSTP/MSTP mode will no longer be brought up unless a port first becomes active in the VLAN. This matches the existing VLAN interface behavior in STP mode. (Q02083039)
• Packet loss on an MLT with RSTP enabled will no longer been seen after a CPU reset/switchover with HA mode enabled or after a complete switch re-boot. (Q02003158-01)
• ERS 8600 will properly retain the MLT path-cost configuration over reboots when configured for RSTP/MSTP mode. (Q02048253)
• ERS8600 will now properly show the MSTP CIST port pathcost info when “show port info mstp” is executed. (Q02048252)

IP Unicast

• The configured filter action is now properly observed for ACL’s configured to match UDP source and destination port ranges between 32752 and 32767. (Q02076252-01)

Static Routes

• ERS 8600 will no longer encounter system (DRAM) memory exhaustion with DHCP-relay configured on a Layer 2 VLAN or at the port level for a non-brouter port. (Q02076879)

BGP

• ERS 8600 will now properly learn the default routes from eBGP peers even after the failover or toggling of the physical port connection. (Q02094999)

IP Multicast

• ERS 8600 will no longer observe periods of sustained high CPU utilization associated with the forwarding of multicast traffic. (Q02067852)
• ERS 8600 will now properly recover its DVMRP status for an ATM interface when a Port/Fiber Fault occurs, and is then restored. (Q02041428)

MLT / SMLT

• Connectivity to NLB servers single homed to one ERS8600 in an IST pair will now function properly for SMLT connected devices when using an nlb-mode of unicast or with arp multicast-mac-flooding enabled. Configurations using nlb-mode of multicast were not affected. (Q02037778-01)
• SLPP will now disable the correct SMLT port when a loop is detected on an SMLT link where the smlt-id configured is not the same as the mlt-id value configured. (Q02089994)
• On ERS8600, FDB and ARP entries will point correctly to SMLT after IST peer reboots. Previously entries learnt on SMLT ports could very occasionally point incorrectly to the IST. (Q02091486)

RSMLT

• With ICMP redirect enabled on RSMLT peer switches, packets destined to the RSMLT-peer’s MAC address will now be forwarded correctly and not dropped as ICMP-redirect packets. (Q02091034)
• In RSMLT environments, ERS8600 will no longer add the RSMLT-peer’s MAC address to its Router MAC table. This will result in packets destined to the IP interface of RSMLT-peer to forward properly. (Q02091350)

VLACP

• ERS 8600 will now always bring down a port via VLACP within the configured timeout value when its VLACP peer goes down. Previously one end of the link would take an extra timeout cycle before downing the port in some scenarios. (Q02088710)
• In scenarios where a port was taken down by VLACP and then the far end switch is rebooted or VLACP recovered to recover the port, Persistent VLACP port flapping will no longer occur. (Q02088709)
• On E-mode enabled switches in full mesh SMLT topologies, protocol traffic will now flow properly on the second MLT link when the first MLT link is disabled. (Q02089615)

VRRP

• Disabling and re-enabling the IST session on an IST switch pair with VRRP configured between them will no longer result in both switches reporting VRRP mastership. (Q02104773)

Cheers!