Michael McNamara

It’s the networks fault #6

Michael McNamara — Mon, 29 Oct 2012 16:53:45 +0000

I’m going to start including articles/posts or discussions that I’ve come across on the Internet that others might enjoy. A few of these are from my fellow Tech Field Day delegates who I met back in mid-October.

Articles

Not everyone needs Five Nines by Anthony Burke – have a read why Anthony believes not everyone needs 99.999% of availability and redundancy. I’ve seen plenty of complex solutions in my time that usually fall over by themselves. They are so inherently complex and tedious that the solution itself causes more downtime and disruption than would a simpler approach. I personally think this is the question that engineers and architect need to consider when designing anything.
Comparing PVLAN to PVLAN Edge by Paul Stewart – read Paul’s clear explanation of a PVLAN and how to provide layer 2 isolation between devices within your network. He even provides a sample configuration if you wish to test it out on your own.
WLAN8100 and ERS8800 Come Together: VENA Unified Access – have a high level peak into Avaya’s new unified wireless LAN architecture. It’s really interesting what Avaya is doing in splitting the control and forwarding planes given all the discussion around SDN. I sense an blog post coming…
Getting Less Interruptions and More Time by Not Answering the Phone by Greg Ferro – Greg continues to give us some insights into how to be more efficient with our time. There’s a reason a few of us enjoy either coming in very early or staying very late at the office – there’s no one to bother you and you can get an incredible amount of work completed.
Y U NO MONITOR DATACENTER TEMPERATURE!? by Jeremy Gaddis – Jeremy entertains us with more stories of how and why data centers sometimes fail. Listen up management and take notes, we’re trying to save your bacon here not ours. We can only say “we told you so” so many times.

Software Releases

Avaya Ethernet Routing Switch 5000 Software Release v6.3.0

Avaya has released software v6.3.0 for the Ethernet Routing Switch 5000 Series.

There are a significant number of new features although I must warn you that some features are only available on the ERS 5600 model switches. Example, VRF-Lite is only available on the ERS 5600 series switches and not the ERS 5500 series switches.

Follow the discussion thread over on the forums that covers the new software release and what other community members are saying about it.

Here are the highlights;

802.1AB customization
802.1AB Integration
802.1AB new default parameters
802.1X non-EAP Accounting
802.1X non-EAP re-authentication
Add DecOtherEther2 to Protocol-Based VLANs
Block Subsequent MAC Authentication in MHMA
Circuitless IP (CLIP)
Content-Based Forward to Next Hop Enhancements
DHCP Snooping External Save
Diagnostic Auto Unit Replacement
Disable CLI Audit
IGMP Multicast Flood Control
IGMP over SMLT
Increased ARP Table Size
Increase MAC Filtering List support
Inexistent VLAN Mapping for MSTI
Internal BGP
IP Manager support for SSH
Lossless Mode Support for 8–High Stack
MLT/DMLT/LAG Dynamic VLAN Changes
PIM-SSM
RADIUS EAP/non-EAP different servers
RADIUS Management Accounting with TACACS+ support
RADIUS Support for Interim Accounting
Routing of IP Directed Broadcasts on a per VLAN basis
Serial Security Enhancement
Show Software Status
SLPP Guard
SNMP Trap Enhancements
SSH Banner
SSH retry
Static MRoute for PIM
Sys-uptime Pre-notification Trap
Syslog Support for 802.1X/EAP/NEAP/UBP
Trace Support for 802.1X
Unified User Authentication
Voice VLAN Integration
VRF Lite

You can find all the details in the release notes on the Avaya website.

Avaya Ethernet Routing Switch 8600/8800 Software Release 7.2

Avaya has released software 7.2 for the Ethernet Routing Switch 8600/8800.

This release introduces two major features (and two significant operational improvements) but I would not recommend it to anyone unless you require these new features;

IP Multicast over SPBM
Avaya VENA Unified Access
Switch Fabric Failure Dynamic Detection enhancement
SWIP Multicast Queue Lockup Detection enhancement

You can find all the details in the release notes on the Avaya website.

There’s no shortage of material to write about looking at all the features above.

Cheers!

Ethernet Routing Switch 8600 Log files

Michael McNamara — Sat, 16 Aug 2008 13:00:13 +0000

The Nortel Ethernet Routing Switch 8600 has multiple log files that can be examined for signs of a problem or during basic troubleshooting. The log files will be stored on the PCMCIA flash card or in memory if there is no flash card available. There is also a trace facility within the ERS 8600 that allows you to log extensive debug information within the switch on different subsystems and processes. I won’t go into the trace facility in this post but if your interested you can find it in by using the command “config trace” from the CLI interface of the ERS 8600 switch.

You can check the contents of the flash memory and PCMCIA cards by issuing the “dir” command;

ERS-8610:5# dir
  size          date       time       name
--------       ------     ------    --------
 8773031    JUL-22-2008  06:11:38   /flash/p80a4163.img
     373    JUL-22-2008  06:30:36   /flash/boot.cfg
       8    AUG-16-2008  13:28:10   /flash/ospf_md5key.txt
      79    AUG-16-2008  13:28:10   /flash/snmp_comm.txt
   16440    AUG-16-2008  13:28:14   /flash/config.cfg
      11    JUL-22-2008  07:33:30   /flash/engboot
 1073634    JUL-22-2008  06:11:44   /flash/p80b4163.img
   55928    JUL-22-2008  06:11:44   /flash/p80c4163.img
   26112    JUL-22-2008  06:11:46   /flash/p80c4163.aes
 8872259    JUL-22-2008  06:16:20   /flash/p80m4163.img
 1272852    JUL-22-2008  06:17:12   /flash/p80j4163.dld
total: 64155648 used: 20639744 free: 43515904 bytes
  size          date       time       name
--------       ------     ------    --------
   29671    AUG-16-2008  14:03:18   /pcmcia/7b700005.000
   26192    AUG-16-2008  14:16:48   /pcmcia/clilog.txt
       1    JUL-22-2008  06:21:20   /pcmcia/clilogoff
       8    JUL-22-2008  07:33:30   /pcmcia/7b700005.num
total: 64710656 used: 584704 free: 64125952 bytes

You can examine the log files with the following commands;

show log file tail

ERS-8610:5# show log file tail
CPU5 [07/22/08 07:44:05] SNMP INFO Communication established with backup CPU
CPU5 [07/22/08 07:44:05] HW INFO System activity performed
CPU5 [07/22/08 07:43:44] SNMP INFO CPU switch over, stand-by CPU in slot # 5 became master
CPU5 [07/22/08 07:43:44] SNMP INFO Sending Warm-Start Trap

CPU5 [07/22/08 07:43:44] SNMP INFO CPU switch over, stand-by CPU becoming master
CPU5 [07/22/08 07:43:37] OSPF INFO Ospf Nbr State Change: rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1  LOADING->FULL Event LOADING_DONE_EVENT
CPU5 [07/22/08 07:43:37] OSPF INFO Ospf Nbr State Change: rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2  LOADING->FULL Event LOADING_DONE_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 EXCHANGE->LOADING Event EXCHANGE_DONE_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 EXCHANGE->LOADING Event EXCHANGE_DONE_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 EX_START->EXCHANGE Event NEGOTIATION_DONE_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 EX_START->EXCHANGE Event NEGOTIATION_DONE_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 2WAY->EX_START Event ADJ_OK_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 2WAY->EX_START Event ADJ_OK_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 INIT->2WAY Event TWO_WAY_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 DOWN->INIT Event HELLO_RECEIVED_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:0.0.0.0 nbr-rtid:10.7.0.2 NULL->DOWN Event HELLO_RECEIVED_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 INIT->2WAY Event TWO_WAY_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 DOWN->INIT Event HELLO_RECEIVED_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:0.0.0.0 nbr-rtid:10.7.0.1 NULL->DOWN Event HELLO_RECEIVED_EVENT
CPU5 [07/22/08 07:43:32] SNMP INFO Spanning Tree Topology Change(StgId=1, PortNum=2/1, MacAddr=00:1d:42:7b:70:01)
CPU5 [07/22/08 07:43:02] NONE INFO Spanning Tree Topology Change. New Root bridge 00:04:38:70:70:01 for StgId = 1
CPU5 [07/22/08 07:43:02] SNMP INFO Link Up(2/1)
CPU5 [07/22/08 07:43:00] SNMP INFO Vlacp link up(1/1)
CPU5 [07/22/08 07:43:00] SNMP INFO Vlacp link down(1/1)
CPU5 [07/22/08 07:43:00] SNMP INFO Link Up(1/2)
CPU5 [07/22/08 07:43:00] SNMP INFO Link Up(1/1)
CPU5 [07/22/08 07:43:00] SW INFO PCMCIA card detected in Master CPU "ERS-8610" slot 5, Chassis S/N SSPNDTxxxx
CPU5 [07/22/08 07:43:00] SNMP INFO Fan Up(FanId=2, OperStatus=2)
CPU5 [07/22/08 07:43:00] SNMP INFO Fan Up(FanId=1, OperStatus=2)
CPU5 [07/22/08 07:43:00] HW INFO System activity performed
CPU5 [07/22/08 07:43:01] SNMP INFO Vlacp link down(1/1)
CPU5 [07/22/08 07:43:01] SNMP INFO Booted with PRIMARY boot image source - /flash/p80a4163.img
CPU5 [07/22/08 07:43:01] SW INFO The system is ready
CPU5 [07/22/08 07:43:01] SW INFO NTP Enabled
CPU5 [07/22/08 07:43:00] SNMP INFO 2k card up(CardNum=2 AdminStatus=1 OperStatus=1)
CPU5 [07/22/08 07:43:00] SW INFO Loading configuration from /flash/config.cfg
CPU5 [07/22/08 07:43:00] HW INFO FFAD:Test Passed OK Slot:6 Tap:1
[07/22/08 07:43:00] The previous message repeated 1 time(s).
CPU5 [07/22/08 07:43:00] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:6 Tap:1
CPU5 [07/22/08 07:43:00] HW INFO FFAD:Test Passed OK Slot:5 Tap:1
[07/22/08 07:43:00] The previous message repeated 1 time(s).
CPU5 [07/22/08 07:43:00] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:5 Tap:1
CPU5 [07/22/08 07:43:00] SNMP INFO 2k card up(CardNum=1 AdminStatus=1 OperStatus=1)
CPU5 [07/22/08 07:43:00] HW INFO Initializing 8648GTR in slot #2 ...
CPU5 [07/22/08 07:42:59] HW INFO FFAD:Test Passed OK Slot:6 Tap:0
[07/22/08 07:42:59] The previous message repeated 1 time(s).
CPU5 [07/22/08 07:42:59] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:6 Tap:0
CPU5 [07/22/08 07:42:59] HW INFO FFAD:Test Passed OK Slot:5 Tap:0
[07/22/08 07:42:59] The previous message repeated 1 time(s).
CPU5 [07/22/08 07:42:59] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:5 Tap:0
CPU5 [07/22/08 07:42:59] HW INFO Initializing 8630GBR in slot #1 ...
CPU5 [07/22/08 07:42:59] SW INFO slot 1 found NP heartbeat - R-Module is online
CPU5 [07/22/08 07:42:57] SW INFO slot 2 found NP heartbeat - R-Module is online
CPU5 [07/22/08 07:42:52] SW INFO Slot  2: Starting software version 4.1.6.3
CPU5 [07/22/08 07:42:51] SW INFO Slot  1: Starting software version 4.1.6.3
CPU5 [07/22/08 07:42:41] SW INFO Slot  2: Loading /flash/p80j4163.dld
CPU5 [07/22/08 07:42:41] SW INFO Slot  1: Loading /flash/p80j4163.dld
CPU5 [07/22/08 07:42:36] SNMP INFO 2k card up(CardNum=6 AdminStatus=1 OperStatus=1)
CPU5 [07/22/08 07:42:36] HW INFO FFAD:Test Passed OK Slot:6 Tap:8
CPU5 [07/22/08 07:42:36] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:6 Tap:8
CPU5 [07/22/08 07:42:34] SNMP INFO 2k card up(CardNum=5 AdminStatus=1 OperStatus=1)
CPU5 [07/22/08 07:42:34] HW INFO Initializing 8692SF in slot #6 ...
CPU5 [07/22/08 07:42:34] HW INFO FFAD:Test Passed OK Slot:5 Tap:8
CPU5 [07/22/08 07:42:34] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:5 Tap:8
CPU5 [07/22/08 07:42:32] HW INFO Initializing 8692SF in slot #5 ...
CPU5 [07/22/08 07:42:32] HW INFO Card inserted: Slot=6 Type=8692SF
CPU5 [07/22/08 07:42:32] SW INFO R-Module inserted: Slot=2 Type=8648GTR, waiting to bootup...
CPU5 [07/22/08 07:42:32] SW INFO R-Module inserted: Slot=1 Type=8630GBR, waiting to bootup...
CPU5 [07/22/08 07:42:32] HW INFO Card inserted: Slot=5 Type=8692SF
CPU5 [07/22/08 07:42:30] SW INFO Killing task tCXCTask after timeout
CPU5 [07/22/08 07:42:28] SW INFO Killing task tChRxTask after timeout
CPU5 [07/22/08 07:42:26] HW INFO Stand-by CPU in slot # 5 becoming master...
CPU5 [07/22/08 07:33:34] HW INFO System activity performed
CPU5 [07/22/08 07:33:32] SW INFO PCMCIA card detected in Stand-by CPU "ERS-8610" slot 5, Chassis S/N SSPNDTxxxx
CPU5 [07/22/08 07:33:32] SW INFO Loading configuration from /flash/config.cfg
CPU5 [07/22/08 07:33:30] SW INFO CPU card entering warm-standby mode...
CPU5 [07/22/08 07:33:30] SW INFO ERS System Software Release 4.1.6.3
CPU5 [07/22/08 07:33:30] SW INFO System boot

If you have CLI logging enabled you can dump that log with the following command;

show clilog file tail

ERS-8610:5# show clilog file tail
Slot5     5 [07/28/08 21:47:29] TELNET:10.1.198.53 rwa show log file tail
Slot5     4 [07/28/08 21:47:27] TELNET:10.1.198.53 rwa box
Slot5     3 [07/28/08 21:47:25] TELNET:10.1.198.53 rwa show log file tail
Slot5     2 [07/28/08 21:47:09] TELNET:10.1.198.53 rwa info
Slot5     1 [07/28/08 21:47:08] TELNET:10.1.198.53 rwa show ip route
Slot5    62 [07/22/08 07:30:47] TELNET:10.1.20.1 rwa boot /flash/p80b4163.img
Slot5    61 [07/22/08 07:30:23] TELNET:10.1.20.1 rwa dir
Slot5    60 [07/22/08 07:30:15] TELNET:10.1.20.1 rwa peer telnet
Slot5    59 [07/22/08 07:26:36] TELNET:10.1.20.1 rwa peer telnet
Slot5    58 [07/22/08 07:26:34] TELNET:10.1.20.1 rwa box
Slot5    57 [07/22/08 07:26:30] TELNET:10.1.20.1 rwa info
Slot5    56 [07/22/08 07:26:28] TELNET:10.1.20.1 rwa ?
Slot5    55 [07/22/08 07:26:05] TELNET:10.1.20.1 rwa more /flash/config.cfg
Slot5    54 [07/22/08 07:25:54] TELNET:10.1.20.1 rwa info
Slot5    53 [07/22/08 07:25:50] TELNET:10.1.20.1 rwa more /flash/boot.cfg
Slot5    52 [07/22/08 07:25:41] TELNET:10.1.20.1 rwa save config
Slot5    51 [07/22/08 07:25:38] TELNET:10.1.20.1 rwa save bootconfig
Slot5    50 [07/22/08 07:25:27] TELNET:10.1.20.1 rwa info
Slot5    49 [07/22/08 07:25:26] TELNET:10.1.20.1 rwa nifo
Slot5    48 [07/22/08 07:25:25] TELNET:10.1.20.1 rwa choice secondary
Slot5    47 [07/22/08 07:25:20] TELNET:10.1.20.1 rwa choice
Slot5    46 [07/22/08 07:25:16] TELNET:10.1.20.1 rwa info
Slot5    45 [07/22/08 07:25:13] TELNET:10.1.20.1 rwa choice secondary image-file /flash/p80a4160.img
Slot5    44 [07/22/08 07:24:57] TELNET:10.1.20.1 rwa choice secondary ?
Slot5    43 [07/22/08 07:24:51] TELNET:10.1.20.1 rwa choice
Slot5    42 [07/22/08 07:24:34] TELNET:10.1.20.1 rwa choice image-file ?
Slot5    41 [07/22/08 07:24:25] TELNET:10.1.20.1 rwa choice ?
Slot5    40 [07/22/08 07:24:22] TELNET:10.1.20.1 rwa ?
Slot5    39 [07/22/08 07:24:21] TELNET:10.1.20.1 rwa bootconfig
Slot5    38 [07/22/08 07:24:19] TELNET:10.1.20.1 rwa config
Slot5    37 [07/22/08 07:24:09] TELNET:10.1.20.1 rwa more /flash/boot.cfg
Slot5    36 [07/22/08 07:24:05] TELNET:10.1.20.1 rwa save bootconfig
Slot5    35 [07/22/08 07:24:02] TELNET:10.1.20.1 rwa config bootconfig choice secondary image-file /flash/p80a4160.img
Slot5    34 [07/22/08 07:24:01] TELNET:10.1.20.1 rwa config bootconfig choice primary image-file /flash/p80a4163.img
Slot5    33 [07/22/08 07:22:50] TFTP:127.0.0.6 get /p80a4163.img
Slot5    32 [07/22/08 07:22:36] TELNET:10.1.20.1 rwa peer telnet
Slot5    31 [07/22/08 07:22:33] TELNET:10.1.20.1 rwa mv /flash/p80a4163.mig /flash/p80a4163.img
Slot5    29 [07/22/08 07:22:01] TFTP:127.0.0.6 get /flash/p80a4163.img
Slot5    28 [07/22/08 07:21:43] TELNET:10.1.20.1 rwa peer telnet
Slot5    27 [07/22/08 07:21:42] TELNET:10.1.20.1 rwa mv /flash/p80a6163.img /flash/p804163.img
Slot5    26 [07/22/08 07:21:31] TELNET:10.1.20.1 rwa dir
Slot5    25 [07/22/08 07:21:19] TFTP:127.0.0.6 get /p80j4163.dld
Slot5    24 [07/22/08 07:17:26] TFTP:127.0.0.6 get /p80m4163.img
Slot5    23 [07/22/08 07:17:25] TFTP:127.0.0.6 get /p80c4163.aes
Slot5    22 [07/22/08 07:17:25] TFTP:127.0.0.6 get /p80c4163.img
Slot5    21 [07/22/08 07:17:19] TFTP:127.0.0.6 get /p80b4163.img
Slot5    20 [07/22/08 07:17:19] TFTP:127.0.0.6 get /p80a4163.img
Slot5    19 [07/22/08 07:16:33] TELNET:10.1.20.1 rwa peer telnet
Slot5    18 [07/22/08 07:16:22] TELNET:10.1.20.1 rwa dir
ERS-8610:5#

Cheers!