27th
August
2008
We’ve just recently come across this problem and I thought it would be a great topic to share and perhaps even solicit some feedback from others. As you might already know I’ve been deploying ADAC across a large number of Nortel Ethernet Routing Switch 5520s with great success. ADAC allows the switch to control the phones voice VLAN configuration.
Well we also ran into a problem after upgrading a number of those switches to v5.1.1.17. A network administrator had made VLAN changes to various ports on the switch prior to the upgrade but after ADAC had been enabled on the ports. After the upgrade the switch ports defaulted back to the original VLAN they were configured for when ADAC was first enabled. We performed some additional testing and found that this problem would occur if the switch was just reset (rebooted) so it doesn’t appear to be tied to the upgrade but rather the action of restarting the switch. Looking at how ADAC works I can understand the problem but I’m disappointed that Device Manager or the CLI interface doesn’t throw a warning when you try to change the VLAN configuration of a port with ADAC enabled.
The lesson here is that you should disable ADAC on any port where you intend to change the VLAN membership.
Anyone else seen this?
Cheers!
posted in EthernetRtngSwitch, Nortel | | 576 views
22nd
August
2008
The Nortel Ethernet Routing Switch 8600 supports a feature called link flap detect. This feature is designed to detect a port whose link is flapping up and down and potentially disable the port and/or issue an SNMP trap. The feature allows the configuration of a specific frequency and interval which can be used to fine tune control of the feature.
A few years ago we deployed about 25 Voice Gateway Media Cards (VGMC) across five different Nortel Call Server 1000Ms. We almost immediately started to notice a problem between the VGMCs and the ERS 8600 switches. The VGMCs would link up and link down multiple times as they booted up. While there is technically no real problem with the VGMC you can run straight into a road block if you reboot the card multiple times in a short period of time. Without getting to crazy on the explanation the ERS 8600 switch would potentially disable the switch port(s) that connected to the VGMCs if they were rebooted to often during a 60 minute period. The problem was only observed while the VGMCs were booting up, once the VGMC was up and running the problem never occurred. I believe the link flap detect feature is enabled by default unless specifically disabled.
This may be an old problem that might have already been corrected by Nortel but I still have the link flap detect feature disabled on the Ethernet Routing Switch 8600s that connect to our Nortel Voice Gateway Media Cards.
ERS8600:5# config sys link-flap-detect interval 60
ERS8600:5# config sys link-flap-detect frequency 10
ERS8600:5# config sys link-flap-detect auto-port-down disable
ERS8600:5# config sys link-flap-detect send-trap enable
The configuration can be confirmed using the following command;
ERS8600:5# show sys link-flap-detect general-info
Auto Port Down: enable
Send Trap : enable
Interval : 60
Frequency : 10
The frequency defines how often a port can go down, where the default is 10 (times). The default interval is 60 minutes; therefore the port will be disabled if the port goes down 10 times within 60 minutes.
Cheers!
posted in EthernetRtngSwitch, Nortel, VoiceOverIP | | 274 views
17th
August
2008
The Nortel Ethernet Routing Switch 8600 will automatically search for several boot configuration files as it boots up. It will look for several files in a specific order;
- /pcmcia/pcmboot.cfg
- /flash/boot.cfg
- /pcmcia/boot.cfg
The file /pcmcia/pcmboot.cfg is designed to allow network engineers to deploy the ERS 8600 remotely instructing local site personnel to simply copy a file (pcmboot.cfg) that could be emailed to them to the PCMCIA flash card, inserted it into the switch and the switch booted up. I’ve never personally used this feature but I have remotely installed several ERS 8600s by having a modem attached to the console port.
What would a normal boot.cfg file look like?
Here’s the current standard that I’m using on the 37 ERS 8600 switches that I maintain.
#
# TUE JUL 22 07:25:38 2008 EDT
# box type : 8k boot configuration file
#
flags daylight-saving-time true
flags ftpd true
flags savetostandby true
flags telnetd true
flags tftpd true
flags verify-config false
tz dst-end M11.1.0/0200
tz dst-name "EDT"
tz dst-start M3.2.0/0200
tz name "EST"
tz offset-from-utc 300
choice primary image-file "/flash/p80a4163.img"
The commands should be self explanatory for the most part. The last line of the file instructs the switch which agent code it should boot, in this example I’m running 4.1.6.3 software so we’ll boot /flash/p80a4163.img.
Cheers!
posted in EthernetRtngSwitch, Nortel | | 480 views
16th
August
2008
A few weeks ago a few friends approached me about Internet based VoIP solutions for their home phone. They were fed up with the $100.00+ phone bills and weren’t really excited about giving any more money to the local cable television company(Comcast). I’ve been an
AT&T CallVantage VoIP customer for the past 2-3 years so I was obviously ready to recommend AT&T CallVantage until I discovered that they are no longer accepting new customers.
While I wasn’t ready to recommend
Vonage or any of the other solutions out there I did comment to them about the recent buzz around a product called
MagicJack. The MagicJack USB adapter itself costs approximately $39.95 and includes the first year of service free while subsequent years are $19.95 a year (yes you read that right $19.95/year). The solution requires a Windows XP or MacOS desktop/laptop and utilizes your broadband Internet connection. I personally know of two folks that are currently utilizing the solution and they absolutely love it and they are admittingly not very technical or computer savy. However, they simply love the solution and they both estimate that it’s saving them between $75 and $100 a month in long distance phone charges. The solution has scored numerous product awards including
PC Magazine’s Editor’s Choice award.
So while I’m not exactly sure what I’ll do myself since it’s probably only a matter of time until AT&T pulls the plug on CallVanage it seems like MagicJack could be a great solution for those teenagers heading off to college. They’d no longer have an excuse for not calling home every once-n-while. :)
Cheers!
posted in PersonalComputing | | 383 views
16th
August
2008
The Nortel Ethernet Routing Switch 8600 has multiple log files that can be examined for signs of a problem or during basic troubleshooting. The log files will be stored on the PCMCIA flash card or in memory if there is no flash card available. There is also a trace facility within the ERS 8600 that allows you to log extensive debug information within the switch on different subsystems and processes. I won’t go into the trace facility in this post but if your interested you can find it in by using the command “config trace” from the CLI interface of the ERS 8600 switch.
You can check the contents of the flash memory and PCMCIA cards by issuing the “dir” command;
ERS-8610:5# dir
size date time name
-------- ------ ------ --------
8773031 JUL-22-2008 06:11:38 /flash/p80a4163.img
373 JUL-22-2008 06:30:36 /flash/boot.cfg
8 AUG-16-2008 13:28:10 /flash/ospf_md5key.txt
79 AUG-16-2008 13:28:10 /flash/snmp_comm.txt
16440 AUG-16-2008 13:28:14 /flash/config.cfg
11 JUL-22-2008 07:33:30 /flash/engboot
1073634 JUL-22-2008 06:11:44 /flash/p80b4163.img
55928 JUL-22-2008 06:11:44 /flash/p80c4163.img
26112 JUL-22-2008 06:11:46 /flash/p80c4163.aes
8872259 JUL-22-2008 06:16:20 /flash/p80m4163.img
1272852 JUL-22-2008 06:17:12 /flash/p80j4163.dld
total: 64155648 used: 20639744 free: 43515904 bytes
size date time name
-------- ------ ------ --------
29671 AUG-16-2008 14:03:18 /pcmcia/7b700005.000
26192 AUG-16-2008 14:16:48 /pcmcia/clilog.txt
1 JUL-22-2008 06:21:20 /pcmcia/clilogoff
8 JUL-22-2008 07:33:30 /pcmcia/7b700005.num
total: 64710656 used: 584704 free: 64125952 bytes
You can examine the log files with the following commands;
show log file tail
ERS-8610:5# show log file tail
CPU5 [07/22/08 07:44:05] SNMP INFO Communication established with backup CPU
CPU5 [07/22/08 07:44:05] HW INFO System activity performed
CPU5 [07/22/08 07:43:44] SNMP INFO CPU switch over, stand-by CPU in slot # 5 became master
CPU5 [07/22/08 07:43:44] SNMP INFO Sending Warm-Start Trap
CPU5 [07/22/08 07:43:44] SNMP INFO CPU switch over, stand-by CPU becoming master
CPU5 [07/22/08 07:43:37] OSPF INFO Ospf Nbr State Change: rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 LOADING->FULL Event LOADING_DONE_EVENT
CPU5 [07/22/08 07:43:37] OSPF INFO Ospf Nbr State Change: rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 LOADING->FULL Event LOADING_DONE_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 EXCHANGE->LOADING Event EXCHANGE_DONE_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 EXCHANGE->LOADING Event EXCHANGE_DONE_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 EX_START->EXCHANGE Event NEGOTIATION_DONE_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 EX_START->EXCHANGE Event NEGOTIATION_DONE_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 2WAY->EX_START Event ADJ_OK_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 2WAY->EX_START Event ADJ_OK_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 INIT->2WAY Event TWO_WAY_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 DOWN->INIT Event HELLO_RECEIVED_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:0.0.0.0 nbr-rtid:10.7.0.2 NULL->DOWN Event HELLO_RECEIVED_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 INIT->2WAY Event TWO_WAY_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 DOWN->INIT Event HELLO_RECEIVED_EVENT
CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:0.0.0.0 nbr-rtid:10.7.0.1 NULL->DOWN Event HELLO_RECEIVED_EVENT
CPU5 [07/22/08 07:43:32] SNMP INFO Spanning Tree Topology Change(StgId=1, PortNum=2/1, MacAddr=00:1d:42:7b:70:01)
CPU5 [07/22/08 07:43:02] NONE INFO Spanning Tree Topology Change. New Root bridge 00:04:38:70:70:01 for StgId = 1
CPU5 [07/22/08 07:43:02] SNMP INFO Link Up(2/1)
CPU5 [07/22/08 07:43:00] SNMP INFO Vlacp link up(1/1)
CPU5 [07/22/08 07:43:00] SNMP INFO Vlacp link down(1/1)
CPU5 [07/22/08 07:43:00] SNMP INFO Link Up(1/2)
CPU5 [07/22/08 07:43:00] SNMP INFO Link Up(1/1)
CPU5 [07/22/08 07:43:00] SW INFO PCMCIA card detected in Master CPU "ERS-8610" slot 5, Chassis S/N SSPNDTxxxx
CPU5 [07/22/08 07:43:00] SNMP INFO Fan Up(FanId=2, OperStatus=2)
CPU5 [07/22/08 07:43:00] SNMP INFO Fan Up(FanId=1, OperStatus=2)
CPU5 [07/22/08 07:43:00] HW INFO System activity performed
CPU5 [07/22/08 07:43:01] SNMP INFO Vlacp link down(1/1)
CPU5 [07/22/08 07:43:01] SNMP INFO Booted with PRIMARY boot image source - /flash/p80a4163.img
CPU5 [07/22/08 07:43:01] SW INFO The system is ready
CPU5 [07/22/08 07:43:01] SW INFO NTP Enabled
CPU5 [07/22/08 07:43:00] SNMP INFO 2k card up(CardNum=2 AdminStatus=1 OperStatus=1)
CPU5 [07/22/08 07:43:00] SW INFO Loading configuration from /flash/config.cfg
CPU5 [07/22/08 07:43:00] HW INFO FFAD:Test Passed OK Slot:6 Tap:1
[07/22/08 07:43:00] The previous message repeated 1 time(s).
CPU5 [07/22/08 07:43:00] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:6 Tap:1
CPU5 [07/22/08 07:43:00] HW INFO FFAD:Test Passed OK Slot:5 Tap:1
[07/22/08 07:43:00] The previous message repeated 1 time(s).
CPU5 [07/22/08 07:43:00] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:5 Tap:1
CPU5 [07/22/08 07:43:00] SNMP INFO 2k card up(CardNum=1 AdminStatus=1 OperStatus=1)
CPU5 [07/22/08 07:43:00] HW INFO Initializing 8648GTR in slot #2 ...
CPU5 [07/22/08 07:42:59] HW INFO FFAD:Test Passed OK Slot:6 Tap:0
[07/22/08 07:42:59] The previous message repeated 1 time(s).
CPU5 [07/22/08 07:42:59] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:6 Tap:0
CPU5 [07/22/08 07:42:59] HW INFO FFAD:Test Passed OK Slot:5 Tap:0
[07/22/08 07:42:59] The previous message repeated 1 time(s).
CPU5 [07/22/08 07:42:59] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:5 Tap:0
CPU5 [07/22/08 07:42:59] HW INFO Initializing 8630GBR in slot #1 ...
CPU5 [07/22/08 07:42:59] SW INFO slot 1 found NP heartbeat - R-Module is online
CPU5 [07/22/08 07:42:57] SW INFO slot 2 found NP heartbeat - R-Module is online
CPU5 [07/22/08 07:42:52] SW INFO Slot 2: Starting software version 4.1.6.3
CPU5 [07/22/08 07:42:51] SW INFO Slot 1: Starting software version 4.1.6.3
CPU5 [07/22/08 07:42:41] SW INFO Slot 2: Loading /flash/p80j4163.dld
CPU5 [07/22/08 07:42:41] SW INFO Slot 1: Loading /flash/p80j4163.dld
CPU5 [07/22/08 07:42:36] SNMP INFO 2k card up(CardNum=6 AdminStatus=1 OperStatus=1)
CPU5 [07/22/08 07:42:36] HW INFO FFAD:Test Passed OK Slot:6 Tap:8
CPU5 [07/22/08 07:42:36] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:6 Tap:8
CPU5 [07/22/08 07:42:34] SNMP INFO 2k card up(CardNum=5 AdminStatus=1 OperStatus=1)
CPU5 [07/22/08 07:42:34] HW INFO Initializing 8692SF in slot #6 ...
CPU5 [07/22/08 07:42:34] HW INFO FFAD:Test Passed OK Slot:5 Tap:8
CPU5 [07/22/08 07:42:34] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:5 Tap:8
CPU5 [07/22/08 07:42:32] HW INFO Initializing 8692SF in slot #5 ...
CPU5 [07/22/08 07:42:32] HW INFO Card inserted: Slot=6 Type=8692SF
CPU5 [07/22/08 07:42:32] SW INFO R-Module inserted: Slot=2 Type=8648GTR, waiting to bootup...
CPU5 [07/22/08 07:42:32] SW INFO R-Module inserted: Slot=1 Type=8630GBR, waiting to bootup...
CPU5 [07/22/08 07:42:32] HW INFO Card inserted: Slot=5 Type=8692SF
CPU5 [07/22/08 07:42:30] SW INFO Killing task tCXCTask after timeout
CPU5 [07/22/08 07:42:28] SW INFO Killing task tChRxTask after timeout
CPU5 [07/22/08 07:42:26] HW INFO Stand-by CPU in slot # 5 becoming master...
CPU5 [07/22/08 07:33:34] HW INFO System activity performed
CPU5 [07/22/08 07:33:32] SW INFO PCMCIA card detected in Stand-by CPU "ERS-8610" slot 5, Chassis S/N SSPNDTxxxx
CPU5 [07/22/08 07:33:32] SW INFO Loading configuration from /flash/config.cfg
CPU5 [07/22/08 07:33:30] SW INFO CPU card entering warm-standby mode...
CPU5 [07/22/08 07:33:30] SW INFO ERS System Software Release 4.1.6.3
CPU5 [07/22/08 07:33:30] SW INFO System boot
If you have CLI logging enabled you can dump that log with the following command;
show clilog file tail
ERS-8610:5# show clilog file tail
Slot5 5 [07/28/08 21:47:29] TELNET:10.1.198.53 rwa show log file tail
Slot5 4 [07/28/08 21:47:27] TELNET:10.1.198.53 rwa box
Slot5 3 [07/28/08 21:47:25] TELNET:10.1.198.53 rwa show log file tail
Slot5 2 [07/28/08 21:47:09] TELNET:10.1.198.53 rwa info
Slot5 1 [07/28/08 21:47:08] TELNET:10.1.198.53 rwa show ip route
Slot5 62 [07/22/08 07:30:47] TELNET:10.1.20.1 rwa boot /flash/p80b4163.img
Slot5 61 [07/22/08 07:30:23] TELNET:10.1.20.1 rwa dir
Slot5 60 [07/22/08 07:30:15] TELNET:10.1.20.1 rwa peer telnet
Slot5 59 [07/22/08 07:26:36] TELNET:10.1.20.1 rwa peer telnet
Slot5 58 [07/22/08 07:26:34] TELNET:10.1.20.1 rwa box
Slot5 57 [07/22/08 07:26:30] TELNET:10.1.20.1 rwa info
Slot5 56 [07/22/08 07:26:28] TELNET:10.1.20.1 rwa ?
Slot5 55 [07/22/08 07:26:05] TELNET:10.1.20.1 rwa more /flash/config.cfg
Slot5 54 [07/22/08 07:25:54] TELNET:10.1.20.1 rwa info
Slot5 53 [07/22/08 07:25:50] TELNET:10.1.20.1 rwa more /flash/boot.cfg
Slot5 52 [07/22/08 07:25:41] TELNET:10.1.20.1 rwa save config
Slot5 51 [07/22/08 07:25:38] TELNET:10.1.20.1 rwa save bootconfig
Slot5 50 [07/22/08 07:25:27] TELNET:10.1.20.1 rwa info
Slot5 49 [07/22/08 07:25:26] TELNET:10.1.20.1 rwa nifo
Slot5 48 [07/22/08 07:25:25] TELNET:10.1.20.1 rwa choice secondary
Slot5 47 [07/22/08 07:25:20] TELNET:10.1.20.1 rwa choice
Slot5 46 [07/22/08 07:25:16] TELNET:10.1.20.1 rwa info
Slot5 45 [07/22/08 07:25:13] TELNET:10.1.20.1 rwa choice secondary image-file /flash/p80a4160.img
Slot5 44 [07/22/08 07:24:57] TELNET:10.1.20.1 rwa choice secondary ?
Slot5 43 [07/22/08 07:24:51] TELNET:10.1.20.1 rwa choice
Slot5 42 [07/22/08 07:24:34] TELNET:10.1.20.1 rwa choice image-file ?
Slot5 41 [07/22/08 07:24:25] TELNET:10.1.20.1 rwa choice ?
Slot5 40 [07/22/08 07:24:22] TELNET:10.1.20.1 rwa ?
Slot5 39 [07/22/08 07:24:21] TELNET:10.1.20.1 rwa bootconfig
Slot5 38 [07/22/08 07:24:19] TELNET:10.1.20.1 rwa config
Slot5 37 [07/22/08 07:24:09] TELNET:10.1.20.1 rwa more /flash/boot.cfg
Slot5 36 [07/22/08 07:24:05] TELNET:10.1.20.1 rwa save bootconfig
Slot5 35 [07/22/08 07:24:02] TELNET:10.1.20.1 rwa config bootconfig choice secondary image-file /flash/p80a4160.img
Slot5 34 [07/22/08 07:24:01] TELNET:10.1.20.1 rwa config bootconfig choice primary image-file /flash/p80a4163.img
Slot5 33 [07/22/08 07:22:50] TFTP:127.0.0.6 get /p80a4163.img
Slot5 32 [07/22/08 07:22:36] TELNET:10.1.20.1 rwa peer telnet
Slot5 31 [07/22/08 07:22:33] TELNET:10.1.20.1 rwa mv /flash/p80a4163.mig /flash/p80a4163.img
Slot5 29 [07/22/08 07:22:01] TFTP:127.0.0.6 get /flash/p80a4163.img
Slot5 28 [07/22/08 07:21:43] TELNET:10.1.20.1 rwa peer telnet
Slot5 27 [07/22/08 07:21:42] TELNET:10.1.20.1 rwa mv /flash/p80a6163.img /flash/p804163.img
Slot5 26 [07/22/08 07:21:31] TELNET:10.1.20.1 rwa dir
Slot5 25 [07/22/08 07:21:19] TFTP:127.0.0.6 get /p80j4163.dld
Slot5 24 [07/22/08 07:17:26] TFTP:127.0.0.6 get /p80m4163.img
Slot5 23 [07/22/08 07:17:25] TFTP:127.0.0.6 get /p80c4163.aes
Slot5 22 [07/22/08 07:17:25] TFTP:127.0.0.6 get /p80c4163.img
Slot5 21 [07/22/08 07:17:19] TFTP:127.0.0.6 get /p80b4163.img
Slot5 20 [07/22/08 07:17:19] TFTP:127.0.0.6 get /p80a4163.img
Slot5 19 [07/22/08 07:16:33] TELNET:10.1.20.1 rwa peer telnet
Slot5 18 [07/22/08 07:16:22] TELNET:10.1.20.1 rwa dir
ERS-8610:5#
Cheers!
posted in EthernetRtngSwitch, Nortel | | 559 views
14th
August
2008
I recently spent some time trying to figure out why there was an “*” (asterisk) in the CLI prompt on a Motorola RFS7000 that I had in our testlab. Jim (Motorola) explained that the Motorola WS5100 (v3.x) and the RFS7000 (v1.x) will place a “*” (asterisk) at the end of the hostname in the CLI prompt if there is a core dump file or crash log that hasn’t been cleared from memory. You can clear the dump files along with all service logs using the command “service clear all”. Once I issued this command the “*” (asterisk) disappeared from the CLI prompt and all was well again.
RFS7000*>
RFS7000*>enable
RFS7000*#service clear ?
all Remove all core, dump and panic files
aplogs Remove all local ap log files (does not clear them off the AP)
clitree Remove clitree.html (created by the save-cli command)
cores Remove all core files
dumps Remove all dump files
panics Remove all kernel panic files
securitymgr Securitymgr parameters
RFS7000*#>service clear all
RFS7000#
Cheers!
posted in Motorola, WirelessLANSwitch | | 175 views
Loading ...
