The Nortel Ethernet Routing Switch 8600 has multiple log files that can be examined for signs of a problem or during basic troubleshooting. The log files will be stored on the PCMCIA flash card or in memory if there is no flash card available. There is also a trace facility within the ERS 8600 that allows you to log extensive debug information within the switch on different subsystems and processes. I won’t go into the trace facility in this post but if your interested you can find it in by using the command “config trace” from the CLI interface of the ERS 8600 switch.
You can check the contents of the flash memory and PCMCIA cards by issuing the “dir” command;
ERS-8610:5# dir
size date time name
-------- ------ ------ --------
8773031 JUL-22-2008 06:11:38 /flash/p80a4163.img
373 JUL-22-2008 06:30:36 /flash/boot.cfg
8 AUG-16-2008 13:28:10 /flash/ospf_md5key.txt
79 AUG-16-2008 13:28:10 /flash/snmp_comm.txt
16440 AUG-16-2008 13:28:14 /flash/config.cfg
11 JUL-22-2008 07:33:30 /flash/engboot
1073634 JUL-22-2008 06:11:44 /flash/p80b4163.img
55928 JUL-22-2008 06:11:44 /flash/p80c4163.img
26112 JUL-22-2008 06:11:46 /flash/p80c4163.aes
8872259 JUL-22-2008 06:16:20 /flash/p80m4163.img
1272852 JUL-22-2008 06:17:12 /flash/p80j4163.dld
total: 64155648 used: 20639744 free: 43515904 bytes
size date time name
-------- ------ ------ --------
29671 AUG-16-2008 14:03:18 /pcmcia/7b700005.000
26192 AUG-16-2008 14:16:48 /pcmcia/clilog.txt
1 JUL-22-2008 06:21:20 /pcmcia/clilogoff
8 JUL-22-2008 07:33:30 /pcmcia/7b700005.num
total: 64710656 used: 584704 free: 64125952 bytesYou can examine the log files with the following commands;
show log file tail
ERS-8610:5# show log file tail CPU5 [07/22/08 07:44:05] SNMP INFO Communication established with backup CPU CPU5 [07/22/08 07:44:05] HW INFO System activity performed CPU5 [07/22/08 07:43:44] SNMP INFO CPU switch over, stand-by CPU in slot # 5 became master CPU5 [07/22/08 07:43:44] SNMP INFO Sending Warm-Start Trap CPU5 [07/22/08 07:43:44] SNMP INFO CPU switch over, stand-by CPU becoming master CPU5 [07/22/08 07:43:37] OSPF INFO Ospf Nbr State Change: rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 LOADING->FULL Event LOADING_DONE_EVENT CPU5 [07/22/08 07:43:37] OSPF INFO Ospf Nbr State Change: rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 LOADING->FULL Event LOADING_DONE_EVENT CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 EXCHANGE->LOADING Event EXCHANGE_DONE_EVENT CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 EXCHANGE->LOADING Event EXCHANGE_DONE_EVENT CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 EX_START->EXCHANGE Event NEGOTIATION_DONE_EVENT CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 EX_START->EXCHANGE Event NEGOTIATION_DONE_EVENT CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 2WAY->EX_START Event ADJ_OK_EVENT CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 2WAY->EX_START Event ADJ_OK_EVENT CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 INIT->2WAY Event TWO_WAY_EVENT CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.6 nbr-rtid:10.7.0.2 DOWN->INIT Event HELLO_RECEIVED_EVENT CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:0.0.0.0 nbr-rtid:10.7.0.2 NULL->DOWN Event HELLO_RECEIVED_EVENT CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 INIT->2WAY Event TWO_WAY_EVENT CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:10.7.1.5 nbr-rtid:10.7.0.1 DOWN->INIT Event HELLO_RECEIVED_EVENT CPU5 [07/22/08 07:43:32] OSPF INFO Ospf Nbr State Change : rtid:10.7.1.9, ipa:0.0.0.0 nbr-rtid:10.7.0.1 NULL->DOWN Event HELLO_RECEIVED_EVENT CPU5 [07/22/08 07:43:32] SNMP INFO Spanning Tree Topology Change(StgId=1, PortNum=2/1, MacAddr=00:1d:42:7b:70:01) CPU5 [07/22/08 07:43:02] NONE INFO Spanning Tree Topology Change. New Root bridge 00:04:38:70:70:01 for StgId = 1 CPU5 [07/22/08 07:43:02] SNMP INFO Link Up(2/1) CPU5 [07/22/08 07:43:00] SNMP INFO Vlacp link up(1/1) CPU5 [07/22/08 07:43:00] SNMP INFO Vlacp link down(1/1) CPU5 [07/22/08 07:43:00] SNMP INFO Link Up(1/2) CPU5 [07/22/08 07:43:00] SNMP INFO Link Up(1/1) CPU5 [07/22/08 07:43:00] SW INFO PCMCIA card detected in Master CPU "ERS-8610" slot 5, Chassis S/N SSPNDTxxxx CPU5 [07/22/08 07:43:00] SNMP INFO Fan Up(FanId=2, OperStatus=2) CPU5 [07/22/08 07:43:00] SNMP INFO Fan Up(FanId=1, OperStatus=2) CPU5 [07/22/08 07:43:00] HW INFO System activity performed CPU5 [07/22/08 07:43:01] SNMP INFO Vlacp link down(1/1) CPU5 [07/22/08 07:43:01] SNMP INFO Booted with PRIMARY boot image source - /flash/p80a4163.img CPU5 [07/22/08 07:43:01] SW INFO The system is ready CPU5 [07/22/08 07:43:01] SW INFO NTP Enabled CPU5 [07/22/08 07:43:00] SNMP INFO 2k card up(CardNum=2 AdminStatus=1 OperStatus=1) CPU5 [07/22/08 07:43:00] SW INFO Loading configuration from /flash/config.cfg CPU5 [07/22/08 07:43:00] HW INFO FFAD:Test Passed OK Slot:6 Tap:1 [07/22/08 07:43:00] The previous message repeated 1 time(s). CPU5 [07/22/08 07:43:00] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:6 Tap:1 CPU5 [07/22/08 07:43:00] HW INFO FFAD:Test Passed OK Slot:5 Tap:1 [07/22/08 07:43:00] The previous message repeated 1 time(s). CPU5 [07/22/08 07:43:00] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:5 Tap:1 CPU5 [07/22/08 07:43:00] SNMP INFO 2k card up(CardNum=1 AdminStatus=1 OperStatus=1) CPU5 [07/22/08 07:43:00] HW INFO Initializing 8648GTR in slot #2 ... CPU5 [07/22/08 07:42:59] HW INFO FFAD:Test Passed OK Slot:6 Tap:0 [07/22/08 07:42:59] The previous message repeated 1 time(s). CPU5 [07/22/08 07:42:59] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:6 Tap:0 CPU5 [07/22/08 07:42:59] HW INFO FFAD:Test Passed OK Slot:5 Tap:0 [07/22/08 07:42:59] The previous message repeated 1 time(s). CPU5 [07/22/08 07:42:59] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:5 Tap:0 CPU5 [07/22/08 07:42:59] HW INFO Initializing 8630GBR in slot #1 ... CPU5 [07/22/08 07:42:59] SW INFO slot 1 found NP heartbeat - R-Module is online CPU5 [07/22/08 07:42:57] SW INFO slot 2 found NP heartbeat - R-Module is online CPU5 [07/22/08 07:42:52] SW INFO Slot 2: Starting software version 4.1.6.3 CPU5 [07/22/08 07:42:51] SW INFO Slot 1: Starting software version 4.1.6.3 CPU5 [07/22/08 07:42:41] SW INFO Slot 2: Loading /flash/p80j4163.dld CPU5 [07/22/08 07:42:41] SW INFO Slot 1: Loading /flash/p80j4163.dld CPU5 [07/22/08 07:42:36] SNMP INFO 2k card up(CardNum=6 AdminStatus=1 OperStatus=1) CPU5 [07/22/08 07:42:36] HW INFO FFAD:Test Passed OK Slot:6 Tap:8 CPU5 [07/22/08 07:42:36] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:6 Tap:8 CPU5 [07/22/08 07:42:34] SNMP INFO 2k card up(CardNum=5 AdminStatus=1 OperStatus=1) CPU5 [07/22/08 07:42:34] HW INFO Initializing 8692SF in slot #6 ... CPU5 [07/22/08 07:42:34] HW INFO FFAD:Test Passed OK Slot:5 Tap:8 CPU5 [07/22/08 07:42:34] HW INFO FFAD Setup: Serializer-Deserializer Connection OK Slot:5 Tap:8 CPU5 [07/22/08 07:42:32] HW INFO Initializing 8692SF in slot #5 ... CPU5 [07/22/08 07:42:32] HW INFO Card inserted: Slot=6 Type=8692SF CPU5 [07/22/08 07:42:32] SW INFO R-Module inserted: Slot=2 Type=8648GTR, waiting to bootup... CPU5 [07/22/08 07:42:32] SW INFO R-Module inserted: Slot=1 Type=8630GBR, waiting to bootup... CPU5 [07/22/08 07:42:32] HW INFO Card inserted: Slot=5 Type=8692SF CPU5 [07/22/08 07:42:30] SW INFO Killing task tCXCTask after timeout CPU5 [07/22/08 07:42:28] SW INFO Killing task tChRxTask after timeout CPU5 [07/22/08 07:42:26] HW INFO Stand-by CPU in slot # 5 becoming master... CPU5 [07/22/08 07:33:34] HW INFO System activity performed CPU5 [07/22/08 07:33:32] SW INFO PCMCIA card detected in Stand-by CPU "ERS-8610" slot 5, Chassis S/N SSPNDTxxxx CPU5 [07/22/08 07:33:32] SW INFO Loading configuration from /flash/config.cfg CPU5 [07/22/08 07:33:30] SW INFO CPU card entering warm-standby mode... CPU5 [07/22/08 07:33:30] SW INFO ERS System Software Release 4.1.6.3 CPU5 [07/22/08 07:33:30] SW INFO System boot
If you have CLI logging enabled you can dump that log with the following command;
show clilog file tail
ERS-8610:5# show clilog file tail Slot5 5 [07/28/08 21:47:29] TELNET:10.1.198.53 rwa show log file tail Slot5 4 [07/28/08 21:47:27] TELNET:10.1.198.53 rwa box Slot5 3 [07/28/08 21:47:25] TELNET:10.1.198.53 rwa show log file tail Slot5 2 [07/28/08 21:47:09] TELNET:10.1.198.53 rwa info Slot5 1 [07/28/08 21:47:08] TELNET:10.1.198.53 rwa show ip route Slot5 62 [07/22/08 07:30:47] TELNET:10.1.20.1 rwa boot /flash/p80b4163.img Slot5 61 [07/22/08 07:30:23] TELNET:10.1.20.1 rwa dir Slot5 60 [07/22/08 07:30:15] TELNET:10.1.20.1 rwa peer telnet Slot5 59 [07/22/08 07:26:36] TELNET:10.1.20.1 rwa peer telnet Slot5 58 [07/22/08 07:26:34] TELNET:10.1.20.1 rwa box Slot5 57 [07/22/08 07:26:30] TELNET:10.1.20.1 rwa info Slot5 56 [07/22/08 07:26:28] TELNET:10.1.20.1 rwa ? Slot5 55 [07/22/08 07:26:05] TELNET:10.1.20.1 rwa more /flash/config.cfg Slot5 54 [07/22/08 07:25:54] TELNET:10.1.20.1 rwa info Slot5 53 [07/22/08 07:25:50] TELNET:10.1.20.1 rwa more /flash/boot.cfg Slot5 52 [07/22/08 07:25:41] TELNET:10.1.20.1 rwa save config Slot5 51 [07/22/08 07:25:38] TELNET:10.1.20.1 rwa save bootconfig Slot5 50 [07/22/08 07:25:27] TELNET:10.1.20.1 rwa info Slot5 49 [07/22/08 07:25:26] TELNET:10.1.20.1 rwa nifo Slot5 48 [07/22/08 07:25:25] TELNET:10.1.20.1 rwa choice secondary Slot5 47 [07/22/08 07:25:20] TELNET:10.1.20.1 rwa choice Slot5 46 [07/22/08 07:25:16] TELNET:10.1.20.1 rwa info Slot5 45 [07/22/08 07:25:13] TELNET:10.1.20.1 rwa choice secondary image-file /flash/p80a4160.img Slot5 44 [07/22/08 07:24:57] TELNET:10.1.20.1 rwa choice secondary ? Slot5 43 [07/22/08 07:24:51] TELNET:10.1.20.1 rwa choice Slot5 42 [07/22/08 07:24:34] TELNET:10.1.20.1 rwa choice image-file ? Slot5 41 [07/22/08 07:24:25] TELNET:10.1.20.1 rwa choice ? Slot5 40 [07/22/08 07:24:22] TELNET:10.1.20.1 rwa ? Slot5 39 [07/22/08 07:24:21] TELNET:10.1.20.1 rwa bootconfig Slot5 38 [07/22/08 07:24:19] TELNET:10.1.20.1 rwa config Slot5 37 [07/22/08 07:24:09] TELNET:10.1.20.1 rwa more /flash/boot.cfg Slot5 36 [07/22/08 07:24:05] TELNET:10.1.20.1 rwa save bootconfig Slot5 35 [07/22/08 07:24:02] TELNET:10.1.20.1 rwa config bootconfig choice secondary image-file /flash/p80a4160.img Slot5 34 [07/22/08 07:24:01] TELNET:10.1.20.1 rwa config bootconfig choice primary image-file /flash/p80a4163.img Slot5 33 [07/22/08 07:22:50] TFTP:127.0.0.6 get /p80a4163.img Slot5 32 [07/22/08 07:22:36] TELNET:10.1.20.1 rwa peer telnet Slot5 31 [07/22/08 07:22:33] TELNET:10.1.20.1 rwa mv /flash/p80a4163.mig /flash/p80a4163.img Slot5 29 [07/22/08 07:22:01] TFTP:127.0.0.6 get /flash/p80a4163.img Slot5 28 [07/22/08 07:21:43] TELNET:10.1.20.1 rwa peer telnet Slot5 27 [07/22/08 07:21:42] TELNET:10.1.20.1 rwa mv /flash/p80a6163.img /flash/p804163.img Slot5 26 [07/22/08 07:21:31] TELNET:10.1.20.1 rwa dir Slot5 25 [07/22/08 07:21:19] TFTP:127.0.0.6 get /p80j4163.dld Slot5 24 [07/22/08 07:17:26] TFTP:127.0.0.6 get /p80m4163.img Slot5 23 [07/22/08 07:17:25] TFTP:127.0.0.6 get /p80c4163.aes Slot5 22 [07/22/08 07:17:25] TFTP:127.0.0.6 get /p80c4163.img Slot5 21 [07/22/08 07:17:19] TFTP:127.0.0.6 get /p80b4163.img Slot5 20 [07/22/08 07:17:19] TFTP:127.0.0.6 get /p80a4163.img Slot5 19 [07/22/08 07:16:33] TELNET:10.1.20.1 rwa peer telnet Slot5 18 [07/22/08 07:16:22] TELNET:10.1.20.1 rwa dir ERS-8610:5#
Cheers!