Layer 3 and vPC Recommendations

I was provided a copy of the Nexus 7000 virtual Port-Channel Best Practices & Design Guidelines which clearly indicates on page 25 that routers should not be connected to a vPC link but should instead be connected via a Layer 3 switch port. Here are some bullet points;

• Use separate L3 links to hook up routers to a vPC domain is still standing.
• Don’t use L2 port channel to attach routers to a vPC domain unless you can statically route to HSRP address
• If both, routed and bridged traffic is required, use individual L3 links for routed traffic and L2 port-channel for bridged traffic

I was still currious to understand more of the inner-workings.. why didn’t it work or wasn’t it allowed? I only had to flip through the next few slides although I can really say that I completely understand just yet.

1. Packet arrives at R
2. R does lookup in routing table and sees 2 equal paths going north (to 7k1 & 7k2)
3. Assume it chooses 7k1 (ECMP decision)
4. R now has rewrite information to which router it needs to go (router MAC 7k1 or 7k2)
5. L2 lookup happens and outgoing interface is port-channel 1
6. Hashing determines which port-channel member is chosen (say to 7k2)
7. Packet is sent to 7k2
8. 7k2 sees that it needs to send it over the peer-link to 7k1 based on MAC address
9. 7k1 performs lookup and sees that it needs to send to S
10. 7k1 performs check if the frame came over peer link & is going out on a vPC.
11. Frame will only be forwarded if outgoing interface is NOT a vPC or if outgoing vPC doesn’t have active interface on other vPC peer (in our example 7k2)

I’m not embarrassed to say that I followed everything up until step 11. Why exactly is it that frames will only be forwarded if the outgoing interface is NOT a vPC or if the outgoing vPC doesn’t have an active interface on another vPC peer? Isthere anyone that can shed any additional light on this topic?

I’ve never experienced such a restriction in all my years of working with the Avaya (formerly Nortel) Ethernet Routing Switch 8600 and their Split Multilink Trunking (SMLT) technology. I actually have a Cisco 3825 router connected via a SMLT attached Ethernet Routing Switch 5520 (Layer 2) with the Cisco 3825 and the Avaya 8600s all running BGP.

Cheers!

With previous Cisco switches such as the 6509, 3750, 2960, etc we know that the following commands (when sent via a Perl script using the Net-SNMP Perl module) would instruct the switch to copy it’s running-config to a TFTP server.

snmpset -v1 -c$COMMUNITY $HOST ccCopyProtocol.$RANDOM i 1
snmpset -v1 -c$COMMUNITY $HOST ccCopySourceFileType.$RANDOM i 4
snmpset -v1 -c$COMMUNITY $HOST ccCopyDestFileType.$RANDOM i 1
snmpset -v1 -c$COMMUNITY $HOST ccCopyServerAddress.$RANDOM a "10.1.1.50"
snmpset -v1 -c$COMMUNITY $HOST ccCopyFileName.$RANDOM s "sw-train-acme.cfg"
snmpset -v1 -c$COMMUNITY $HOST ccCopyEntryRowStatus.$RANDOM i 1
sleep 5
snmpget -v1 -c$COMMUNITY $HOST ccCopyState.$RANDOM
#if not successful sleep 3 and re-check ccCopyState else continue and destroy table entry
snmpset -v1 -c$COMMUNITY $HOST ccCopyEntryRowStatus.$RANDOM i 6

I know that the both the Cisco Nexus 7010 and 5010 both balk at the SNMP OIDS/MIBS used above. So I’m searching for a set of equivalent SNMP OIDS/MIBS as those in CISCO-CONFIG-COPY-MIB for NX-OS. I’m not sure that such a OID/MIB even exists for NX-OS but it doesn’t hurt to search and ask.

I’m curious if anyone else has come across this issue? I know that there is an XML interface available but I would prefer to keep using the PERL/SNMP script that I’ve already developed. In the interim I’ll probably write an Expect script (or add some Expect code to my existing Perl script) to remotely connect to the switches and issue the appropriate copy commands.

Cheers!

Updated: Monday June 27, 2011

I’ve finally found the issue and now I’m able to backup the Cisco Nexus switches as expected.
[ad name=”ad-articlefooter”]