Posts tagged NEXUS

New Data Center – Tidbits

2
June 15, 2010
by Michael McNamara in ,

I thought I would follow-up my last post with some additional tidbits and background on the data center build.

Let me start by saying that we haven’t abandoned Nortel/Avaya as a viable network electronics vendor. When we first started this effort back in January 2009 we initially looked at Brocade, Cisco, HP, Juniper and Nortel. After the initial cut we were left with Brocade, Cisco and Nortel/Avaya. When the final decision was made, back in November 2009, there was still a lot of concern about Avaya’s purchase of Nortel’s Enterprise Division so the decision really came down to Brocade or Cisco. Ultimately the decision was made to go with Cisco. As I commented in the previous post we still have well over 33,000 ports on Nortel/Avaya switches and less than 1,000 ports on Cisco switches. We are still purchasing and installing/deploying Nortel/Avaya switches in our hospitals and business offices .

Nexus 5010 Virtual PortChannel bug?

I’ve been known to tell people that we’re a partner with Nortel/Avaya because of the number of bugs that we identify and help to resolve.  Well if I thought it was going to be any different with Cisco I can throw that idea right out the window as we have identified a bug in the vPC code for the Nexus 5000 switch. We discovered the issue with a pair of Nexus 5010 switches when one of the switches up and died (hardware failure). While trying to troubleshoot the problem we restarted the remaining Nexus 5010 switch only to find that the remaining Nexus 5010 would no longer adopt the Nexus 2148 switches/FEXes after it was rebooted if the peer Nexus 5010 wasn’t online. If we removed the vpc configuration from the portchannel then the FEX would come online. After speaking with Cisco they have created bug CSCth01969 which is supposedly going to be addressed in the next software release.

HP Virtual Connect Flex-10/NIC issues/ESX 4.0 Update 1 Drivers

We also ran into a number of issues with the HP BL490c blades and HP Virtual Connect Flex-10 Interconnect. On a number of blades we had an issue where the ESX installer was unable to locate any network interfaces. The installer would error with something like “The script 32.networking-drivers failed to execute and the installation can not continue.” The solution was to re-apply the NIC firmware after the Virtual Connect profile had been applied to the server. You can find additional detail over at Brian’s blog if interested.

We also ran into another issue when performing our redundancy and high-availability testing. We configured Virtual Connect to use SmartLink, however, when we unplugged the uplinks from the B side interconnect the Virtual Connect Flex-10 interconnect didn’t disable the server side NICs. The solution to this problem was to update the NIC driver that shipped with ESX 4.0 Update 1. You can find the driver here.

Lots of additional stories to come including a comparison of vPC and SMLT.

Cheers!

New Data Center – Where have I been?

8
June 11, 2010
by Michael McNamara in

I thought I would post a few quick words on where I’ve been for the past 2 months (certainly not writing quality content for this blog).  The past 60 days have been very hectic as I’ve started down the final stretch of designing, building and lighting a new data center. Thankfully the team and I are no strangers to moving computer rooms or constructing new buildings so we’re keenly aware of all the technical details needed to be successful in such a large endeavor.

I have so many short stories to share but no time to share them… In any event I’m now getting up to speed with a lot of new equipment, specifically Cisco’s Nexus gear.

What equipment did we use?

  • Cisco Nexus 7010
  • Cisco Nexus 5010
  • Cisco Nexus 2148
  • Cisco Catalyst 3750E
  • Cisco Catalyst 2960G
  • Cisco ASA5520
  • Cisco ACE 4710
  • Cisco AS5300 (yes we still have some dial-up users/vendors)
  • Cisco 7301 Router
  • Cisco 2821 Router

What racks did we use for the network equipment?

  • Liebert Knurr Racks
  • Liebert MPH/MPX PDUs

What equipment did we use for the servers/blades?

  • HP Rack 10000 G2
  • HP Rack PDU (AF503A)
  • HP IP KVM Console (AF601A)
  • HP BladeSystem c7000 Enclosure
  • HP Virtual Connect Flex-10 Interconnect
  • HP SAN 8Gb Interconnect
  • HP BL460c G6
  • HP BL490c G6
  • HP DL380 G6
  • HP DL360 G6

What are we using for storage?

  • IBM XIV System Storage (SAN) (w/4 1Gbps iSCSI replication ports)
  • IBM SAN80B-4 SAN Switch

Additional miscellaneous equipment;

  • MRV LX-4048T (terminal server)
  • Brother P-Touch PT-2100 / Brady ID PRO Plus label makers

As some of you might know we selected Cisco as the network electronics vendor and have implemented their Cisco Nexus 7010 switches as our core, followed by the Nexus 5010 switches as distribution to the Nexus 2148 (FEX) switches in a top of rack configuration. We also utilized Catalyst 2960G switches for our management/out-of-band network given that the Nexus 2148 only supports 1000BaseT, no 10Mbps or 100Mbps connectivity. Of course Cisco is in the process of releasing the Nexus 2248 which supports 100/1000Mbps connectivity to edge devices. We chose to utilize the HP Virtual Connect Flex-10 in our VM enclosures and we’ll utilize the Cisco 3120X in our non-VM enclosures. We’ve also installed and configured the Nexus 1000V in coordination with our VMware vSphere 4 environment. We decided that the CEE/DCE/FCoE revolution wasn’t quite here yet, or perhaps we weren’t quite ready for it so we stayed with a traditional Fiber Channel infrastructure around two IBM (oem Brocade) 80 port 8Gbps SAN switches. For SAN replication we’ll be using multiple 1Gbps iSCSI ports over a 10GE WAN. We ended up with an IBM XIV so we’ll have to see if it can keep up with all the traffic that’s going to be thrown it’s way.

So there should certainly be no shortage of material to talk about with all this new equipment, however, I’m certainly going to be very busy the next six months.

Here are some pictures of the cage (under 800 sq ft) , if interested. You’ll notice the chair and the upgrade that we performed on it in the last two pictures.

Cheers!

Go to Top