platypus is our little joke in a few different ways… juniper is not entering the storage business however there are no storage arrays supporting end to end fcoe using VN2VN (layer 2) and so the only way for us to demo is for us to make a mock storage device. by the way not just the switch (and its version of junos with vn2vn fip snooping) but also the server (and its vn2vn driver) are full production code. we are hoping to see real vn2vn storage and know some customers who are asking their storage vendors for it.

that day was the first time (i remember) also hearing the term latency bubble :-) and we would be very happy for you to blog about it – as i note there is much missunderstanding on the area of incast, microburst, congestion, and its increasing importance in the modern datacenter.

Thanks for taking the time to post a reply.

I was confused myself about the QFX3500 since I had gleaned from industry rags that it had been shipping for sometime. I will admit that it was really loud and difficult to catch a lot of what was being said while it was running – not sure if it was the switch or servers that was making all the noise. It was only when I went back to the actual video footage that I heard a lot of what was being said. Thanks for clarifying the point – I’ll issue a correction in the article above.

Any additional information available online regarding the prototype storage array named Platypus? Is it just a development platform for internal testing or is it an actual product offering that Juniper is developing?

Thanks for the explanation around the term ‘latency bubble’. I understood the meaning behind it but had never heard the term myself until that day. If you don’t mind I’ll probably feature your definition in an upcoming blog post – What’s a latency bubble?

Cheers!

firstly a couple of fun points:

1) it was to be precist the famous juniper prototype storage array nickname platypus that a) crashed and b) was not released code. the QFX3500 was production hw running production software and did not crash at all – though i know it was noisy !!!

2) my distinguished engineer, friend, and former partical physisist may think it funny that you refer to the term ‘latency bubble’ as bullshit lingo because it was him and not me the product manager (talking sales and marketing) that used the term…

ok, latency bubble… we used the term because the term ‘microburst’ does not well describe the real issue in hand and anyway is often miss discribed both in terms of what it is and what the effect is… putting my marketing hat on and remembering i am british… if you have a water system (water radiators in your car or your house) bubbles trapped in the system and moving around can cause all sorts of nasty things to happen including your car overheating and breaking down.

now, back to technical, if i have a single switch with a single wire speed asic and run all my ports at the same speed and have a pair of devices talking to each other then i will have very consistent latency especially if the destination device can also soack traffic at wire speed (this only matters for lossless). this is because traffic will drain out at wire speed and so you can NEVER get queue build up and so never get a ‘bubble of high latency’ or micro burst (or nano burst or mili burst depending on duration of traffic surge).

however, even in a completely wire speed non blocking any to any switch if you introduce any form of speed missmatch then you can get spikes of queue build up when the faster input(s) exceed the slower output. this is missleadingly referred to as a microbirst problem and its really an incast problem or speed missmatch problem that occurs when you have a brief period of time (hence micro birst) when input traffic vollume exceeds output traffic capabilitiy.

your speed missmatch could be a 10g input going to a 1g output, or it could be multiple 10g inputs feeding to a single 10g output (aka incast). in the later case you can see that you ONLY have a problem if you get a coordinated birst of traffic on multiple inputs at the same time.

this causes a bubble of increased latency as you get a egress q build up, on best effort if you exceed q depth you will then get egress drop. on lossless you will then get back propogation of congestion from egress to ingress and then to the next device.

now, going back to my water system analysis, this CAN be a problem when you have a single switch, however its even more likely to be a problem in a cascaded network of switches, as the microbursts will propogate. firstly most networks have oversubscription and so naturally generate incast from access to aggregation and aggregation to access, and then the first switch we have a little spike, this cases a birst of traffic on the link to the next switch, somewhere else on the network there was another similar burst, multiple of the bursts cross the aggregation switch, these bursts (or bubbles) meet and become bigger bursts, etc etc.

in other words the more complex the network the more likely you are to see these problem, and rumors to the contrary building switches or networks with no oversubscription DOES NOT prevent the problem with the nature of the traffic patterns we see increasingly.

anyway, having joe my techy sidekick using the term latency bubble i as product manager actually think i will use the term more along with the analogy of as i say bubbles in my water cooling system…

hope that helps :-)