It’s Here: Revealing the QFabric™ Architectu​re

by andyingram







At last the big day is here!  Juniper has been talking about the power of a data center fabric for two years, since we first publicly disclosed the existence of Project Stratus.  We’ve discussed why we were building a fabric and what it would mean to our customers.  But we have intentionally been silent on the specifics of the architecture and the underlying technological magic.

That changes today.  In simultaneous locations around the world, we are sharing the secrets of QFabric, the fruit of the Stratus project, via webcast; and introducing the first component of the fabric, the QFX3500 Switch.

A few hundred years ago, a Franciscan friar named William of Ockham professed a concept that today is known as Occam's Razor.  He stated that when confronted with multiple alternatives, the simplest path is usually the correct one.  That is the defining concept behind QFabric.  Pradeep Sindhu, our founder and spiritual leader, looked at the data center network and saw a simpler, more correct path to solve this most difficult of network problems.  We believe the path he chose can eventually transform every data center in the world.

I have written extensively on why flat fabrics are the only natural solution for interconnecting the infrastructure of the modern data center.  Other vendors are attempting to build data center fabrics using their existing Ethernet switching components, but instead of focusing on the network, they focused on their switches.  It’s like being a brick manufacturer and deciding to build a high-rise building in California entirely out of bricks.  It is possible; it’s just a really bad idea.

The Simplest Path

Pradeep took a clean sheet approach, the simplest path.  That it would be a flat fabric was a given.  That it would adhere to network standards and not change the way applications, servers, storage, and other infrastructure components connect to the network was also a given.  Where the brilliance lies is in the inherent simplification of the network.  Push the intelligence to the edge of the fabric.  Minimize the amount of processing and hardware required to transport data across the fabric while maintaining any-to-any connectivity.  Enable the fabric to scale from tens of ports to thousands or even tens of thousands of ports while maintaining the simplest and most proven operational model—that of a single switch.

The challenge with describing new products that fundamentally transform the world around them is placing them in the right context to discuss and measure the revolution.  Simply put, the QFabric delivers the performance and operational simplicity of a single switch while delivering the scale and resilience of a network.  We believe there is nothing else like it in the world.

To achieve this, Pradeep and the architects rethought almost every aspect of a switch.  The resulting innovation is captured in more than 125 new patent applications filed as part of the project.

Design Principles of QFabric

QFabric is built on three design principles:

1. Build a data plane that ensures any-to-any connectivity across the entire data center while minimizing network processing.  This requires pushing the intelligence necessary to process incoming packets and handle  the complexity of the real world to the edge of the fabric, The interior of the fabric focuses on merely “transporting” the bits—a simpler, less costly activity.  This is the only way to enable the fabric to economically scale across the entire data center while delivering the blazing performance and low jitter required by modern applications.
2. Build a control plane that distributes and federates the real-time control of the fabric, placing the intelligence that controls the fabric into every edge node to ensure low latencies and inherent resilience.  Then provide an out-of-band control plane to ensure the federation of this intelligence, enabling the fabric to behave as a single switch without resorting to flooding, without worrying about or managing loops, and without needing to run spanning tree or a Trill.
3. Build a management plane on top of the control plane that automates and abstracts the operation of the fabric while presenting the simplest and best understood operational model to the administrator—that of a single switch.  This is not a network management application that enables a single pane of glass to manage multiple autonomous switches.  QFabric is in fact a single switch.

The result is a fabric that:

• Is blazingly fast.  In an internal test we found that QFabric can deliver sub five-microsecond latency across an entire data center.  And that is assuming maximum length interconnect cables which add more than one microsecond of that latency due to the speed of light.  With shorter cables we have measured latencies as low as 3.7 microseconds.  That QFabric is the world’s fastest fabric (over 500 ports) is a given.  What may be a surprise is that QFabric is faster than any chassis-based Ethernet switch ever built. And the fabric is consistently fast, delivering extremely low jitter under load.

• Scales in a linear fashion from tens to more than 6,000 10GbE ports. In the future, we also plan to deliver mega-fabrics that will scale to tens of thousands of ports and micro-fabrics that scale under 1000 ports.  It’s a fabric for data centers of all sizes.

• Delivers the ultimate in operational simplicity.  Because QFabric behaves as a single switch, it is possible to manage the entire fabric with a single network operator. 

• Delivers a quantum change in the total cost of ownership.  This is not the normal puffery of marketing; QFabric is more efficient because of its design.  Reducing the processing elements required to build the fabric not only makes it fast but reduces manufacturing costs.  Fewer hardware elements also require less power, cooling, and floor space.  Fewer hardware elements also mean greater reliability and lower support costs.  And finally, the inherent simplicity of the fabric will reduce management costs.  Hard to beat that.

Introducing the QFX3500

Today we also introduced the QFX3500, a remarkable new product that not only serves as the edge node of the QFabric, but can also function as a standalone, 48-port 10GbE top-of-rack data center access switch.  It too is blazingly fast, delivering under 780 nanoseconds of latency.  We recently teamed with IBM to run the STAC benchmark which measures application throughput and latency on a financial services workload; the result showed the fastest Ethernet performance ever recorded and only one microsecond slower than Infiniband. With the QFX3500, you can start by implementing a more traditional network topology, have it interoperate with any existing vendor’s infrastructure and upgrade to QFabric when you are ready.

What does this mean to the business?

While great technology is fascinating, at least to me, what does QFabric mean to the business?  It's quite simple:  QFabric will unleash the true power of the data center.  A fast fabric will enable every application to perform better, delivering a better user experience.  A flat, scalable, transparent fabric delivers a more elastic data center and ultimately greater business agility.  And the inherent simplicity of QFabric enables the entire data center to operate more efficiently and reliably.  Better experience and better economics.  What more could you ask for?