LAN Physical to Overlay

It is terribly difficult to express the design elements of VxLAN technology in a single drawing.  As seen in previous blogs, the elements of the physical construct of an ECMP (Equal-Cost Multi-Path) spine and leaf is astonishingly complex.  These become represented in abstracted views of the total in graduations from quite simple to piece-by-piece elements in order to express their true nature.

Part of the problem is associated with these complexities, but having put together multiple descriptive models, it really is because the OVERLAY doesn't look anything like the physical design.  Networkers got used to broadcast domains being associated with a wire.  Then being associated with an L2 abstraction.  Now networkers need to figure out how to represent the multiple levels of networking with additional L2-in-L3 tunnels.

It's also not as if this hasn't happened before.  VPN drawings look much like VxLAN drawings, but in a VPN drawing, there may be one or a couple, in VxLAN there could be a lot more.  Ultimately, the issue becomes how complex the drawing is to layer in all of the information.

Starting from the view of a Modern Platform for Enterprise (as opposed to public cloud), networkers need to connect particular security elements, network services, control plane services, backup, networking and platforms.  Architecturally, the model looks very much like this.

Figure 1.  Modern Platform

DC LAN (red) plays a dominant role in this platform architecture, providing connectivity from any element to any element within the construct.  Be it a physical device, a virtual machine on a hypervisor or a container on an operating system.

Figure 2.  Any to Any

It's not as simple as that though.  The network attributes are overlaid on top of hardware, software and logical (or abstracted) networking mechanisms in this model.  In this extremely simplified model, use case 1 is where the hypervisor communicates with another hypervisor (green) and use case 2 where the hypervisor communicates with a bare metal object.  A third use case exists where a container communicates with hypervisor or hardware, but it's not used frequently yet in the Enterprise.

The VxLAN (Virtual eXtensible LAN) kernel module of the hypervisor (in this case VMware ESX) communicates through it's vSwitch to the physical medium, a network adapter on the physical machine.  This is then passed to the physical switch to be passed in accordance to the VNI (Virtual Network Identifier) associated with the packets.  In case 1, it is then picked up by the network card on the second physical host, working though the vSwitch to get to the virtual machine.  In case 2, the endpoint is a VTEP (Virtual Tunnel EndPoint) where it is de-encapsulated from L3 to L2 to arrive at the physical server.

In both cases, two sets of interacting Control Planes, VMware NSX and Arista CVX provide the path information to instantiate the VxLAN tunnel.

Figure 3.  VxLAN Delivery (gratuitous re-use from earlier blog)

Now, there needs to be a bold red blinking sign that states the network still exists, both in its physical and logical form.  Herein lies the Spine-and-Leaf.  It is similar to a Clos design (as in Charles Clos) that does have some oversubscription at different levels.  Physically it utilizes the spine as a one hop transport route between all leaf nodes.

The concept though is quite simple.  Scale out horizontally as large as possible.  When that is exceeded, add another spine and make a 2 hop transport route.  The Leaf nodes come in two basic flavors, one that provides transport access to hosts, the other leaf flavor provides Services.  Services Leaf are used to establish specific service functions.  The two examples shown here are WAN access with perimeter security Firewall and IP protocol based storage.

Moving from Brown to Green in this model should be relatively easy.  Once the Spine and Leaf is established, legacy networks may be connected at a Leaf node to provide access utilizing the switch VTEP.  It's an extra hop, but the latency in this network type is extremely low.

Figure 4.  Spine and Leaf (high level)

Combining Figure 3 and Figure 4 is terribly problematic.  It may not even be useful at large scale due to the enormous details necessary to show it in its entirety.  

What networkers can do is abstract the drawing a bit.  Pull it up from the physical layer.  The view in Figure 5 is just such a drawing.

Utilizing Figure 4 as the groundwork (the physical) and "pulling up" the abstractions and overlays that reside on it.  

In this scenario, the spine and each leaf are BGP Autonomous Systems.  The first layer above the physical equipment is an IP network routed solely within the confines of the spine and leaf (Blue routers).  Additionally, a VRF is run on the same structure to propagate a management network (Red routers) to all switches.  It also acts as the distribution layer for any management switches deployed within the Leaf cabinets.

The VNI framework is then "pulled up" to the final level.  This is the tunnel path.  

Below the physical switch VTEP, an OVSDB (OpenVSwitch DataBase) is used to manage the interactions of all platform systems that are virtualized with, in this case, VMware NSX.

At and above the physical switch are the VTEP, managed by another OVSDB for all physical element connections not associated with VMware NSX.  The control plane in this model is Arista CVX.

As in Diagram 3, the OVSDB (VMware) communicates with the OVSDB (Arista) to manage the entirety of the tunnel formation.

Figure 5.  An high level view of the network from physical to overlay

In light of creating a view of the network that expresses as much as possible with the least amount of complexity, this drawing has proven to be quite good at showing as much as possible without making it impossible to follow.

If you use this model to describe your network, please do let me know how it turned out.

I'm interested in any other way you may have to show this type of information graphically.  Please tweet them to @abusedbits or link them to my Google+.

SDN and NFV and WAN Modernization

The role of network infrastructure is seeing a rapid change from the historical perspective of component by component design to one where those capabilities are created in mixed modes of pure hardware (let's call this the classical model), hardware and software (let's call this hybrid networking) and software alone (defined as Network Function Virtualization or NFV).

Each of these can be programatically defined with increasing software definition as the capabilities move toward NFV.  So, one may have both a NFV instance or instances of functionality AND software defined functionality.

SDN and NFV vs Classical WAN

Using the picture above, Classical site level networking introduces discreet hardware devices to provide specific service capabilities in at the location.  These could and do typically include routers, switches, firewalls, load balancers, wan optimization devices and wireless controllers.

These also have a very low degree of unified Software Definition, but will typically have high performance.

In the case of the Software Defined WAN (Cloud WAN, SD-WAN, etc), those functions typically assigned to discreet hardware at the site are virtualized into their Network Function, yes, Network Function Virtualization.  Integration of these virtual services with a management and control plane provides a key element in addressing Software Defined WAN.  Each element is intended to operate in unison in a pre-determined way to provide the site level WAN Service as well as other services for the site.

This would have a much higher degree of Software Definition.

The limitations of this method may include sufficient processing power (CPU) and processing memory.  Where, as an example, an x86 server is used to provide the host for virtualization, there may also be a critical limit to the capabilities of the box relative to performance.  Where network hardware typically uses well defined processes, in ASICs to amplify the performance, much of the work done within a virtualization system is accomplished in software.

This means that there will be a performance difference, escalating as the number and type of NFV services that are applied to the same server host.  Expect more dramatic drops in performance when services like firewall rules increase significantly as this may cause more software lookups and therefore slower packets per second.

This is not static though.  Performance of NFV services will continue to increase and become optimized in software.  Performance of x86 hardware is absolutely guaranteed to continue to increase in performance and applicable memory per CPU.  Network card vendors are building addressable hardware functions within the network interface.  Parts of the NFV services may be broken out to optimize performance.

The recommendation, look at the specs and decide what you want/need to do.  Where necessary, break out to a hybrid network solution to solve today's problems and look to the future where these functions integrate even tighter with better performance.

Software Defined LAN for the DC

In the discussion of the Enterprise Cloud, the DC LAN becomes a topic for consideration.  The network need of a modern platform is to create connectivity from any device to any device in the DC LAN.

Not discounting the East-West requirements in bandwidth (serviced by Spine and Leaf) and the possibility of delivering storage over an IP protocol, it is an imperative for any IDC 3rd Platform mechanism to be software defined.  This is, in no small part, to accommodate the new rate of change associated with 3rd Platform application development included over the entirety of the application life-cycle.  Manual ticket processing should not be allowed to stand in the way of the life-cycle process.

It is important to remember that, in Enterprise Cloud, not all of the applications deployed there will be 3rd Platform.  There's also a high probability that there will not only be 2nd Platform applications living within the same infrastructure, but there is also the possibility of 1st Platform connectivity requirements.  It's the old adage in IT, the pendulum swings but some of this stuff just doesn't go away.

So, there also needs to be a means to connect to legacy or classical systems.  The combination is further complicated by a need, that is turning into a requirement, to isolate communication between individual applications to reduce the possible threat perimeter.

The connectivity managed between all of these systems must be API driven.

Consider this example architectural pattern:

SDN Any to any network architecture pattern

In this example, the individual hypervisors are physically connected to the network, but rely upon a point-to-point tunnel to communicate with other hosts.  In this example, it's based on VxLAN, but in reality could be any tunneling mechanism.

The virtual switch VTEP in the hypervisor is managed from a control plane, in this case, VMware NSX, that maintains Open VSwitch DataBase (OVSDB) information about its hosts under management.

The physical switch level VTEPs are managed by Arista Cloud Vision CVX, which maintains principal authority over the OVSDB and inner-connectivity of the Virtual Network Interfaces (VNI).  It also creates the means to connect to bare-metal systems (far right) utilizing the VTEP in the switch.

VMware NSX acts as a substituent to the Arista Cloud Vision CVX such that one state mapping is managed across the entirety of the network in the OVSDB.

The result:

• The network is programatically defined
• There is software definition of both the physical and logical network endpoints
• It is managed through the interaction of API
• Provides isolation between application communications

Network based APM

We've all been hearing quite a lot about Big Data and what the possibilities are associated with it.  Ultimately, the discussion comes down to, how to get data and how is it made relevant.

Software Defined Networking is making it possible to separate the control plane from the data plane.  The control plane information really isn't all that interesting, unless you're debugging an issue.  But the data plane quite literally contains all of the relevant data associated not only with the application, but how it operates.  

Assuming that the applications relevant data is also stored somewhere, Big Data would be able to produce interesting information from the stored data.  But what about the operation of the application?  User experience?  SLA?

The scenario for elevating the value of the more transient aspects of network communications related specifically to an application have been around in networking for quite some time.  At the most base level, packet counters exist on a host-by-host reading from within the connecting switch.  Sflow and Netflow are more recent additions to look at the communications on the network.  For about as long, and significantly more granular and expensive, SPAN or tap to a more granular and extensive analytics engine produce Application Performance information.

Consider this, with the addition of container based technologies, it could be possible to create a standardized mechanism for collection of developer based application awareness from within software defined networks.

Similarly, with sufficiently advanced and capable CPUs, it may also be possible to run the analytics from within the switching platform.  We can table this for now, but the possibility is there.

The model could utilize a particular aspect of the developer's framework.  Java or .Net that, from within the application, makes a call to a switch to start an analytics process. 

With the application making calls specifically to the network infrastructure, it could possibly request the switch spin up a collector in a container on the switch.  It could also make a call to the switch to determine which VNI is being used to transport it's data and tell the collector to extract the associated TCP information or deeper packet information from the application transport.

Once you have TCP or deeper packet data from the application, analytics could be applied to develop specific awareness of the application that could, potentially, become the basis for application performance.  With the ensuing granularity of deeper inspection, both User Experience and SLA could potentially be derived from the mechanism.  Not to mention, the possibility of extremely specific dev based tests on the application to determine things like fitness.

APM in an SDN network

Assuming the network was already established, in the simplest of these methods, the application would:

• make an API call to the switch to determine which VNI it is associated with
• make an API call to the Switch to spin up a collector
• issue a TEST case for the collector to capture from within that VNI
• send the captured data to an analytics engine to process for value

In more general testing, and assuming applicable path information, bandwidth and CPU capacity, the application would make a similar request to the control plane to issue the test case against its entire path.  It could create capture on deeper packet information.  Perform continuous Application Performance Management.  Ultimately leading to extensive SLA level information about the application.