VMware: VLANs and vSwitches

Switches work at Layer2 and basically their job is pretty simple: they just forward Ethernet frames received from one port to another port based on the content of their MAC Address Table.

Let's start giving some basic concepts about Ethernet frames and VLANs.

An IP Packet is encapsulated at Layer2 (Data Link) in an Ethernet Frame.

By default Ethernet packets at Layer3 have a maximum Ethernet payload MTU of 1500bytes. This MTU depends on network protocol and for example is 1478 for PPPoA, 1446 for PPPoE, but it cannot grow over 1500bytes (if it is >= 1501bytes it is a Jumbo Packet).

As depicted in image above an Ethernet frame is composed by:

-Preamble
-SFD (Start of frame delimiter)
-Destination MAC Address
-Source MAC Address
-VLAN ID - IEEE 802.1Q tag (optional)
-Ethertype
-Payload
-FCS (Frame check sequence)

At Layer2, by default, an Ethernet frame is 1518bytes long. Preamble and SFD are not computed in frame size because they are used for synchronization and not for actual data transmission.

When dealing with VLANs an 802.1Q tag of 4bytes is added by the switch between Source MAC Address field and Ethertype field. With VLANs an Ethernet frame grows up to 1522bytes. By default switches drops packets that are greater than 1518bytes so when using VLANs switches must be prior configured.

Actually the condition above doesn't always apply becase for example Cisco switches supports by default Ethernet frames up to 1548bytes to accomodate encapsulation of an Ethernet packet into an ISL packet.

VLAN tagging is performed ONLY by switches and despite a VM itself could tag frames, in this article we consider tagging is ONLY performed by switches.

A switch adds a VLAN ID when it recieves a packet from a port belonging to a VLAN. VLAN tag is added in the correct frame position and, since frame has been modified, Frame check sequence (32bit CRC) has to be recalculated.

When a frame reach the switch to which reciever is directly attached it:

-Read VLAN tag
-Perform a lookup to check if Destination MAC Address from recieved frame is contained in switch MAC Address Table and to which port frame has to be forwarded.
-If step above is successful it recalculates Frame check sequence
-Forward frame to destination

Let's now see how VLANs and vSwitches work in vSphere:

We have two VMs: VM1 and VM2

First scenario: VM1 and VM2 are on the same VLAN, same vSwitch and same PortGroup VM Network 100

Using vswitchInfo.sh from virtuallyGhetto we can retrieve port ID to which VMs are connected to vSwitches in order to allow us to build a MAC Address Table.

Launching from ESXi host via SSH:

~ # ./vswitchInfo.sh -l

We get the following:

This allow us to deduce how the vSwitch MAC Addres Table would look like. We suppose that MAC Addres Table is completely/correctly built so every switch either virtual and physical knows to what interfaces forward packets.

vSwitch1
Vlan    Mac Address          Ports
----    -----------          -----
100    00:50:56:8a:7e:34      50331665
100    00:50:56:8a:25:f9      50331669

Since VM1 and VM2 are both on same PortGroup on same vSwitch when Ethernet frame is send from VM1 to VM2 vSwitch1 recieves frame, it performs a lookup in MAC Addres Table and it find that VM2 is directly attached to it so it forward frame to port ID 50331669 WITHOUT VLAN tag.

VM1 is able to communicate with VM2

Second scenario: VM1 and VM2 are on the same VLAN, same vSwitch but different PortGroup (VM Network 100 and VM Network 100-2)

We have this:

Same MAC Addres Table from first scenario only Port ID has changed for VM2 since it has been moved to another PortGroup.

vSwitch1
Vlan    Mac Address          Ports
----    -----------          -----
100    00:50:56:8a:7e:34      50331665
100    00:50:56:8a:25:f9      50331668

Since VM1 and VM2 are both on same switch vSwitch1 recieves frame, it performs a lookup in MAC Addres Table and it find that VM2 is directly attached to it so it forward frame to port 50331668 WITHOUT VLAN tag.

Third scenario: VM1 and VM2 are on the same VLAN but different vSwitch. Both VM Network 100 and VM Network VLAN 100 PortGroups have VLAN ID=100.

Please note that when VMs are on different vSwitches frames are always forwarded to physical switches.

We have this:

Two MAC Addres Table one for vSwitch1 and one for vSwitch0.

vSwitch1
Vlan    Mac Address          Ports
----    -----------          -----
100    00:50:56:8a:7e:34      50331665

vSwitch0
Vlan    Mac Address          Ports
----    -----------          -----
100    00:50:56:8a:25:f9      33554462

VM1 and VM2 are on different switches. vSwitch1 recieves Ethernet frame from VM1 addressed to VM2. Destination MAC Address is not contained in vSwitch1 MAC Addres Table so vSwitch1:

-Add VLAN tag specifying that frame destined to VM2 belongs to VLAN 100
-Recalculate FCS
-Forward frame to physical switch

Physical switch:

-Recieves frame from vSwitch1
-Forward frame to vSwitch0

Of course even physical switch uses a MAC Addres Table for frames forwarding.
Usually intra switch links are trunks. This means that a single port/cable can carry frames from multiple VLANs.

vSwitch0:

-Recieves frame
-Read VLAN tag
-Perform a lookup to check if VM2 Destination MAC Address (00:50:56:8a:25:f9) is contained in its MAC Address Tableand is connected to port 33554462
-Since previous step is successful it recalculates Frame check sequence
-Forward frame to VM2 via port 33554462


VM1 is able to communicate with VM2

Fourth scenario: VM1 and VM2 are on different VLAN, same vSwitch, different PortGroup (VM Network 100 and VM Network 200)

We have this:

vSwitch1
Vlan    Mac Address          Ports
----    -----------          -----
100    00:50:56:8a:7e:34      50331665
200    00:50:56:8a:25:f9      50331671

VM1 and VM2 are on different VLANs, vSwitch1 recieves frame lookup in MAC Addres Table and find out that VM2 is on a different VLAN than VM1. vSwitch1 drops the frame.

VM1 is NOT able to communicate with VM2

Fifth scenario: VM1 and VM2 are on different VLAN and different vSwitch

We have this:

vSwitch1
Vlan    Mac Address          Ports
----    -----------          -----
100    00:50:56:8a:7e:34      50331665

vSwitch0
Vlan    Mac Address          Ports
----    -----------          -----
200    00:50:56:8a:25:f9      33554462

As in third scenario VM1 and VM2 are on different switches so frames will be forwarded to physical switch. vSwitch1 recieves Ethernet frame from VM1 addressed to VM2.

Destination MAC Address is not contained in vSwitch1 MAC Addres Table so vSwitch1:

-Add VLAN tag specifiyng that frame destined to VM2 belongs to VLAN 100
-Recalculate FCS
-Forward frame to physical switch

Physical switch:

-Recieves frame from vSwitch1
-Forward frame to vSwitch0

vSwitch0:

-Recieves frame
-Read VLAN tag
-Perform a lookup to check if VM2 Destination MAC Address (00:50:56:8a:25:f9) is in its MAC Address Table
-Since VM2 is on a different VLAN vSwitch1 drops the frame

VM1 is NOT able to communicate with VM2

So, it's important to understand that in vSphere when a VM sends packets/frames to another VM on a different vSwitch those packets/frames will always hit physical switch.
Another major concept to remember is that VLAN tagging occurs only when frames leave vSwitch. Intra-vSwitch communications are not VLAN tagged.

That's all!!