VMware: manage networking with esxcfg

In this post I will explain how to manage vNetworking using esxcfg.

For explanation purpouses let's simulate a scenario. Pretend we need a second management network for redundancy's sake (network heartbeating for HA). For this management network we will use a dedicated network adapter.

Here's what we want to achieve:

These are our steps:

-Create an additional vSwitch
-Assign a network adapter to this vSwitch
-Create a new vmkernel
-Enable Jumbo Frame on vmkernel
-Use this vmkernel for Management Traffic

Let's start by connecting to the host via SSH (I use Putty) and listing all network adapters attached to host:

~ # esxcfg-nics -l

List all vSwitches with corresponding network adapters

~ # esxcfg-vswitch -l

As you can see in this scenario I only have one vSwitch named vSwitch0. It would be advised to add the new vmkernel for management to vSwitch0 itselfs but for the sake of this post I will create another vSwitch and then add the management vmkernel to that vSwitch.

Please bear in mind that by creating a new vSwitch you introduce a new element in your environment so when you are in "troubleshooting mode" the less items you have to check the faster you are in identifying trouble causes. A thumb rule is to use as less vSwitches as possible.

Command above told us vmnic3 it's the only available network adapter so we will use that in our new vSwitch.

If there are no vmnics available it is also possible to remove some from vSwitch0.

For example to remove vmnic2 from vSwitch0

~ # esxcfg-vswitch -U vmnic2 vSwitch0

Now let's first create a vSwitch named vSwitch1

~ # esxcfg-vswitch -a vSwitch1

Then we add network adapter vmnic3 to vSwitch1

~ # esxcfg-vswitch -L vmnic3 vSwitch1

Create a Management Network 2 vmkernel in vSwitch1

~ # esxcfg-vswitch -A "Management Network 2" vSwitch1

Assign it an IP/netmask

~ # esxcfg-vmknic -a -i 10.0.1.100 -n 255.255.255.0 "Management Network 2"

Set VLAN ID 300 for Management Network 2 vmkernel

~ # esxcfg-vswitch -v 300 -p "Management Network 2"

Let's pretend we need to use Jumbo Frames. For your interest it's pretty pointless enabling Jumbo Frames in a management only vmkernel since benefits of transmitting Ethernet packets with a bigger payload are primarily pointed out when there is a heavy network activity. Jumbo frames could be useful on vMotion or iSCSI/NFS vmkernels. Anyway to enable Jumbo Frames we need to edit the Maximum Transmission Unit setting it to 9000bytes.

You also need to change MTU on you physical switch in order to use Jumbo Frames. If your vSwitch sends packets with an MTU of 9000bytes and your pSwitch expects them of 1500bytes (default Ethernet packet MTU) packets could be either fragmented or dropped. Here's a good article by Cisco!

~ # esxcfg-vmknic -m 9000 -p "Management Network 2"

We created a "general purpouse" vmkernel and since we need to use this for Management Traffic we need to specify it.

First we retrieve informations on newly created vmkernel:

~ # esxcfg-vmknic -l

As you can see in picture above our vmkernel is vmk3.

William Lam in this VMware Community post explains how to bind specific traffic to a vmkernel.

As for management traffic the following command is used:

~ # vim-cmd hostsvc/advopt/update Net.ManagementIface string vmk3 

To briefly recap we used esxcfg to manage our vSphere networking. We introduced a fictional scenario to point out what commands need to be used in order to edit/create/manage networking in ESXi hosts.

That's all!!