Changing NIC order in XenServer

Changing NIC order in XenServer

I ran in to an issue when attempting to join a freshly reinstalled and fully patched XenServer 6.5 host back to a pool. This host was one of four Cisco blades in a chassis which had the exact same hardware configuration as the other blades.  Every time I’d try to add the blade to the pool, the network configuration (which included bonded NICs) would go awry.

I was able to track own the problem after much investigation.  The original pool master had  a NIC ordering which had been changed from the default in the XenServer configuration.  This meant that any subsequent pool nodes had to have their NICs changed to the same scheme.  After some head scratching, Googling, and sacrificing a RAM stick or two to the Gods of computing, I found the solution!

Start off by going to the console on the pool master.  Run the following command:

interface-rename --list

This will result in something akin to the following:

Name MAC               PCI          ethN Phys SMBios  Driver Version      Firmware
eth0 00:30:48:cf:7a:c2 0000:02:00.0 eth0 em2          e1000e 3.1.0.2-NAPI 1.9-0 
eth1 00:30:48:cf:7a:c1 0000:01:00.0 eth1 em1          e1000e 3.1.0.2-NAPI 1.9-0 
eth2 00:30:48:cf:7a:c4 0000:04:00.0 eth2 em4          e1000e 3.1.0.2-NAPI 1.9-0 
eth3 00:30:48:cf:7a:c3 0000:03:00.0 eth3 em3          e1000e 3.1.0.2-NAPI 1.9-0

You’ll notice under the Phys heading, the NICs are out of order, and the PCI ordering matches the Phys order.

Time to list the interfaces on the new node we are adding to the pool:

interface-rename --list

This results in something akin to:

Name MAC               PCI          ethN Phys SMBios  Driver Version      Firmware
eth0 00:30:48:cf:7b:c1 0000:01:00.0 eth0 em1          e1000e 3.1.0.2-NAPI 1.9-0 
eth1 00:30:48:cf:7b:c2 0000:02:00.0 eth1 em2          e1000e 3.1.0.2-NAPI 1.9-0 
eth2 00:30:48:cf:7b:c3 0000:03:00.0 eth2 em3          e1000e 3.1.0.2-NAPI 1.9-0 
eth3 00:30:48:cf:7b:c4 0000:04:00.0 eth3 em4          e1000e 3.1.0.2-NAPI 1.9-0

You’ll  notice how the adapters have a sequential ordering to them.  How do we change this to match the pool master?  Easy!

On the new node, make sure you take down the interfaces that you are changing:

ifconfig ethN down

Where ethN is the name of the interface you are taking down,  in our case it would be:

ifconfig eth0 down
ifconfig eth1 down
ifconfig eth2 down
ifconfig eth3 down

Now we get to the fun and nailbiting part, changing the ordering of the NICs.  It is surprisingly simple.  Make a note of the Name/ethN and Phys mappings that you want from the table.  This should match the output from the pool master node.  So for us, we want:

eth0 = em2
eth1 = em1
eth2 = em4
eth3 = em3

Next, run the interface-rename command on the new node:

interface-rename --update eth0=em2 eth1=em1 eth2=em4 eth3=em3

You can use the MAC, PCI, Phys, or SMBios as identifiers.  In this case we used the Phys as the identifiers for the renaming.

Now reboot, and when it comes back up you should find that the new node has the same NIC ordering as the pool master and is ready to be added.  You can verify by performing an

interface-rename --list

Which, for us, should result in something akin to:

Name MAC               PCI          ethN Phys SMBios  Driver Version      Firmware
eth0 00:30:48:cf:7b:c2 0000:02:00.0 eth0 em2          e1000e 3.1.0.2-NAPI 1.9-0 
eth1 00:30:48:cf:7b:c1 0000:01:00.0 eth1 em1          e1000e 3.1.0.2-NAPI 1.9-0 
eth2 00:30:48:cf:7b:c4 0000:04:00.0 eth2 em4          e1000e 3.1.0.2-NAPI 1.9-0 
eth3 00:30:48:cf:7b:c3 0000:03:00.0 eth3 em3          e1000e 3.1.0.2-NAPI 1.9-0

I was then able to add the new node to the pool without any issues!

Leave a Reply