I ran in to an issue when attempting to join a freshly reinstalled and fully patched XenServer 6.5 host back to a pool. This host was one of four Cisco blades in a chassis which had the exact same hardware configuration as the other blades. Every time I’d try to add the blade to the pool, the network configuration (which included bonded NICs) would go awry.
I was able to track own the problem after much investigation. The original pool master had a NIC ordering which had been changed from the default in the XenServer configuration. This meant that any subsequent pool nodes had to have their NICs changed to the same scheme. After some head scratching, Googling, and sacrificing a RAM stick or two to the Gods of computing, I found the solution!
Start off by going to the console on the pool master. Run the following command:
This will result in something akin to the following:
Name MAC PCI ethN Phys SMBios Driver Version Firmware eth0 00:30:48:cf:7a:c2 0000:02:00.0 eth0 em2 e1000e 22.214.171.124-NAPI 1.9-0 eth1 00:30:48:cf:7a:c1 0000:01:00.0 eth1 em1 e1000e 126.96.36.199-NAPI 1.9-0 eth2 00:30:48:cf:7a:c4 0000:04:00.0 eth2 em4 e1000e 188.8.131.52-NAPI 1.9-0 eth3 00:30:48:cf:7a:c3 0000:03:00.0 eth3 em3 e1000e 184.108.40.206-NAPI 1.9-0
You’ll notice under the Phys heading, the NICs are out of order, and the PCI ordering matches the Phys order.
Time to list the interfaces on the new node we are adding to the pool:
This results in something akin to:
Name MAC PCI ethN Phys SMBios Driver Version Firmware eth0 00:30:48:cf:7b:c1 0000:01:00.0 eth0 em1 e1000e 220.127.116.11-NAPI 1.9-0 eth1 00:30:48:cf:7b:c2 0000:02:00.0 eth1 em2 e1000e 18.104.22.168-NAPI 1.9-0 eth2 00:30:48:cf:7b:c3 0000:03:00.0 eth2 em3 e1000e 22.214.171.124-NAPI 1.9-0 eth3 00:30:48:cf:7b:c4 0000:04:00.0 eth3 em4 e1000e 126.96.36.199-NAPI 1.9-0
You’ll notice how the adapters have a sequential ordering to them. How do we change this to match the pool master? Easy!
On the new node, make sure you take down the interfaces that you are changing:
ifconfig ethN down
Where ethN is the name of the interface you are taking down, in our case it would be:
ifconfig eth0 down ifconfig eth1 down ifconfig eth2 down ifconfig eth3 down
Now we get to the fun and nailbiting part, changing the ordering of the NICs. It is surprisingly simple. Make a note of the Name/ethN and Phys mappings that you want from the table. This should match the output from the pool master node. So for us, we want:
eth0 = em2 eth1 = em1 eth2 = em4 eth3 = em3
Next, run the interface-rename command on the new node:
interface-rename --update eth0=em2 eth1=em1 eth2=em4 eth3=em3
You can use the MAC, PCI, Phys, or SMBios as identifiers. In this case we used the Phys as the identifiers for the renaming.
Now reboot, and when it comes back up you should find that the new node has the same NIC ordering as the pool master and is ready to be added. You can verify by performing an
Which, for us, should result in something akin to:
Name MAC PCI ethN Phys SMBios Driver Version Firmware eth0 00:30:48:cf:7b:c2 0000:02:00.0 eth0 em2 e1000e 188.8.131.52-NAPI 1.9-0 eth1 00:30:48:cf:7b:c1 0000:01:00.0 eth1 em1 e1000e 184.108.40.206-NAPI 1.9-0 eth2 00:30:48:cf:7b:c4 0000:04:00.0 eth2 em4 e1000e 220.127.116.11-NAPI 1.9-0 eth3 00:30:48:cf:7b:c3 0000:03:00.0 eth3 em3 e1000e 18.104.22.168-NAPI 1.9-0
I was then able to add the new node to the pool without any issues!