Thursday, July 22, 2010

No network on Xen user domain

Today I installed Xen 3.2 for my work and created a guest VM using the this link as a guide. However I found that the network on my VM didn't work. The problem as Pierre explained was that the link between my physical network device and the virtual TAP device was not set up.

This was in spite of the fact that I re-started the xend daemon on my machine and used the static IP address method that was recommended in the blog mentioned above.So it seems that the xm create and xend scripts are not doing what they are supposed to do!

David Wolinsky came up with the following fix:
On the console of your control domain, do the following:
$ brctl addif tmpbridge eth0
$ ifup tmpbridge
$ ifconfig eth0 0.0.0.0 promisc
$ brctl show
$ dhclient tmpbridge

Now when you do an ifconfig on you domain-0, the tmpbridge interface should show up with the IP that was originally assigned to eth0.

And since eth0 is set to "promiscuous" mode, it will accept all network packages coming to it.

Now using the route command find the default gateway used by your domain-0. The following terminal snapshot is provided to clarify which IP I am talking about:

user@Domain0:~$ route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.5.144.0 * 255.255.240.0 U 0 0 0 tmpbridge
default 10.5.144.1 0.0.0.0 UG 0 0 0 tmpbridge


In the last line, the default gateway is highlighted in red. Make a note of this IP.
Now open the console of your user domain and check the route:

root@UserDomain:~# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.5.144.0 * 255.255.240.0 U 0 0 0 eth0

As you can see, in my case the default gateway was not set. To do so use the route add default gw command and the IP we obtained from the default gateway of domain-0.

root@UserDomain:~# route add default gw 10.5.144.1

Verify using route, and then try pinning something:

root@UserDomain:~# route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.5.144.0 * 255.255.240.0 U 0 0 0 eth0
default 10.5.144.1 0.0.0.0 UG 0 0 0 eth0
root@UserDomain:~# ping google.com
PING google.com (72.14.253.104) 56(84) bytes of data.
64 bytes from mia04s03-in-f104.1e100.net (72.14.253.104): icmp_seq=1 ttl=56 time=15.5 ms
64 bytes from mia04s03-in-f104.1e100.net (72.14.253.104): icmp_seq=2 ttl=56 time=14.0 ms
64 bytes from mia04s03-in-f104.1e100.net (72.14.253.104): icmp_seq=3 ttl=56 time=13.9 ms



Dadaaa! Many thanks to Pierre and David!
As a side note, if you are adding more domains, you do not need to repeat all the commands on domain-0, as its interface is already in the "promiscuous" mode. Only run the dhclient again. You will however need to set up the default gateway in every new domain that you create.