Three problems with the Nimbus testbed on cglc

We met three problems when trying to make Nimbus work on cglc.

1. The ethernet interface id on cglc5 starts from eth1 in stead of eth0. So the default network script setting of Xen will cause some strange effects on the network layout: no "peth"s, xenbr0 having no interfaces attached, etc. The solution is replace "(network-script network-bridge)" with "(network-script 'network-bridge netdev=eth1')" in the file /etc/xen/xend-config.sxp. cglc6 to cglc8 also have the same ethernet interface id sequences, and thus facing the same problem as cglc5. If any networking problems come up in the future on these machines, we should remember checking this configuration.

2. Ebtables of version numbers later than 2.0.6 have some bugs with Xen x86_64. Since ebtables are used by Nimbus, version numbers such as 2.0.8 or 2.0.9 will cause some exceptions with the following messages when new Nimbus workspaces are created:

"The kernel doesn't support a certain ebtables extension, consider recompiling your kernel or insmod the extension."

Accompanied by this dmesg error: "kernel msg: ebtables bug: please report to author: entries_size too small"

The solution is to build ebtables 2.0.6 on VMMs in Nimbus. A Nimbus user, Matt, has solved this before, and provided a rpm to the Nimbus site:

http://workspace.globus.org/downloads/ebtables-2.0.6-3.rf-mm-el5-x86_64.rpm
http://workspace.globus.org/downloads/ebtables-2.0.6-3.rf-mm-el5.src.rpm

sha1sums:
45982dacfaddfe8f828a720c94d7435a65a4bbc2 ebtables-2.0.6-3.rf-mm-el5-x86_64.rpm
df61bb2a43faa4fdc8e592c04b65242c71db790f ebtables-2.0.6-3.rf-mm-el5.src.rpm

And if you really want to build it manually, follow the steps bellow (this instruction is provided by Luna from the Nimbus user mail-group):
1) Down load the source codes of ebtables 2.0.6 from https://launchpad.net/ubuntu/dapper/+source/ebtables/2.0.6-3ubuntu2/+files/ebtables_2.0.6.orig.tar.gz;
2) Untar the source codes;
3) Down load the patches on http://svn.exactcode.de/t2/trunk/package/security/ebtables/, and apply them to the source codes;
4) Back up your previous ebtable installation (this is optional, just in case if you might need to use it in the future);
5) Do make and make install in the source directory of ebtables 2.0.6.

3. The default network association in worksp.conf provided by the Nimbus site is "association_0: private; xenbr0; vif0.0 ; none; 192.168.0.0/24", but this caused an exception on our testbed when I tried to deploy a new workspace: Nimbus is looking for an association named "public" instead of "private" when trying to configure the new virtual machine. So the solution is to replace "private" with "public" in this line. I don't know if this is general problem with the document on the Nimbus site http://workspace.globus.org/vm/TP2.2/admin/quickstart.html, but it works with our testbed. Maybe it's just because of some specific configuration on clgc5. Of course, you can always check the logs under /opt/workspace/logs to investigate the problems with workspace creation and termination.