VBS Architecture

The job of VBS is done on two ends: the volume server, the Xen manager, and through the interactions between the two. So our architecture is composed of three components: the VBS web service, the volume server delegate web service, and the VMM delegate web service.

The VBS web service is the front service that answers the requests from VBS clients, and coordinates the operations on the volume server and Xen VMMs. It provides the following method interfaces to the VBS client:
- create-volume
- delete-volume
- describe-volumes
- describe-volume
- create-snapshot
- delete-snapshot
- describe-snapshots
- describe-snapshot
- attach-volume
- detach-volume

The volume sever delegate web service is the service sitting on the logical volume server and encapsulating the LVM and ISCSI command lines, which are necessary to complete VBS operations. It is called by the VBS web service, and completes the operations of creating/deleting volumes/snapshots, exporting/deleting ISCSI targets by executing LVM and ISCSI commands.

The VMM delegate web service is the service sitting on VMM nodes and encapsulating VMM and ISCSI command lines. It is called by the VBS web service, and completes the operations of discovering and importing ISCSI targets, and attaching and detaching the ISCSI device to/from Xen DomU VMs.

The work flow of VBS is controlled by the VBS web service to complete the functions. A typical work flow for the attach-volume operation is composed of the following invocations carried out by the VBS web service:
(1) call the volume server delegate service to export an ISCSI target for ;
(2) call the VMM delegate service on to discover the new ISCSI target, and log in to it. This will result in a new ISCSI device created on the VMM node;
(3) call the VMM delegate service to attach the corresponding ISCSI device to , which will be visible as inside the VM;
(4) update the metadata about the , so that consequent calls to describe-volume(s) about this volume will return consistent information.

Manual scripts for performing the functions of VBS

We'll have a series of documents explaining the design and implementation of the Virtual Block Store (VBS) system.

This post presents the basic manual steps to complete the VBS functions: create a new logical volume, attach it to some Xen DomU VM, detach it from the VM, create a snapshot of it, delete the snapshot, and delete the volume. The manual scripts are based on existing softwares, including LVM (Logical Volumen Manager), ISCSI, and Xen.

Test steps for creating, attaching and detaching volumes:
1) create a new volume on the volume server:
lvcreate -n gxmVol01 -L32M /dev/vg
2) export a new iscsi target:on the volume server
ietadm --op new --tid=2 --params Name=iqn.2009-03.edu.iu.cgl:cglch.lun0
ietadm --op new --tid=2 --lun=0 --params Path=/dev/vg/gxmVol01
3) discover and login the new iscsi target from a Xen dom0:
iscsiadm --mode discovery --type sendtargets --portal 192.168.1.254:3260
iscsiadm --mode node --targetname iqn.2009-03.edu.iu.cgl:cglch.lun0 --portal 192.168.1.254:3260 --login
After log-in, check the disk directory for the newly created ISCSI device: ls /dev/disk/by-path
4) do block-attach in Xen dom0 to attach the iscsi block device to a domU:
xm block-attach ttylinux phy:/dev/disk/by-path/ip-192.168.1.254:3260-iscsi-iqn.2009-03.edu.iu.cgl:cglch.lun0-lun-0 /dev/sdb w
5) to detach a volume from a domU, do xm block-detach in dom0:
xm block-detach ttylinux /dev/sdb
Note: don't do "xm block-attach" while the block device is mounted on the corresponding domU VM. If there is no xenconsole connected to the VM when the block-detach happens, you will get an error the next time you try to connect to the VM through xenconsole, and it's probable that you'll never be able to login to the VM again by xenconsole. So always remember to do umount before block-detach.
6) logout from the iscsi target in dom0:
iscsiadm --mode node --targetname iqn.2009-03.edu.iu.cgl:cglch.lun0 --portal 192.168.1.254:3260 --logout
7) delete the iscsi target from the volume server:
ietadm --op delete --tid=2
8) create a snapshot of a volume:
lvcreate --size 32m --snapshot --name gxmSnap01 /dev/vg/gxmVol1
9) delete the volume or snapshot:
lvremove /dev/vg/gxmVol1