Swappable root disks with Oracle VM Server for Sparc (LDOMs
Posted by Steve Putre on Fri, Apr 08, 2011 @ 01:36 PM
One of the many advantages of virtualization is the ability to take arbitrary backend storage and make it look like whatever you need it to for the guest domain. How you do this is unique to every implementation and brand of virtualization.
Oracle VM Server for sparc (formerly known as LDOMs (Logical Domains) before the marketers got hold of it) is a type-II hypervisor which is integrated into oracle's T-series line of servers. It consists of a Control Domain which by default owns all of the server's physical resources and can allocate them to a number of guest domains running on the physical hardware.
Here are some hands-on steps for provisioning the root disk of an LDOM in such a way that the disk can be snapshot, cloned, and re-provisioned to the same or to other guest LDOMs on the server.
Uses:
- Backup LDOM before patching or upgrading
- Quick provisioning of new LDOMs off a standard or 'golden' image
- Return LDOM to a known good state
Caveats:
- The only way to get a consistent image of the LDOMs operating system is to shutdown the LDOM before taking the snapshot. A snapshot of a running LDOM is unlikely to be stable.
Starting assumptions:
a. The root disk storage for the guests is implemented as a ZFS zpool (which we will call 'guestpool' which sits on top of arbitrary physical storage.
b. The virtual disk service and all other services necessary to run LDOMs are also already set up. I am focusing only on the storage setup.
c. This is a new LDOM named 'guest1' which is not yet bound or running.
The first step is to provision a root volume for the guest.
control_domain # zfs create -V 10g guestpool/sys1
control_domain # ldm add-vdsdev /dev/zvol/dsk/guestpool/sys1 sys1@primary-vds0
control_domain # ldm add-vdisk sys1 sys1@primary-vds0 guest1
The ZFS filesystem must be a 'zvol'; a ZFS filesystem which exports a block device interface.
We now have a 10Gb root disk on which to build. let's also provision a swap disk, which can be shared among all the various system disks:
control_domain # zfs create -V 10g guestpool/swap
control_domain # ldm add-vdsdev /dev/zvol/dsk/guestpool/swap swap@primary-vds0
control_domain # ldm add-vdisk swap swap@primary-vds0 guest1
Check that the bindings are correct:
control_domain # ldm list-bindings guest1
...
...
...
DISK
NAME VOLUME TOUT ID DEVICE SERVER MPGROUP
swap swap@primary-vds0 1 disk@1 primary
sys1 sys1@primary-vds0 0 disk@0 primary
...
...
...
Note that the 'DEVICE' column dictates what target ID the LDOM sees. The NAME and VOLUME are local to the control domain, and are not seen by the LDOM. This mapping enables us to assign any backend to be 'disk@0' in the LDOM, and to become the boot disk.
Next, start the LDOM and perform the initial build
control_domain # ldm start guest1
control_domain # telnet localhost....
...
...
...
After the LDOM is built and you are satisfied with the build, shutdown the LDOM and stop it:
...
...
control_domain # ldm stop guest1
Take a snapshot of the volume and clone it:
control_domain # zfs snapshot guestpool/sys1@basebuild
control_domain # zfs clone guestpool/sys1@basebuild guestpool/sys2
Note that once you have cloned from a snapshot, the snapshot cannot be removed. The snapshot is the parent of the clone.
Configure the new volume into the virtual disk service:
control_domain # ldm add-vdsdev /dev/zvol/dsk/guestpool/sys2 sys2@primary-vds0
Swap the current root disk with the new root disk:
control_domain # ldm remove-vdisk sys1 guest1
control_domain # ldm add-vdisk id=0 sys2 sys2@primary-vds0 guest1
control_domain # ldm add-vdisk id=2 sys1 sys1@primary-vds0 guest1
These lines have configured the cloned root volume as disk@0 in the LDOM, and added the original root volume back into the LDOM as disk@2.
Start and boot the LDOM:
control_domain # ldm start guest1
control_domain # telnet localhost ...