[virt-tools-list] Virt Tools Survey: What to do about virt-clone

Tue May 10 14:29:19 UTC 2011

Given a couple of questions I have gotten in response to my survey
responses... let me clarify, although I imply we do a significant volume
of V2V, which we do as well, the responses below are targeted and
applicable to cloning.  Especially the need to re-personalize clones
beyond the OS scope to the application scope when and where possible.

We maintain the original VMs in the original environment, often running
clones in parallel on different environments, so cloning is done, both
within the same hypervisor scope and across different hypervisor
environments.  Our work with SPECVirt is one concrete example where we
use cloning routinely, to build multiple tiers or tiles.  Our focus thus
far has been in the lab/test/development space, but our client base is
pushing for same flexibility in QA/production environments, so the need
for better/mature tools and methods in the KVM space, including cloning,
is key to our future goals.

Schorschi Decker

VP; Sr. Consultant Engineer
ET&D Emerging Technologies / Virtualization Platform Engineering Team
Bank of America

-----Original Message-----
From: Decker, Schorschi 
Sent: Tuesday, 10 May, 2011 07:01
To: virt-tools-list at redhat.com; Cole Robinson
Cc: virt at lists.fedoraproject.org; qemu-devel at nongnu.org
Subject: RE: [virt-tools-list] Virt Tools Survey: What to do about
virt-clone

(a) Is cloning guests useful for you or not?  Often or infrequently?

Our end-users are quite virtualization aware, they have demanded that we
support a number of features in our virtual infrastructure, this
included.  It will be used frequently and often, maybe 100s of times a
day across our global infrastructure.

(b) Do you currently use virt-clone to clone guests?

Only in the lab, cloning or templating is a new initiative in our
maturing design, so the timing of revamping this is perfect for our
plans over the next few months.

(c) Do you have a homebrew method to clone guests?  What does it do?

Yes, we often migrate and/or clone VMs in the lab between VMware and KVM
and Hyper-V and back again.  VMware and KVM is quite easy, given the
flexibility of common shared storage support and VM disks in RAW format
adaptable to VMware and KVM.  Hyper-V has its own challenges given its
limitations by design.  For VMware to KVM and back, we have a few bash
scripts that we use.  Migrations are straight-forward, since no
personality customization is needed, but true clones, which we use for
SPECVirt analysis for example, we have to remove/reapply personality.
This is not trivial, since we have a lot of applications in our base
image that are infrastructure aware, and require reconfiguration, or
re-initialization, beyond the typical sysprep or OS level changes.
Windows is especially painful on this point.  Our personality management
scripting is in a constant state of change because we are routinely
updating our core images.

(d) Do you use another tool to clone guests?  (And how is it?)

We have used platespin off and on, and of course use various tools and
methods core to WAIK tool set for Windows.  But custom scripting is the
significant 800 pound gorilla in the room on this point.  A specific in
VM agent or service that would explicitly support re-personalization
regardless of OS or application level would be of great benefit, since
meta-data or a back-end database could drive the re-personalization,
provided a more stateless model.  Especially for Windows this might be
an idea worth consideration.  BladeLogic, Ops-ware, Altaris, etc. all
use some form of this idea, but they are also focused on provisioning in
general so the meta-data driven re-personalization gets lost in the
overall product.

(e) When you clone a guest, do you "sysprep" it or would you like to?
(Using the term "sysprep" generically here, I mean any sort of
reinitialization for Linux or Windows guests).

Sysprep is fine as a place to start, but even when Microsoft wrote
sysprep, it was an idea that was ignored for a long time, and it was
never focused on, the world has since gone forward, so sysprep is at
best, only a starting point for what is needed to abstract personality.
A better solution would be that we image/clone the personality specific
deltas and archive them to a library, rather than complete clones that
have personality logic staged for implementation on first boot.

(f) How do you feel about a multi-step process?

  virt-clone -> virt-sysprep -> virt-resize (for example)

As long as any GUI and CLI (as well as API) are consistent in feature
set and flow, multi-step is fine.   We need to be able to completely
automate the process, since we would drive it from a self-serve web
portal concept.

(g) Have you had other problems with cloning guests?

Nothing beyond what was outlined above.  Abstracting of personality at
OS and application level is key for us.

(h) What have I missed out in this analysis?  What other features have
you missed in virt-clone?

No specifically to virt-clone but in general, abstracting the common
code, to common library or binary, so that virt-manager, virsh, etc. all
reference the same design, feature, and executables, is the strategic
direction that should be followed.  The idea that different code is
trying to implement same concepts in different tools makes no long term
strategic sense for the KVM platform in general.

Schorschi Decker

VP; Sr. Consultant Engineer
ET&D Emerging Technologies / Virtualization Platform Engineering Team
Bank of America

-----Original Message-----
From: virt-tools-list-bounces at redhat.com
[mailto:virt-tools-list-bounces at redhat.com] On Behalf Of Richard W.M.
Jones
Sent: Tuesday, 10 May, 2011 04:57
To: virt-tools-list at redhat.com; Cole Robinson
Cc: virt at lists.fedoraproject.org; qemu-devel at nongnu.org
Subject: [virt-tools-list] Virt Tools Survey: What to do about
virt-clone

I've volunteered for the task of fixing virt-clone[0].  There are a
number of bugs which need to be addressed.  Unfortunately the current
virt-clone is broken-by-design since it cannot make changes inside the
guest.

  [0] http://linux.die.net/man/1/virt-clone

The bugs boil down to what Microsoft calls "sysprepping" the clone,
which is to say, removing its existing identity, hostname, ssh host
keys, persistent network rules, host SID and workgroup name (for
Windows).  It's helpful for Linux guests to remove some of this stuff[1]
-- it will make the cloning process smoother.  For Windows it's
absolutely required[2].

  [1]
https://rwmj.wordpress.com/2010/09/24/tip-my-procedure-for-cloning-a-fed
ora-vm/
  [2]
http://technet.microsoft.com/en-us/library/cc721940%28WS.10%29.aspx

All that virt-clone can do now is to copy the guest and make some simple
changes to the libvirt XML (eg. giving it a new MAC address).
It doesn't even address the sysprepping problem.

The problem with sysprepping is that it's hard to do, and it's different
for every operating system.  I've summarized some of the techniques
below.  Worse than that, for some OSes there are different levels of
sysprepping that an administrator might want; also see below.

So I'd like feedback from "virt-clone next generation" users:

(a) Is cloning guests useful for you or not?  Often or infrequently?

(b) Do you currently use virt-clone to clone guests?

(c) Do you have a homebrew method to clone guests?  What does it do?

(d) Do you use another tool to clone guests?  (And how is it?)

(e) When you clone a guest, do you "sysprep" it or would you like to?
(Using the term "sysprep" generically here, I mean any sort of
reinitialization for Linux or Windows guests).

(f) How do you feel about a multi-step process?

  virt-clone -> virt-sysprep -> virt-resize (for example)

(g) Have you had other problems with cloning guests?

(h) What have I missed out in this analysis?  What other features have
you missed in virt-clone?

Sysprepping Windows
-------------------

This is a complex, manual process.  We do some steps to automate it in
RHEV.  It's best to read Microsoft's online documentation at [2][3][4].

  [3] http://support.microsoft.com/kb/302577
  [4] http://blogs.technet.com/b/megand/archive/2005/01/20/357570.aspx

Fedora
------

In theory you can just write a file /.unconfigured in the root, and
Fedora will go through the firstboot process at next boot (it will reset
timezone, root password, netconfig, keyboard, authentication).

Some admins will *not* want all of these things to be reset, and will
want either a lesser degree of unconfiguration, or will want to control
each thing manually.

I'm not totally convinced that this hasn't been broken by systemd
introduction in Fedora 15.

general Linux
-------------

See [1].

Rich.

--
Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones virt-df lists disk usage of guests
without needing to install any software inside the virtual machine.
Supports Linux and Windows.
http://et.redhat.com/~rjones/virt-df/

_______________________________________________
virt-tools-list mailing list
virt-tools-list at redhat.com
https://www.redhat.com/mailman/listinfo/virt-tools-list

----------------------------------------------------------------------
This message w/attachments (message) is intended solely for the use of the intended recipient(s) and may contain information that is privileged, confidential or proprietary. If you are not an intended recipient, please notify the sender, and then please delete and destroy all copies and attachments, and be advised that any review or dissemination of, or the taking of any action in reliance on, the information contained in or attached to this message is prohibited. 
Unless specifically indicated, this message is not an offer to sell or a solicitation of any investment products or other financial product or service, an official confirmation of any transaction, or an official statement of Sender. Subject to applicable law, Sender may intercept, monitor, review and retain e-communications (EC) traveling through its networks/systems and may produce any such EC to regulators, law enforcement, in litigation and as required by law. 
The laws of the country of each sender/recipient may impact the handling of EC, and EC may be archived, supervised and produced in countries other than the country in which you are located. This message cannot be guaranteed to be secure or free of errors or viruses. 

References to "Sender" are references to any subsidiary of Bank of America Corporation. Securities and Insurance Products: * Are Not FDIC Insured * Are Not Bank Guaranteed * May Lose Value * Are Not a Bank Deposit * Are Not a Condition to Any Banking Service or Activity * Are Not Insured by Any Federal Government Agency. Attachments that are part of this EC may have additional important disclosures and disclaimers, which you should read. This message is subject to terms available at the following link: 
http://www.bankofamerica.com/emaildisclaimer. By messaging with Sender you consent to the foregoing.