[virt-tools-list] virt-v2v question involving esx

[Matthew Booth]

On 18/03/11 13:04, Richard W.M. Jones wrote:
> On Fri, Mar 18, 2011 at 08:40:31AM -0400, Kenneth Armstrong wrote:
>> On Fri, Mar 18, 2011 at 8:06 AM, Dennis Jacobfeuerborn
>> <dennisml at conversis.de>  wrote:
>>> On 03/18/2011 11:34 AM, Matthew Booth wrote:
>>>>
>>>> On 17/03/11 19:27, Richard W.M. Jones wrote:
>>>>>
>>>>> [Adding Matt]
>>>>>
>>>>> On Wed, Mar 16, 2011 at 02:59:40PM -0400, Kenneth Armstrong wrote:
>>>>>>
>>>>>> I have a vm on an esx server that I want to convert using virt-v2v. I
>>>>>> know that it can connect to the esx server and copy the vmdk files
>>>>>> down to do its conversion, but can I just use wget or something like
>>>>>> that to copy the vmdk files down first, then use virt-v2v to convert
>>>>>> the vm?
>>>>>>
>>>>>> The reason that I ask is that it would be better for me to do it this
>>>>>> way, so that after I copy the vm files down, I can start the vm back
>>>>>> up and let it be used while virt-v2v does the conversion on the copied
>>>>>> image files. I like virt-v2v, but it takes a long time to do it's job
>>>>>> with the copying.
>>>>
>>>> Are you saying that wget is substantially faster than virt-v2v's copying?
>>>> I
>>>> ask because virt-v2v uses the same http interface to retrieve the image.
>>>> It
>>>> actually streams and writes directly to the target destination, so unless
>>>> the target destination is slow, I wouldn't expect this to save you a
>>>> significant amount of time. The actual conversion process usually takes
>>>> about a minute, so the copying time dominates almost completely.
>>>>
>>>> It is possible to do what you're asking, but it's a hassle. You'd need to
>>>> obtain the storage files, create a local libvirt XML file which points to
>>>> the local copies, then run virt-v2v using the libvirtxml input method.
>>>
>>> If the vmdk file is sparse then you might end up copying a lot less data
>>> compared to a virt-v2v copy and since as you point out the copying is the
>>> thing that dominates the migration time that can potentially be a big win.
>>> It would be nice if virt-v2v had an option to not copy the block device but
>>> instead the contents of a filesystem but that would probably make things
>>> much more complicated since you'd have to support all kinds of filesystems,
>>> LVM, potentially also copy UUID's etc., etc.
>>
>> Dennis is spot on.
>>
>> If I copy down the file straight from the ESX server's web interface,
>> it's quite quick.  However, when I try it through virt-v2v, the
>> process takes hours downloading/converting the same file.
>>
>> The vmdk file in question is a sparse disk.  So I assume that virt-v2v
>> is expanding the disk out during the whole process?
>>
>> -Kenny

Sparseness isn't doing anything for you here. HTTP doesn't have the 
ability to represent sparseness[1], so it's copying all the zeroes 1 by 
1 in both cases. In order to get any benefit from sparseness in this 
case you'd need filesystem level access. I considered sshfs for this, 
but unfortunately it can't support the ioctl required to efficiently 
detect sparseness on the remote end. This means you're still left 
copying over all the data in order to find sparse sections. The only 
remaining advantage to sshfs is that it supports compression, which 
obviously works well on sparse files.

To summarise, in order to exploit sparseness on ESX we'd need either:

1. An agent running on the ESX server which can exploit sparseness.
2. An explicitly sparse file format.

Unless the former already exists (e.g. sftp-server) and I've missed it, 
this approach likely won't fly, as it would require the ESX 
administrator to install non-VMware software on their server.

I believe vmdk is able to do explicit sparse, although in practise it 
never seems to be configured this way. Even if the guest was configured 
this way, qemu doesn't support it, so we can't use it yet.

So this comes down to: why is virt-v2v's copy so much slower than wget 
doing the same thing? It could be down to inefficient chunking or 
similar, but I'd guess it's more likely related to where it's writing 
to. Is it writing to NFS by any chance? If so, what's the network like 
between you and the NFS target?

Matt

[1] Unless it supports compression, which nothing does in practise.
-- 
Matthew Booth, RHCA, RHCSS
Red Hat Engineering, Virtualisation Team

GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490