[virt-tools-list] [libvirt] question about syntax of storage volume <target> element

[Cole Robinson]

On 10/02/2018 01:09 PM, Jim Fehlig wrote:
> On 10/2/18 9:27 AM, Cole Robinson wrote:
>> On 10/02/2018 10:53 AM, Jim Fehlig wrote:
>>> On 10/2/18 5:13 AM, Pavel Hrdina wrote:
>>>> On Mon, Oct 01, 2018 at 02:28:09PM -0400, Cole Robinson wrote:
>>>>> On 09/28/2018 12:54 AM, Jim Fehlig wrote:
>>>>>> I've attempted to use virt-manager to create a new VM that uses a 
>>>>>> volume
>>>>>> from an rbd-based network pool, but have not been able to progress 
>>>>>> past
>>>>>> step 4/5 where VM storage is selected. It appears virt-manager has
>>>>>> problems properly detecting the volume as network-based storage, but
>>>>>> before investigating those further I have a question about the 
>>>>>> syntax of
>>>>>> the <target> element of a storage volume.
>>>>>
>>>>> Yeah virt-manager is known to be lacking WRT rbd. I did some work a 
>>>>> few
>>>>> years back but didn't finish it. At least it doesn't know how to 
>>>>> correctly
>>>>> use a volume with any auth data in the XML. I need to get another 
>>>>> rbd setup
>>>>> to test with and fix it all
>>>>
>>>> I was investigating the RBD support as well and in order to add proper
>>>> support for it into virt-manager we need to add support for secrets.
>>>
>>> Right. But I was starting with the assumption that a "storage admin" 
>>> setup an rbd-based pool, necessary secrets, etc. Then user creating a 
>>> new VM could select existing volumes in the pool or create new ones 
>>> at step 4/5 of the VM creation wizard. Currently a user can select an 
>>> existing volume or create a new one, but can't progress beyond that 
>>> point. I have a hack (attached, based against 1.5.1) to workaround 
>>> the problem in virt-manager. Commit 582c1d3d fixed virt-install to 
>>> copy auth data from the pool to the device config in domXML and as a 
>>> first step I'm trying to do the same with virt-manager.
>>>
>>
>> dropping libvir-list and adding virt-tools-list
>>
>> Cool, thanks for working on this. Can you provide:
>>
>> - pool XML you are using
> 
> <pool type="rbd">
>    <name>rbdpool</name>
>    <source>
>      <name>libvirt-pool</name>
>      <host name='foo.bar.com' port='6789'/>
>      <auth username='libvirt' type='ceph'>
>        <secret uuid='c451d49d-a48b-43d8-bd78-97ccebd7d67a'/>
>      </auth>
>    </source>
> </pool>
> 
>> - vol XML
> 
> <volume type='network'>
>    <name>test-image</name>
>    <key>libvirt-pool/test-image</key>
>    <source>
>    </source>
>    <capacity unit='bytes'>21474836480</capacity>
>    <allocation unit='bytes'>5481955328</allocation>
>    <target>
>      <path>libvirt-pool/test-image</path>
>      <format type='raw'/>
>    </target>
> </volume>
> 
>> - the backtrace you are hitting
> 
> The first hunk in diskbackend.py of my hack fixes this backtrace 
> (including a bit of preceding context)
> 
> [Tue, 02 Oct 2018 10:53:41 virt-manager 20533] DEBUG (storagebrowse:53) 
> Showing storage browser
> [Tue, 02 Oct 2018 10:53:57 virt-manager 20533] DEBUG (storagebrowse:138) 
> Chosen volume XML:
> <volume type="network">
>    <name>test-image</name>
>    <key>libvirt-pool/test-image</key>
>    <source>
>    </source>
>    <capacity unit="bytes">21474836480</capacity>
>    <allocation unit="bytes">5481955328</allocation>
>    <target>
>      <path>libvirt-pool/test-image</path>
>      <format type="raw"/>
>    </target>
> </volume>
> [Tue, 02 Oct 2018 10:53:57 virt-manager 20533] DEBUG (storagebrowse:66) 
> Closing storage browser
> [Tue, 02 Oct 2018 10:54:04 virt-manager 20533] DEBUG (diskbackend:170) 
> Attempting to build pool=libvirt-pool 
> target=/usr/share/virt-manager/virtinst/libvirt-pool
> [Tue, 02 Oct 2018 10:54:04 virt-manager 20533] DEBUG (storage:524) 
> Creating storage pool 'libvirt-pool' with xml:
> <pool type="dir">
>    <name>libvirt-pool</name>
>    <target>
>      <path>/usr/share/virt-manager/virtinst/libvirt-pool</path>
>    </target>
> </pool>
> [Tue, 02 Oct 2018 10:54:04 virt-manager 20533] ERROR (error:143) 
> Validation Error: Storage parameter error. Could not start storage pool: 
> cannot open directory '/usr/share/virt-manager/virtinst/libvirt-pool': 
> No such file or directory
> Traceback (most recent call last):
>    File "/usr/share/virt-manager/virtManager/create.py", line 2305, in 
> _validate_storage_page
>      path=path)
>    File "/usr/share/virt-manager/virtManager/addstorage.py", line 262, 
> in validate_storage
>      disk.path = path or None
>    File "/usr/share/virt-manager/virtinst/devicedisk.py", line 519, in 
> _set_path
>      (vol_object, parent_pool) = diskbackend.manage_path(self.conn, 
> newpath)
>    File "/usr/share/virt-manager/virtinst/diskbackend.py", line 176, in 
> manage_path
>      pool = poolxml.install(build=False, create=True, autostart=True)
>    File "/usr/share/virt-manager/virtinst/storage.py", line 559, in install
>      raise RuntimeError(errmsg)
> RuntimeError: Could not start storage pool: cannot open directory 
> '/usr/share/virt-manager/virtinst/libvirt-pool': No such file or directory
> 
> After fixing the above I see errors trying to change the permissions on 
> the volume
> 
> [Tue, 02 Oct 2018 10:58:35 virt-manager 20822] DEBUG (addstorage:143) No 
> search access for dirs: ['libvirt-pool', '']
> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (addstorage:156) 
> Attempting to correct permission issues.
> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (devicedisk:290) 
> Ran command '['setfacl', '--modify', 'user:qemu:x', '']'
> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (devicedisk:292) 
> out=b''
> err=b'setfacl: : No such file or directory\n'
> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (devicedisk:325) 
> setfacl failed, trying old fashioned way
> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (devicedisk:297) 
> Setting +x on
> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (devicedisk:297) 
> Setting +x on libvirt-pool
> [Tue, 02 Oct 2018 10:59:46 virt-manager 20822] DEBUG (addstorage:171) 
> Permission errors:
>   : [Errno 2] No such file or directory: ''
> libvirt-pool : [Errno 2] No such file or directory: 'libvirt-pool'
> It is very likely the VM will fail to start up.
> 
> The hunk in devicedisk.py fixes this by skipping the permissions check 
> for network-based volumes.
> 
>> Maybe I can distill that stuff into something that's easy to reproduce 
>> with the testdriver
> 

Ahh now that I look at the diffs, they are both in sections of the code 
that only run or fail in real scenarios: manage_storage will always 
succeed with the testdriver, and generally the path perms checking isn't 
run in the test suite. I can probably hack up some unit tests to 
demonstrate the fixes though.

But yeah as you've seen this boils down to virtinst having the wrong 
assumption that a network volume <path> is easily identifiable as a URL, 
which while it works for gluster doesn't work for rbd/sheepdog.

I'll help with unittests when the patches are ready for posting

Thanks,
Cole