[virt-tools-list] virt-manager test suite hangs on FreeBSD

[Cole Robinson]

On 12/12/19 5:17 AM, Daniel P. Berrangé wrote:
> On Thu, Dec 12, 2019 at 09:47:42AM +0100, Andrea Bolognani wrote:
>> On Wed, 2019-12-11 at 18:01 -0500, Cole Robinson wrote:
>>> On 12/11/19 7:40 AM, Andrea Bolognani wrote:
>>>> On Wed, 2019-12-11 at 07:28 -0500, Cole Robinson wrote:
>>>>> On 12/11/19 5:22 AM, Andrea Bolognani wrote:
>>>>>> I have no idea how to debug this thing. Can an actual virt-manager
>>>>>> developer jump in?
>>>
>>> I reproduced. Part of the test suite was forking off /bin/true to hit a
>>> code path. But when /bin/true wasn't available it would leave a stranded
>>> process which caused the hang. Fixed both now upstream
>>>
>>>>> You can
>>>>> use './setup.py test --debug' which may give more info where it is
>>>>> hanging. Possibly somewhere in the cli tests where we try to handle mock
>>>>> stdin or try to fake --wait timeouts
>>>>
>>>> Doing so doesn't really result in additional information being
>>>> printed out: it still just quietly hangs there.
>>>
>>> This was a separate bug, calling into the cli tools for clitest.py would
>>> reset the test suite debugging. That's fixed too
>>
>> That's awesome, thank you so much! I already posted a patch[1] that
>> enables the test suite on FreeBSD in the CentOS CI environment, so
>> if a regression manages to sneak in we'll catch it right away.
>>
>> One more thing. Right now virt-manager is the only project we have on
>> CentOS CI that uses the python3-libxml2 bindings in the first place,
>> with all others using python3-lxml instead, and that leads to a
>> reduced test matrix for it because the former are not available on
>> Ubuntu 16.04 and CentOS 7.
>>
>> So that makes me wonder, does python3-lxml provide a better API or
>> something? Would it be a massive amount of work to port virt-manager
>> to it, and would it even make sense to do so? In due time we'll stop
>> supporting those two targets anyway...
> 
> They both use libxml native library under the hood. The lxml bindings
> are exposing libxml in a way that is "more pythonic" and thus easier
> and safer to use.  So in long term I expect virt-manager would likely
> benefit from lxml, if anyone wants to do the grunt work.

I would really like to use lxml and I explored this in the past:

https://github.com/crobinso/virt-manager/tree/xml-etree

It works with lxml from pip, which contains a static copy of libxml2.
But with distro packages using dynamically linked libxml2, lxml can not
co-exist with any other non-trivial usage of libxml2 in process, aka
libvirt.

https://lxml.de/installation.html#using-lxml-with-python-libxml2
https://bugs.launchpad.net/lxml/+bug/1748019

Ignore my comments in the bug about it being fixable in libvirt; it may
be, but by doing a lot of gross hackery that is just working around
libxml2 API limitations

- Cole