shm as domain to virt-viewer protocol?

[David Geise]

Hi,

I've been using an open-source project that's currently in beta called Looking Glass (https://looking-glass.hostfission.com/) to achieve high-performance VM graphics in a standalone workstation scenario where the guest VM and viewer are both on the host. I use it to virtualize my windows development environment.  I need it to do work in the Unreal Engine, which requires full GPU acceleration.  My understanding is that currently GPU isn't supported directly in the virt-manager/virt-viewer environment.  I know the cloud is the big thing these days but this windows vm-on-host scenario is an increasingly common use-case and it seems like there's a great opportunity to improve virt-viewer/virt-manager here.  I expect if Linux were to better support a high-performance windows+gpu on linux scenario, this should improve consumer pc uptake of linux.

The gist of Looking Glass is that it uses a shared memory (shm) device for IO from the guest to the viewer, bypassing serialization & network layers that are required for networked scenarios but is an unnecessary overhead & a performance bottleneck in a standalone scenario.  It's great in that it does achieve quite low latency, but it's a bit problematic in other respects, mostly due to the developer's 'roll my own' approach which currently results in poor integration into libvirt's extensive features & good integration among other things.  I inquired and the developer doesn't seem interested in contributing his work to virt-manager.  He seems to think the architects in charge are intrinsically opposed to the idea of using shm as a communication transport.  This doesn't really align with my understanding around what's happening in paravirtualization and things like virtio-fs's memoyr-backed-file, etc, but the developer seems to have pretty strong views.  Perhaps he'll achieve his goals, but it'll be a heavy lift for a single developer or even a small group, especially without support from the larger community.

Looking glass's approach uses a windows 'host' application running in the vm that starts at login, essentially performing screen-captures and block-copies the data into shm.  A 'client' application running on the host then reads shm and renders into a custom window, etc.  It works surprisingly well.  However, as I see it, it's lacking in several aspects:

1. It's not multi-platform, it's windows-only and intended primarily for playing windows computer games.

2. Running as an unsigned client application, only minimal VGA/VNC/Spice support, makes it somewhat problematic.  For initial setup or if anything goes wrong, you really need a 2nd monitor/keyboard/mouse available & correctly configured to troubleshoot issues.  The client application really needs to become a signed driver.

3. Incomplete support for Spice, etc results in poor UX.  Things that virt-viewer do well are not supported - like lack of audio support, inability to change client screen resolution, etc are generally problematic and the developer states he has no interest in fixing them.  The goal of perfection in low-latency seems to be interfering with the good of the best overall user experience.

With all that being said, I'd like to ask the virt-viewer developers:

1. Is there any interest in developing a low-latency 'standalone' transport for virt-viewer?  Is any such work contemplated or underway?  Or is some alternative (virtio-gpu comes to mind) being worked on that is an even better alternative?

2. Is there truly an intrinsic opposition to, say, a proposed Gtk-shm widget & corresponding driver being added to the virt viewer project?  Would it be possible / non-problematic to develop a platform-neutral solution to optimize latency for this local vm scenario?

I'm contemplating whether to try to fork virt-viewer and do a shm transport integration myself or if I should contribute to the looking-glass project instead.  Would the virt-developers like to participate or take the lead in this?  Would the repository owners be open to a pull request to such a contribution?  Would RedHat sign such a driver?  I expect there might be strategic / political implications at play here.

For what it's worth, I've got 25 years of multi-language experience, mostly heavy C++, mostly on windows.  My C / DDK experience is rusty (like 15 years since I've coded in raw C, even longer since unix!) but I might be able to contribute something worthwhile.

Apologies if this went tl;dr.  I hope you guys are open to input & new ideas - perhaps we can improve this already great software and get linux uptake to the next level.

Regards,
Dave G