[virt-tools-list] [virt-manager PATCH RFC 3/4] virt-manager: OVMF support

Tue Sep 16 14:41:31 UTC 2014

On 09/16/2014 07:17 AM, Laszlo Ersek wrote:
> On 09/15/14 23:10, Cole Robinson wrote:
>> On 09/11/2014 12:56 PM, Giuseppe Scrivano wrote:
>>> Signed-off-by: Giuseppe Scrivano <gscrivan at redhat.com>
>>> ---
>>>  ui/create.ui          | 325 +++++++++++++++++++++++++++++++++++++++++++++++---
>>>  virtManager/create.py |  59 ++++++++-
>>>  2 files changed, 369 insertions(+), 15 deletions(-)
>>
>> The UI is fine for a 'nuts and bolts' type view but IMO we should strip away
>> some of the power while also drastically simplifying the common case.
>>
>> I think we should shoot for just adding a simple dropdown with BIOS and UEFI
>> options in the VM Details Boot page, and if needed at install time people have
>> to go through 'Customize before install'.
> 
> In general, I'm OK with this, as long as all options are available to
> the user ultimately, in some view; but it won't be very convenient. See
> below.
> 
>> When the user selects UEFI we try to
>> give them the optimal config.
>>
>> To do this the friendly way, we need to hardcode the location for the UEFI
>> roms and NVRAM templates. This sucks :) Ideally these would be treated like
>> <emulator> and xen <loader> paths, detected and exposed to clients via
>> capabilities XML. This 1) gives apps a way to see that UEFI is installed, even
>> on remote machines, 2) allows us to specify optimal defaults, and 3) tells us
>> that libvirt actually supports all the relevant rom/nvram bits for that
>> hypervisor. Triple whamy! :)
> 
> I don't know how <emulator> and xen <loader> paths are interrogated, but
> I guess libvirtd could somehow return the first components of elements
> in the nvram list, in qemu.conf. That would tell you what OVMF binaries
> libvirtd has a matching varstore template for, which basically means
> what OVMF binaries libvirtd knows about. You could consider the first
> element in that list the "default OVMF choice" on the host.
> 
> Of course I defer to Michal :)
> 
>> #3 there is also important, because we will want to disable the UEFI UI option
>> if qemu/libvirt is too old, or if UEFI roms aren't available on the host
>> machine, and we will want to do it with an informative warning message or
>> tooltip. People will definitely want to try this and we will need to make it
>> as clear as possible when the needed bits are missing, which is going to be
>> common since ex. Fedora isn't even officially shipping OVMF.
> 
> The easiest way for upstream users to play with *fresh* OVMF right now
> is to install Gerd's RPMs (and keep updating them)
> <https://www.kraxel.org/repos/>. The RPM you care about most is
> edk2.git-ovmf-x64, and the files in it that you care about most are
> 
> /usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd
> /usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd
> 
> These are not "standard" pathnames on any distro I know of or can
> imagine, and that's not a coincidence -- these files shouldn't conflict
> with anything anywhere. But, you *do* want to make them easily available
> for users in virt-manager.
> 
> One thing you can say is that whoever installs this RPM better modify
> the "nvram" stanza in "/etc/libvirt/qemu.conf", and then libvirt will
> "know" about it, both for the (original) purpose of locating the
> matching varstore template, and for the (proposed) purpose of offering
> it as an option to virt-manager (in the capabilities XML or elsewhere).
> 
> The other possibility is a notice for users that they simply navigate
> virt-manager to the "Customize before install" dialog, where they can
> freely provide an OVMF binary and its matching varstore template file.
> 
> We have three guidelines / facts here:
> (a) Dan convinced me that users should be able to set these values
> without reconfiguring libvirtd,
> (b) you're saying that OVMF should be easy to use,
> (c) the currently easiest way to use *fresh* OVMF (which you really do
> want to use) is Gerd's RPMs.
> 
> These together are telling me that you can't *bury* the custom OVMF
> selection widgets on some obscure dialog. Right now the optimal way for
> upstream users *is* the custom OVMF selection. It might appear as a nuts
> n' bolts view, but right now that provides the most direct route for
> upstream users.
> 

I agree with your points, but we are talking about different levels of easy.

Yes, the easiest way to consume bits right now is using gerds custom repo.
However anyone that will go to that effort will also likely be fine consuming
a virt-install command off a wiki page. They aren't who we should target with
virt-manager UI IMO.

I'm thinking about the future when a distro is shipping an ovmf package.
People that want to use VMs + UEFI shouldn't need to know the paths to the
files, etc. The instructions should be along the lines of 'find the Firmware
option in the UI and change it to UEFI'.

>> Granted, with that approach libvirt will have to hardcode OVMF paths to
>> inspect, but that's better for clients, and actually much simpler if other
>> distros package their bits at different locations, they can just patch libvirt
>> and well behaved tools should do the right thing.
> 
> Again, the first components of the nvram list elements in qemu.conf
> could double for this purpose (so no patching needed), but, again :),
> DanPB expressily wanted to allow users to work with custom OVMF installs
> without changing system-wide configuration.
> 

Right, we aren't taking that option away, I'd just prefer not to expose it in
the UI. More for us to maintain + test for an audience that would be nearly as
well served with a virt-install or virt-xml example.

>> I'd prefer this to not be in the new VM wizard, since I think most people
>> won't know what choice to make, so we should take it out of the common path.
> 
> I believe that upstream users who want to run *fresh* OVMF will have to
> know anyway. The optimal way is to install Gerd's RPMs, and either
> reconfig libvirtd accordingly (and then the custom selection can remain
> buried), or specify the custom binary & varstore template paths.
> 
>> There's also potential problems, like does windows XP install on pure UEFI?
> 
> No. ("I very seriously doubt that".) But why is that a problem? The
> default selection is SeaBIOS, and to get to that option in the current
> version of the patchset, you already have to open an expander called
> "Advanced" or some such.
> 

Yeah that wasn't a good point on my part :) It's just that with the new VM
wizard I like to try and avoid situations where users clicking around can
shoot themselves in the foot. Ignorantly selecting UEFI + winxp is one of
them... but we'd have the same problem in the customize dialog as well so its
kind of moot.

>> If there are people out there that want to always default to UEFI maybe we
>> could add a global preference for them to set, but that could come later if we
>> decide it's needed.
>>
>> So, thoughts on that?
>>
>> Assuming that's okay, some clarifying questions:
>>
>> - What is the optimal config for UEFI? The template based NVRAM? From a
>> management perspective that certainly sounds simplest.
> 
> Yes, split files (binary & varstore template) are optimal.
> 
> The unified OVMF.fd files are a dead-end for management purposes.
> 
>> - Will the OVMF package be shipping an NVRAM 'template'?
> 
> Yes.
> 
> Old OVMF builds exist of course that don't provide the split files, but
> virt-manager should expressly ignore those builds. Libvirtd *does*
> accommodate them, and so does virt-install, but virt-manager shouldn't.
> It would be possible, but it would make the GUI even more complex. The
> unified OVMF.fd files (where the file contains a live varstore, and a
> basically non-updateable binary) is just a dead-end, management-wise.
> 
>> - Would we want a CSM option in that list (or UEFI + BIOS, or some other name)?
> 
> You don't want to dedicate a separate option to that on the GUI. The
> custom UEFI selection (by pathname) allows the user to run whatever he
> likes.
> 
> OVMF's build can be influenced by several orthogonal build time
> switches, which leads to "combinatorial explosion". CSM vs. no-CSM,
> secure-boot vs. no-secure-boot, debug log on serial port vs. debug log
> on qemu debug port, old UEFI shell vs. new UEFI shell, Intel PROEFI
> e1000 driver built into the firmware vs. not, and so on.
> 
>> - Also, further down the line, will we ever use UEFI by default? Maybe for
>> certain OS versions? Just something to consider?
> 
> Yes, that would be a reasonable feature request, later. Especially if
> you restrict OVMF as default choice to some OSes at first.
> 
>> - What's the minimum qemu version we should target with this stuff as well? Is
>> the pflash stuff new?
> 
> It's not easy to say. OVMF doesn't only depend on pflash support, there
> are dependencies on the ACPI payload that qemu generates as well. I
> recommend to stick with the latest stable qemu release, 2.1.1.
> 
> There's no reason for a distro to upgrade virt-manager & libvirtd to
> bleeding edge without doing the same with qemu.
> 
>> And some minor implementation bits to consider:
>>
>> - For existing VMs, we can show the firmware field in VM Details -> Boot as a
>> readonly label
> 
> I agree, that's a good idea.
> 
>> - Make sure to hide the UEFI UI on archs that don't support it
> 
> Probably useful, yes.
> 

Cool, thanks for all that info.

>> - We should also come up with a virt-install convenience option, like '--boot
>> uefi' or similar. But of course keep the fine grained options as well.
>>
>> - People that need advanced config, like pointing to custom nvram, or custom
>> ovmf, we should point them at virt-install and virt-xml.
> 
> I disagree (see near the top).
> 
> (1) On a distribution that makes fresh OVMF readily available, the
> optimal config comes together as follows:
> 
> (a) as part of the libvirtd package, /etc/libvirt/qemu.conf is
> immediately installed with an nvram stanza that references the
> OVMF_CODE.fd and OVMF_VARS.fd files that are installed by the
> distribution's own OVMF package.
> 
> (b) virt-manager either hardcodes the pathname of the same OVMF_CODE.fd
> file (visible in this patchset), or (as you propose) retrieves it from
> libvirtd in some way.
> 
> (c) The user then picks "system-wide UEFI" (or, in your proposal, it
> picks the *only* UEFI option).
> 
> (d) virt-manager then generates the following XML:
> 
>   <loader readonly='yes' type='pflash'
>    >/usr/share/OVMF/OVMF_CODE.fd</loader>
> 
> That's it. The OVMF_CODE.fd pathname here will equal that from step (1b).
> 
> (e) In response, libvirtd will derive the pathname of the varstore
> template (based on qemu.conf / "nvram" in step (1a)), and copy that to a
> VM-specific file (== the VM's private varstore instance).
> 
> (f) When the distro's own OVMF package is updated, the OVMF_CODE.fd and
> OVMF_VARS.fd files are replaced automatically, but the VM keeps its own
> private varstore instance.
> 
> (2) On a distribution that makes only ancient OVMF available, or none at
> all, the optimal config is this:
> 
> (a) The user installs Gerd's package.
> 
> (b) The user *optionally* bothers to edit /etc/libvirt/qemu.conf so that
> the nvram stanza begins with
> 
> nvram = [
> 
> "/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd:/usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd",
>   ... further OVMF binaries and their varstore templates ...
>   ]
> 
> (c) If the user performed step (2b), *and* libvirtd is enhanced to
> provide "/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd" from the
> above nvram stanza to virt-manager, then the simplest for the user is to
> pick the "system-wide UEFI" option (or, according to your suggestion,
> the *only* UEFI option).
> 
> The XML that virt-manager generates in this case is
> 
>   <loader readonly='yes' type='pflash'
>    >/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
> 
> and (1e) will apply.
> 
> (d) If the user did not perform step (2b), or he did but libvirt cannot
> (yet) return that value to virt-manager, then the easiest way for the
> user is to specify those two pathnames manually under the "custom UEFI"
> option.
> 
> Virt-manager then generates
> 
>   <loader readonly='yes' type='pflash'
>    >/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
>   <nvram template='/usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd'/>
> 
> In response, libvirtd will directly know the pathname of the varstore
> template, and copy that to a VM-specific file (== the VM's private
> varstore instance).
> 
> (e) When Gerd's RPMs are automatically rebuilt and the user runs "yum
> update" (or whatever his distro calls that command), then the
> /usr/share/edk2.git/ovmf-x64/OVMF_{CODE,VARS}-pure-efi.fd files are
> replaced automatically, but the VM preserves its private varstore.
> 
> 
> In other words, if you prevent the upstream user from specifying a
> custom OVMF binary (and the matching varstore template) in virt-manager,
> then you force the user to edit /etc/libvirtd/qemu.conf, *and* you force
> libvirtd developers to expose the contents of "nvram" in qemu.conf to
> virt-manager in some dynamic way.
> 

Users aren't forced to edit qemu.conf, only 'forced' to use virt-install instead.

And I'm not forcing libvirt developers to do anything: if the suggestion is
rejected, then we will cope. FWIW I'm happy to patch libvirt to do what I
want, time permitting I'll take a crack at it later today + review michal's
patches.

>> In fact if developers
>> want to switch back and forth between different uefi configurations, virt-xml
>> is far more optimal then poking around in virt-manager UI anyways.
> 
> I agree with that, but it's not about developers.
> 
> Consider a minimal howto for users that want to create a *fresh* OVMF
> virtual machine. It goes like this (with this version of the patchset):
> 
> (1) Install Gerd's repo:
> 
>   wget -O /etc/yum.repos.d/kraxel-firmware.repo \
>     https://www.kraxel.org/repos/firmware.repo
> 
> (2) Install OVMF:
> 
>   yum install edk2.git-ovmf-x64
> 
> (3) Start virt-manager, and proceed with VM creation as you normally do
> until Step 5.
> 
> (4) In step 5:
>   - open Advanced options
>   - select Firmware Type as "Custom UEFI"
>   - enter the following values:
>     UEFI binary:
>       /usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd
>     UEFI varstore template:
>       /usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd
>   - click Finish; you're done
> 
> According to you suggestion, it would say:
> 
> (1) same
> (2) same
> 
> (3) edit /etc/libvirt/qemu.conf:
> 
>   nvram = [
> 
> "/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd:/usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd"
> ]
> 
> (4) restart libvirtd
> 
> (5) Make sure you have a libvirtd version running that can expose the
> contents of "nvram" in /etc/libvirt/qemu.conf to virt-manager dynamically
> 
> (6) Start virt-manager, and proceed with VM creation as you normally do
> until Step 5. [unchanged]
> 
> (7) In step 5:
>   - open Advanced options
>   - select Firmware Type as "UEFI"
> 
> Step 7 would be easier, but steps 3, 4 and 5 are harder.

Indeed, the latter set of options is more confusing no doubt. But if people
want to use custom ovmf paths we would not recommend virt-manager, and instead
point them at virt-install. So the documentation would be:

- Do you want to use the recommended UEFI config using stock distro packages?
Create a VM as usual, on the last page select customize before install, go to
the boot page, select firmware=uefi, finish install.

- Do you want to use custom UEFI config? Use this virt-install command for
fedora 20 install: virt-install --name f20 --ram 2048 --os-variant fedora20
--cdrom $FEDORA-DVD --disk size=10 --boot loader=FOO,loader_type=pflash,nvram=...

- Cole