[virt-tools-list] virt-install and cloud-init, feedback wanted

[Daniel P. Berrangé]

On Thu, Nov 21, 2019 at 07:44:38PM -0500, Cole Robinson wrote:
> On 11/21/19 5:28 AM, Daniel P. Berrangé wrote:
> > One option is to take the lazy approach and always enable cloud init
> > if we are given a pre-built disk image.
> > 
> >  * If the disk image doesn't support cloud init, then its fairly
> >    harmless to provide cloud-init data.
> > 
> >  * If the disk image does support cloud init but has already been
> >    booted, and the user has marked cloud init as disable, then it
> >    is again harmless to provide cloud-init data
> > 
> >  * If the disk image does support cloud init but has already been
> >    booted, and the user left cloud init enabled so that it runs on
> >    every boot, then passing cloud init data is probably good thing.
> > 
> >  * If the disk image does support cloud init and has never been
> >    booted, then obviously providing cloud init data is good
> >    
> 
> If we had a SMBIOS or -fw_cfg option, then maybe this is an option. But
> I don't think I could stomach doing this by default with the current
> 'cdrom' approach, surely there would be complications, and at minimum
> complaints about a needless attached CDROM device.

Yeah, I can understand this concern.

> I think the 'cdrom' case is good enough for a first crack at this and
> anything else can build on that initial support. I think the command
> line is flexible enough that in the future we could have a --cloud-init
> method=SMBIOS/... for users to explicitly choose what injection method
> they want, but otherwise virt-install will choose a default, which may
> be different depending on the arch.

That sounds reasonable.

> >> * Do you have an opinion of what behavior bare '--cloud-init' should give?
> > 
> > My feeling is that a bare --cloud-init should inject the current user's
> > SSH identity and/or authorized keys list.
> > 
> > ie either we grab $HOME/.ssh/id_rsa.pub, or we grab all the keys from
> > the SSH agent connection (ie ssh-add -l)
> > 
> > Given that virt-intsall is run as root, there probably isn't a
> > /root/.ssh/id_rsa.pub file for most people. Grabbing the ssh-agent
> > authorized keys is probably best default approach.
> 
> How to handle ssh keys kinda makes my head spin. I've shrunk the CC list
> on this mail to just the folks that mentioned ssh in some way in their
> replies
> 
> If I'm thinking of the ideal interactive virt-install UI, it would be:
> 
> $ virt-install --name MYVM --disk Fedora-Cloud-...qcow2 --cloud-init
> INFO: SSH keys added to MYVM
> INFO: Connecting to MYVM with: ssh fedora at 192.168.122.XXX
> ...
> [fedora at localhost:~] $
> 
> 
> If all the pieces come together, that's pretty nice. But there's a lot
> of missing pieces needed for that, and some unclear bits:
> 
> * What is the cloud-init user name? Passing in an ssh key gives access
> to the default cloud-init configured username, which is different
> depending on the distro (might be fedora, ubuntu, etc). To properly
> handle this, we either need libosinfo distro detection, or use
> libguestfs to fish the default user out of /etc/cloud/cloud.cfg in the
> VM. And root account is typically locked, and ssh access disabled, and
> even if we could hack things to give root ssh access it seems like going
> against the grain WRT cloud images.

IIRC, you can pass an SSH key for root separately, but not sure how
widely supported that is. Sticking to just providing 1 SSH key for
the default unprivileged user likely suffices.

> So best I can think of for an immediate solution for SSH is to print
> something explanatory:
> 
> INFO: you can ssh to the VM using the distro's default username. This is
> usually one of: fedora, ubuntu, ...
> 
> Or something along those lines. Maybe we can tie that into --os-variant
> as well so the user can inform us what distro it is. Still it will take
> some work.

This is a bit of a tangent, but this makes me wonder if qcow2 format
could benefit from having support for arbitrary user metadata fields

People building cloud images could then tag their images with useful
things including vendor, product name, product version, whether it
supports cloud-init or ignition, what the default user account name
is, suggested min ram, and so on.

Of course people might say this is all stuff that's within scope
of external metadata like OVF, but I've rarely, if ever, seen anyone
say anything nice about OVF. Also there's a certain convenience in
being able to directly pass around a bootable disk image, rather than
a disk image + separate metadata file, or worse a ZIP file containing
both which you have to unpack.

Having some simple metadata in the qcow2 file would simplify life I
think. If someone really wanted to use OVF, they could stuff the OVF
XML descriptor into a qcow2 metdata field.

> * Where to get ssh keys from? You mention ssh-add -l which is nice, I
> didn't know about that. There's the issue with running virt-install
> under sudo, but not sure there's anything we can do about that to make
> it 'just work'. Sounds like `ssh-add -l` is the way though for some sort
> of ssh-key=auto mode

Yeah, sudo breaks it, as does "su -".   The ssh agent only gets
forwarded if you're accessing the virt host over SSH. This is
the case for me usually, as my virt servers are separate from
my laptop.

I don't see much of an alternative to ssh-agent though, besides
just defining that virt-install will grab a key from a well known
place like $HOME/.config/virt-install/ssh-injection-keys

> * What to do if we can't find any ssh-keys to add? I guess explicitly
> fail, and print a descriptive error: We didn't find any SSH keys to add.
> Maybe you are running under sudo. You can manually specify a key with
> XXX or generate root password with YYY. (something better than that of
> course)

Depends whether we want to also auto-generate root and/or user passwords
by default, or only work with SSH keys by default.

My personal preference is to only deal with SSH keys by defaults, as
passwords are just terrible in general. This would mean if we can't
find a SSH key, we should abort the install & require an explicit
setting.

> * virt-install support for launching an ssh session as the UI. This is
> probably worth doing regardless. I'd like to add an explicit
> --autoconsole none|graphical|text anyways, adding ssh to that seems
> reasonable. Requires waiting for VM IP address to pop up, but that
> shouldn't be too bad to implement. The main problem for the --cloud-init
> case is that I don't know if it makes sense to use this if we are just
> guessing about the what the default username is?

We would need to solve the default username problem reliably
for this to be usable, otherwise it'll be a neverending source of
bug reports.

> So, to get that ideal UI, or at least point the user in the right
> direction, we need to:
> 
> * determine the default user name for the disk image, which we don't
> have a solution for presently
> * successfully inject a relevant ssh key, which may not be available
> like under sudo
> 
> So there's missing non-trivial work, and there's ways it can fail.
> 
> The current UI, with root-password-generate=on (plus some work to
> default to '--console text' when --cloud-init is passed), is more
> reliable as it should work in all situations, but even when it works
> it's not as friendly as the working ssh case (need to copy the printed
> password and paste it into a terminal). And it doesn't sound like it's
> what people expect the default to be. And giving root access seems to be
> against the spirit of the cloud images, but maybe that's just for ssh access

Personally I think from a security POV we should do our best to
discourage & avoid use of passwords, at least for server based
installs.

IOW, if using the JEOS profile then we should prefer SSH keys.
Essentially my view is that passwords are just there as a fallback
for disaster recovery when you have to boot single user mode to
fix a broken system. Even then we don't really need that for
VMs, because virt-rescue provides a backdoor into any broken
VM.

The key place where passwords are needed are when we are doing
graphical installs. The user connects to the graphical console
with VNC/SPICE, and password is the only option there really.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|