[virt-tools-list] [virt-manager PATCH][v2][RFC] Introduction of cloud-init configuration in virt-install

[Cole Robinson]

On 6/24/19 12:40 PM, Daniel P. Berrangé wrote:
> On Mon, Jun 24, 2019 at 12:12:16PM -0400, Cole Robinson wrote:
>> On 6/21/19 1:18 PM, Athina Plaskasoviti wrote:
>>> I am sending a v2 of the patch for cloud init basic configuration during cloud image installation, applying previous suggestions.
>>>
>>> Fixed: 
>>>
>>> -the way the installation is triggered (no_install is no longer required):
>>> 	--install is_cloud=yes ... --import
>>>
>>> -added tmpfile for the files that are being generated in the installation process 
>>> (meta-data, user-data, config.iso)
>>>
>>> -changed condition in installerinject.py from label='cloud' to cloudinit=True
>>>
>>
>>
>> I did some digging into cloud-init behavior to try and get a better idea
>> of what the minimal viable config is, and get ideas for future work.
>>
>>
>> cloud-init behavior
>> ===================
>>
>> The patch generates an .iso with volume label 'cidata' which is the
>> magic to make cloud-init search it for config data. cloud-init must find
>> a 'meta-data' and 'user-data' file in order to consider the iso a valid
>> data source. There isn't any requirements on the contents though. They
>> can both basically be empty AFAICT.
>>
>> This patch is setting meta-data 'instance-id' and 'hostname' fields but
>> neither are required: instance-id seems to only be needed on real cloud
>> providers (and cloud-init fills in a 'nocloud' value for us by default).
>> hostname will predictably set the VM hostname but that's not something
>> we need to set by default. So in fact meta-data is fine to leave empty
>> and I think that should be our default behavior.
>>
>> Every distro cloud image has some default config in
>> /etc/cloud/cloud.cfg. This is generated from cloud-init.git
>> tools/render-cloudcfg which generates cloud.cfg from
>> config/cloud.cfg.tmpl. Basically the only input to the template is the
>> distro name. For example fedora generates in in the RPM spec with simply:
>>
>> python3 tools/render-cloudcfg --variant fedora >
>> $RPM_BUILD_ROOT/%{_sysconfdir}/cloud/cloud.cfg
>>
>> cloud images seem to have this default behavior:
>> - ssh is enabled and running
>> - root account is locked
>> - cloud.cfg will explicitly disallow ssh to the root account, via
>> disable_root. Attempts to ssh to root will print an error message
>> pointing to the default account
>> - cloud.cfg will create a default account, locked, with full sudo access.
>>
>> The name of the default account depends on the value passed to the
>> tools/render-cloudcfg script. For Fedora it's 'fedora', Ubuntu is
>> 'ubuntu'. The openstack page here documents the common ones:
>> https://docs.openstack.org/image-guide/obtain-images.html
>>
>> An extra note: I was wrong when I said in other threads that fedora
>> images have a default password, they just have a default account name
>> which is locked by default.
> 
> That good, so I was in fact right in remembering that.
> 

Yeah, sorry for the confusion.

>> virt-install cli
>> ================
>>
>> I think the virt-install entry point for requesting cloud-init stuff
>> should actually be its own top level option, like --unattended, because
>> we may want to add suboptions and I don't want to mix those in with
>> --install. So I'm thinking:
>>
>> # does whatever we choose for default behavior, like --install is_cloud
>> virt-install --cloud-init ...
>>
>> # User provides their own user-data file. Maybe for meta-data too?
>> virt-install --cloud-init user-data=/path/to/user-data
>>
>> # Specify password for the default user. Yeah I know this isn't
>> # a good pattern to put a password on the cli but we could combine it
>> # with immediate password expiration so the user has to change it
>> # on first log in
>> virt-install --cloud-init password=FOOBAR
>>
>> # Something like this to inject an ssh key? But honestly I'm not
>> # convinced we should go down that route and instead just make users
>> # manually set this themselves like they would for any other VM
>> virt-install --cloud-init sshkey=/path/to/key
> 
> I think ssh keys is the preferred way to setup cloud images in many
> cases as no one sane should ever be using passwords for ssh auth these
> days given the continual brute force attacks they face.
> 
> Saying users must set a password and then manually copy their ssh key
> in afterwards is crazy when cloud init supports getting the ssh key
> in straight away without needing passwords.
> 
> I would not be surprised to even find cloud images which have
> explicitly disabled SSH passwords entirely leaving SSH keys as the
> only option. Some organizations will have a complete ban on use of
> passwords for login purposes too.
> 

My impression is that most people trying to use these images are just
trying to get quick access to a distro VM and think a small
pre-installed image is the way to go about it. They expect it to 'just
work' like say a virt-builder image but don't realize that cloud-init is
going to get in their way. They see the VM hang for a while then drop to
a login prompt and they don't know how to log in.

I think the goal of this work should be to make the images work like a
typical virt-install/virt-manager import, not to mirror the expected
cloud image interaction model. If a user is coming to these images with
knowledge of cloud-init they can probably google their way out of the
problem. We can give them the tools like an sshkey= option to make it
work like they want

Trying to facilitate ssh by default opens this whole can of worms. You
won't be able to login by default with virt-viewer/virt-manager or virsh
console. We have to have some logic for finding user ssh keys, figure
out how to deal with multiple keys, and error if the user doesn't have a
key. If we want to autolaunch ssh from virt-install, that means the user
sees no boot output, we need to wait for the guest IP address which
means validating that's possible with the selected network method, it
would only work remotely if we used an ssh tunnel. There's probably more
issues. Honestly I don't think it's worth trying to figure out this
whole new paradigm for a reasonably niche case like this.

>  
>> --cloud-init default behavior
>> =============================
>>
>> This is the bit I'm not really sure about. We need to manipulate some
>> password in some way for the user to be able to log in.
>>
>> For starters I think no matter what option we choose we should set the
>> password to expire on first log in, thus forcing the user to choose a
>> password of their own.
>>
>> The safest: generate a long random password for the default user. Print
>> it to the virt-install console. 'virt-builder fedora-30' does this, for
>> example, using what looks like 16 char [a-zA-Z0-9]. User has to
>> transcribe it into the VM twice (once for initial login, once for
>> immediate password expiration). We probably want to pause for a couple
>> seconds after we print it to the console so the user actually notices it
>> was there before we launch the graphical console and steal their
>> attention. Having to hand type a password into a graphical console kinda
>> sucks because copy/paste isn't going to work out of the box with the
>> minimal cloud images. This interaction can be improved if we use 'virsh
>> console' by default for cloud images, which for Fedora at least are
>> smart enough to print to the serial console, I suspect others are as well.
> 
> I guess it depends how you expect the VM to be access. Typically cloud
> images are deployed & accessed headless, with SSH being the primary access
> point. While clouds provide graphical console, that's not normally used for
> more than troubleshooting when it fails to boot such that SSH isn't working.
> 
> For virt-install deployed VMs I think expecting headless access via SSH
> is a reasonable assumption & SSH keys are desirable.
> 
> For virt-manager though, you're directed straight to graphical console
> for which you're going to want a password.
> 
> For virt-manager presumably there'll be some UI to enable cloud init
> during new VM creation ?  In which case the user can simply enter a
> password during initial create. Could let them select an SSH public
> key as an option.
> 

Yeah I think we will end up with explicit UI like that

>> The simplest: use the guest name as the default password. It's not
>> hardcoded, it's simple to type. Less safe though because the VM is
>> exposed to ssh for that first bootup with a possibly predictable password.
> 
> I would consider any use of a predictable default password to be a
> security flaw. Guest name is predictable and visible to any process
> on the host so is not safe.
> 

That's fair. But I'm really not into the idea of trying to make ssh work
automagically. Providing explicit options to use cloud-init ssh key
injection is fine with me though, for the cli at least. UI I'm unsure

But for the default, I think that leaves generating a random password
for root, similar to what is done with virt-builder. root is not
accessible over ssh, so we drop them into 'virsh console' and they can
copy/paste it if they want. Then the password is expired and they set
their own on first login.

This simplifies things in the short term too because we don't need
libosinfo to track the default username for each cloud distro

>> Some of the general downsides of the 'default user' approach is that we
>> need to communicate the default user name to the user as well, which is
>> different per distro image. This means we will need to track it in
>> libosinfo. This seems worthwhile to do anyways. In the mean time we
>> could just print a warning about looking it up, or take a guess.
>>
>> Something involving root: Maybe we don't touch the default user at all
>> and instead set a root password. The 'root' user is well known, and root
>> is blocked from ssh by default so there isn't any fear of network
>> access. but this kinda violates the intentions of the distro disk images
>> which lock down root by default and want people to use an unprivileged
>> user but with sudo access. It also means we invalidate some of the
>> documentation out there about the image's default user name.
>>
>> So I'm not really sure what to do, I'm interested in suggestions. We
>> should also consider how this will could possibly be exposed in
>> virt-manager as well. If we can ever detect that an image needs
>> cloud-init, we will either need to force the user to provide a password,
>> or inform them in some way what the default user/pass is
> 
> I don't think we need to be able to detect it. During new VM wizard
> we simply need an checkbox option to enable cloud-init data source.
> When ticked we let them either enter a password or select an SSH
> key. We can prefill the password box with a randomly generated choice
> which can be changed or deleted as desired.

Yeah but this doesn't help the ignorant user who doesn't understand
cloud images do not 'just work'. So I think considering this is still
valuable. Even if we don't find a solution I'd probably run a regex over
the image filenames and if it sounds 'cloudy' print a warning pointing
users to --cloud-init

- Cole