[virt-tools-list] virt-install and cloud-init, feedback wanted

Thu Nov 21 15:49:40 UTC 2019

* Cole Robinson <crobinso at redhat.com> [2019-11-20 17:48]:
> Hi all. The purpose of this mail is to get some feedback on pending
> cloud-init support in virt-install. If you're on the CC list here, I
> either pulled your email from a cloud-init discussion on the the
> virt-tools-list mailing list, or from the CC list of this virt-install bug:

Hi Cole,

Thanks for starting this thread.  I'm a maintainer for cloud-init
upstream[1], happy to help answer questions and provide some suggestions
on virt-install cloud-init support.

> 
> RFE: Provide cloud-init integration for VMs
> https://bugzilla.redhat.com/show_bug.cgi?id=981693
> 
> For GSOC 2019 Athina Plaskasoviti completed some cloud-init integration
> work in virt-install. You can read her wrap up here:
> https://athinapl.home.blog/2019/08/25/gsoc-2019-cloud-init-configuration-for-virt-manager-virt-install/
> 
> Right now the code is sitting in a virt-manager.git branch, not yet
> pushed to master:
> https://github.com/virt-manager/virt-manager/tree/cloudinit
> 
> I'll summarize the current behavior, and then ask some questions.
> 
> 
> The branch adds a new 'virt-install --cloud-init' argument with several
> sub options. When specified, virt-install generates an empty meta-data
> file, a user-data file with the requested changes, stuffs them both into
> a cidata iso, which is used for the first VM boot and then deleted.

cidata maps to the NoCloud datasource[2].  iso file is not a
requirement; any block device with the 'cidata' filesystem label will
be detected and used.

> 
> This behavior is only triggered when --cloud-init is specified in some
> form, there's no automagic invocation of this support.
> 
> The command sub options are:
> 
> $ ./virt-install --cloud-init=help
> --cloud-init options:
>   disable
>   root-password-file
>   root-password-generate
>   ssh-key
>   user-data
> 
> Their behavior:
> 
> * disable=yes: boolean option to disable cloud-init in the VM for
> subsequent boots. Adds this block to user-data:
>   runcmd:
>   - [ sudo, touch, /etc/cloud/cloud-init.disabled ]
> 
> * root-password-file=/MY/PATH: set the desired root password from the
> content of /MY/PATH on the host
> 
> * root-password-generate=yes: boolean option, generate a random root
> password, set it in user-data, print it to the host text console and
> pause for a bit for the user to see and copy it. sorta inspired by
> virt-builder
> 
> * ssh-key=/MY/KEY.pub: inject the ssh key from /MY/KEY.pub on the host
> into the cloud-init user-data
> 
> * user-data=/PATH/TO/user-data: ignore all other options and copy this
> file to the .iso as user-data

You may be interested in allowing users to specify a meta-data file
as well; the meta-data file is used for setting things like the
instance-id, hostname and network-config[3].

> 
> When bare '--cloud-init' is specified, we default to generating a random
> root password and disabling cloud-init for subsequent boots:
> --cloud-init root-password-generate=yes,disable=yes
> 
> We've explicitly rejected something like root-password=MY-PASSWORD
> because of the security implications of encouraging a password to end up
> in command line history. We've already had a CVE for something similar
> in virt-install.
> 
> Also, I don't want virt-install to be in the business of specifying
> every cloud-init option under the sun, there's gotta be a better tool
> for that already. So I'd like to keep --cloud-init suboptions
> specifically targeted to expected virt use cases, and anything else can
> be served with custom user-data=

Agreed.  I think the user-data/meta-data files will provide the bulk
of what users need when working with cloud-init.

> 
> One more point: my main interaction with cloud-init has historically
> been by grabbing a Fedora/RHEL cloud image, passing it to
> virt-install/virt-manager, and watching the boot hang, because there's
> no data provider and cloud-init times out talking to the network, and
> then I can't log in. I expect many people have hit this issue before.
> I've always worked around this by using 'virt-customize' to disable
> cloud-init and reset the root password. That's about the extent of my
> usage here, which is broadly why the bare `--cloud-init` is the way it was.

For some time now, cloud-init uses a systemd-generator to detect[4] on 
which platform it is running and determine if user-data or config
has been provided and if so, enabling the cloud-init service, other
wise it stays disabled.  Your experience above is the primary reason
for this feature as now booting a cloud image without any user-data
should not mean the image hangs during boot.

The feature has been around since 2017, cloud-init 17.1 and newer
support this feature.

> 
> I'm also thinking to the future, if one day virt-install can detect that
> it was passed a distro cloud-init image, perhaps we can invoke some
> default behavior that gives the user a better chance of this config
> being usable out of the box. I figure that will match whatever we choose
> for the bare '--cloud-init' behavior

> 
> 
> So, my questions are:
> 
> * What are the usecases you see for virt-install cloud-init support?

 Exercising/testing user-data configuration locally
 Creating a template image
  - utilizing cloud-init user-data to customize the image and then
    cleaning, sanitizing for subsequent use.

> * Does the above meet your expectations?
> * Are we missing anything vital?
> * Do you have an opinion of what behavior bare '--cloud-init' should give?
> * Any other ideas, thoughts, feedback?
> 
> 
> Thanks,
> Cole

1. https://github.com/canonical/cloud-init
2. https://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html
3. https://cloudinit.readthedocs.io/en/latest/topics/network-config.html
4. https://cloudinit.readthedocs.io/en/latest/topics/boot.html

-- 
Ryan Harper
Canonical, Ltd.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: Digital signature
URL: <http://listman.redhat.com/archives/virt-tools-list/attachments/20191121/19b1ad43/attachment.sig>