[virt-tools-list] virt-install and cloud-init, feedback wanted

[Richard W.M. Jones]

On Fri, Nov 22, 2019 at 09:56:17AM +0000, Daniel P. Berrangé wrote:
> On Thu, Nov 21, 2019 at 09:39:46PM -0500, Dusty Mabe wrote:
> > 
> > 
> > On 11/21/19 6:20 AM, Daniel P. Berrangé wrote:
> > > On Thu, Nov 21, 2019 at 11:07:24AM +0000, Richard W.M. Jones wrote:
> > >> On Thu, Nov 21, 2019 at 10:34:14AM +0000, Daniel P. Berrangé wrote:
> > >>> On Wed, Nov 20, 2019 at 08:18:01PM -0500, Dusty Mabe wrote:
> > >>>> Basically in Fedora CoreOS we need a generic user data mechanism that works across
> > >>>> platforms (x86_64, aarch64, ppc64le, s390x) and doesn't have possible race conditions.
> > >>>> Right now we're using `-fw_cfg` but it's not cross platform. We don't have an answer
> > >>>> yet: https://github.com/coreos/ignition/issues/666#issuecomment-452835654
> > >>>
> > >>> For platform portability you need to find some device that is common
> > >>> across all platforms, and either disk or network are the only two
> > >>> good options that exist today or for the forseeable future.  If those
> > >>> aren't acceptable then all we have left are platform specific options.
> > >>
> > >> While it's not a "good option that exists today", AF_VSOCK would be a
> > >> good choice to settle on in the future.  It's completely cross
> > >> platform, available for Windows, and doesn't interfere with existing
> > >> network or disk devices.
> > >>
> > >> Would needing virtio be a barrier?  Our impl of AF_VSOCK runs over
> > >> virtio, but there are other transports.
> > > 
> > > From a cloud-init POV, I don't see virtio as a barrier. Defining an
> > > AF_VSOCK data source for it should be quite straightforward really
> > > and they already have so many data sources, it seems reasonable
> > > that they'd accept one more.
> > > 
> > > On the host side there's still the task of providing a metdata
> > > service to expose the data, which is outside scope of virt-install.
> > > 
> > 
> > Thanks for the fruitful conversation here regarding a cross platform data source
> > that we could use. Is this worth us writing up into a request for an issue tracker
> > somewhere where it could be discussed further?
> 
> Possibly - depends if AF_VSOCK is viable or not for Ignition.
> 
> Previously when we suggested use of a plain disk for Ignition, the issue
> raised was that the /dev/sdXXX device nodes take a non-deterministic
> amount of time to appear, and ignition doesn't know how long to wait
> for them, because the disks might not exist at all in the first place.
> 
> Using AF_VSOCK is going to have exactly this same problem, so personally
> I can't see it can satisfy Ignition, where virtio-blk failed.

Be interesting what exactly their concerns were.  Obviously both PCI
initialization and (especially) kernel disk enumeration can take a
long time with guests that have a lot of PCI slots or disks.  But
that's what udev is for, right :-)  Couldn't they just hook up their
initialization to a udev rule?

> The main benefit I see of AF_VSOCK over virtio-blk disk / cdrom, is that
> it is a live data channel so you are not restricted to providing data that
> is fixed at boot time, it can be live updated on the fly.
> 
> The only mechanism that can avoid the problem of waiting for some
> device driver to initialize asynchonrously is to use a directly
> mapped memory region that is immediately exposed by the core kenrel
> functionality. This is what fw_cfg, SMBIOS, and/or kernel command
> line args all achieve, and the other options not based on mapped
> memory will all fail to achieve.
> 
> Regards,
> Daniel

I added Stefan to CC because he might be interested in this use case too.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW