[virt-tools-list] virt-install and cloud-init, feedback wanted

[Daniel P. Berrangé]

On Thu, Nov 21, 2019 at 09:39:46PM -0500, Dusty Mabe wrote:
> 
> 
> On 11/21/19 6:20 AM, Daniel P. Berrangé wrote:
> > On Thu, Nov 21, 2019 at 11:07:24AM +0000, Richard W.M. Jones wrote:
> >> On Thu, Nov 21, 2019 at 10:34:14AM +0000, Daniel P. Berrangé wrote:
> >>> On Wed, Nov 20, 2019 at 08:18:01PM -0500, Dusty Mabe wrote:
> >>>> Basically in Fedora CoreOS we need a generic user data mechanism that works across
> >>>> platforms (x86_64, aarch64, ppc64le, s390x) and doesn't have possible race conditions.
> >>>> Right now we're using `-fw_cfg` but it's not cross platform. We don't have an answer
> >>>> yet: https://github.com/coreos/ignition/issues/666#issuecomment-452835654
> >>>
> >>> For platform portability you need to find some device that is common
> >>> across all platforms, and either disk or network are the only two
> >>> good options that exist today or for the forseeable future.  If those
> >>> aren't acceptable then all we have left are platform specific options.
> >>
> >> While it's not a "good option that exists today", AF_VSOCK would be a
> >> good choice to settle on in the future.  It's completely cross
> >> platform, available for Windows, and doesn't interfere with existing
> >> network or disk devices.
> >>
> >> Would needing virtio be a barrier?  Our impl of AF_VSOCK runs over
> >> virtio, but there are other transports.
> > 
> > From a cloud-init POV, I don't see virtio as a barrier. Defining an
> > AF_VSOCK data source for it should be quite straightforward really
> > and they already have so many data sources, it seems reasonable
> > that they'd accept one more.
> > 
> > On the host side there's still the task of providing a metdata
> > service to expose the data, which is outside scope of virt-install.
> > 
> 
> Thanks for the fruitful conversation here regarding a cross platform data source
> that we could use. Is this worth us writing up into a request for an issue tracker
> somewhere where it could be discussed further?

Possibly - depends if AF_VSOCK is viable or not for Ignition.

Previously when we suggested use of a plain disk for Ignition, the issue
raised was that the /dev/sdXXX device nodes take a non-deterministic
amount of time to appear, and ignition doesn't know how long to wait
for them, because the disks might not exist at all in the first place.

Using AF_VSOCK is going to have exactly this same problem, so personally
I can't see it can satisfy Ignition, where virtio-blk failed.

The main benefit I see of AF_VSOCK over virtio-blk disk / cdrom, is that
it is a live data channel so you are not restricted to providing data that
is fixed at boot time, it can be live updated on the fly.

The only mechanism that can avoid the problem of waiting for some
device driver to initialize asynchonrously is to use a directly
mapped memory region that is immediately exposed by the core kenrel
functionality. This is what fw_cfg, SMBIOS, and/or kernel command
line args all achieve, and the other options not based on mapped
memory will all fail to achieve.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|