[virt-tools-list] [PATCH RFC virtio-win-pkg-scripts v2 0/2] helpers to standardize driver directory layout

[Vadim Rozenfeld]

On Fri, 2016-01-22 at 15:30 -0500, Cole Robinson wrote:
> On 01/22/2016 01:27 PM, Roman Kagan wrote:
> > On Fri, Jan 22, 2016 at 06:05:01PM +0300, Roman Kagan wrote:
> >> The scripts are based on the idea that the most actual information about
> >> the suitability of a driver for a particular flavor / architecture is
> >> contained in the driver's catalog file (in particular, the process of
> >> ISV or WHQL siging may affect it).
> >>
> >> Since the catalog files are DER-encoded ASN.1 structures the first patch
> >> introduces a module to extract relevant information from a .cat file
> >> using PyASN1 library.
> >>
> >> The second patch introduces a script that lays out the drivers by
> >> arch/flavor.
> >> It assumes that
> >>
> >> - every driver for a particular arch/flavor is contained in a separate
> >>   directory
> >>
> >> - the directory contains a single .inf file; the basename of the file is
> >>   taken as the name of the package
> >>
> >> - the .cat file for the package is in the same directory and has the
> >>   same basename as the .inf
> >>
> >> - all the files contained in that directory are associated with the
> >>   driver and go together with it no matter if they are listed in the
> >>   .inf or in the .cat or not.
> >>
> >> The virtio-win driver packages I could get my hands on all matched the
> >> above assumptions.
> >>
> >> [ There's no integration with the rest of the system yet as I'd like to
> >> sort out some issues first which may affect the logic (in a followup
> >> mail). ]
> > 
> > Now to the issues I'd like to discuss before moving on.
> > 
> > The current make-*.py scripts can probably be simplified dramatically
> > with this new logic.  However they contain a fair amount of consistency
> > checks which I'd like to maintain.  Besides, I'd like to figure out
> > what's on input and what's the desired output of the process.
> > 
> > Here's the way I thought of using the submitted scripts (in bash for
> > simplicity of explanation):
> > 
> > ====
> > set -e
> > 
> > wrkdir=$(mktemp -d windrvXXXXXX)
> > dstdir=$wrkdir/virtio-win
> > 
> > # fetch the drivers (in real life will come from a windows build machine
> > # directly)
> > curl https://fedorapeople.org/groups/virt/virtio-win/repo/srpms/virtio-win-0.1.112-1.src.rpm |
> >   rpm2cpio |
> >   cpio -i --to-stdout virtio-win-\*-bin-for-rpm.tar.gz |
> >   tar -xzf - -C $wrkdir
> > 
> > srcdir=$(echo $wrkdir/*)
> > 
> > rm -fr $srcdir/{*.vfd,vfddrivers}
> > 
> > # get rid of duplicates (already done for the package in the above srpm
> > # but will be needed in general)
> > hardlink $srcdir
> > 
> > # lay out the drivers hardlinking them to the source
> > util/cpdrivers.py -m link $srcdir $dstdir/drivers
> > 
> > # indentify files left behind
> > find $srcdir -type f -links +1 -exec rm \{\} \;
> > echo "unpackaged files:"
> > find $srcdir -type f -printf '%P\n'
> > # do something if any is found
> > : ...
> > 
> > # add license, qga, etc. to $dstdir
> > : ...
> > 
> > # create iso image with everything
> > mkisofs -o $dstdir/virtio-win.iso -J $dstdir/{drivers,qga}
> > 
> > # create per-arch/os floppies with basic drivers
> > for archdir in $dstdir/drivers/*; do
> >     arch=${archdir##*/}
> >     for osdir in $archdir/*; do
> >         os=${osdir##*/}
> > 
> > 	img=$dstdir/${arch}_{os}.vfd
> > 	truncate -s 2880k $img
> > 	mkdosfs $img
> > 	mcopy -i $img $osdir/{netkvm,vioscsi,viostor}/*.{inf,cat,sys} ::
> >         # locate and copy txtsetup.oem (dunno if necessary)
> >         : ...
> >     done
> > done
> > 
> > # $dstdir is to be found as /usr/share/virtio-win in the resulting rpm
> > ======
> > 
> > The discussion items:
> > 
> > 1) is it a sensible thing to do in general, with these inputs and
> >    outputs?
> > 
> 
> Yes the above sounds reasonable.
> 
> > 2) the search for unpackaged files yields some.  E.g. the srpm used
> >    above gives
> > 
> >    viostor/xp/x86/viostor.sys
> >    viostor/xp/x86/viostor.inf
> >    viostor/xp/x86/viostor.pdb
> >    viostor/xp/x86/viostor.cat
> > 
> >    meaning that there's another viostor driver in the tree which claims
> >    xp-x86 support.  Indeed, 2k3-x86 driver, being different from xp-x86,
> >    claims compatibility with both xp and 2k3, and happens to take over.
> > 
> >    There are even more leftovers in the latest RHEL package, that is,
> >    there are more drivers intersecting in the supported OS lists.
> > 
> 
> Can you list the RHEL conflicts?
> 
> >    And I'm afraid this may be a legitimate situation.  Assume a driver
> >    is signed for, say, 2k8 and 2k8R2, and then a newer version is built
> >    and signed for 2k8R2 and 2k12.  The script will make arbitrary choice
> >    of the driver for 2k8R2.  Any suggestion on what to do with this?
> > 
> 

In this case, I would go with "2k8R2 and 2k12" driver for 2k8R2.
For me "2k8 and 2k8R2" means that the driver was initially designed
and tested to be running on 2k8 platform, but later, at some point,
after successful testing on 2k8R2, the build script was changed to sign
the same binary for 2k8R2 platform as well.
While "2k8R2 and 2k12" was initially designed and built to be running on
2k8R2 platform.

Vadim.

> Hmm. This is a bit weird. Probably some combination of:
> 
> 1) plain old packaging error. some wires got crossed and we never updated the
> xp driver to share with 2k3 or something similar
> 
> 2) qe/supportability concerns. maybe the newer 2k3 build was done to fix a 2k3
> specific driver, so we only updated 2k3 in the tree to avoid having to re-QE
> the xp driver as well. or something in that ballpark.
> 
> Maybe vadim knows more, CCd
> If #2 is involved I'd rather we work around that some other way: either re-QE
> the driver, or only WHQL it for the windows version we care about (not sure if
> that's possible). Having to track this statically is a pain
> 
> As for figuring it out in your script, does the .cat file have a date in it?
> For the WHQL bits maybe we can just choose the latest WHQL'd driver.
> 
> For the non-WHQL bits, do the driver files end up interchangable? Or is the xp
> driver 'xp only' but the 2k3 driver is '2k3 + xp' ?
> 
> - Cole