[virt-tools-list] setup.py and packaging for distribution

[Gene Czarcinski]

On 03/26/2013 05:48 PM, Cole Robinson wrote:
> On 03/26/2013 11:59 AM, Gene Czarcinski wrote:
>> OK, I have been looking into how packaging and distribution works for example
>> of when you want to get a snapshot of your development.
>>
>> 1. python ./setup.py build seems to work
>> 2. python ./setup.py rpm fails looking for some patch file (?) but does create
>> a tarball in ./dist/
> 'python setup.py rpm' works for me... can you show the exact error?
I found the problem.

I have my own rpmbuild tree under my home directory and a tailored 
.rpmacros.  When I temporarily renamed ".rpmmacros", the "python 
setup.py rpm" worked.

I consider this a but with setup.py.

I was also surprised to find that setup.py had created its own rpmbuild 
tree under my home directory.  This was a very unexpected and 
undesirable side effect! ... another "bug" in my opinion!

That said, I really have no need or want to have the "noarch binary" 
rpms produced directly from the git clone repository.  What I really 
what is a tarball with a valid spec file (that works with rpmbuild 
-ts).  There may exist others but I have yet to encounter a project 
(with a git repository) which creates the binary rpms directly from the 
git.  I have my own rpmbuild tree and mock to create the binary rpms and 
I suspect that this is the case with most others.

More below.
>
>> 3. python ./setup.py sdist works and creates a tarball in ./dist/
>> 4. in both of the above cases the tarball is completely unsatisfactory!  They
>> contain everything in the directory.
>>
>> Suggestions:
>>
>> 1. Use "git archive" since it will only gather files that branch currently
>> manages
>>
>> 2. This is a snapshot.  The tarball name should be something like
>> virt-manager-git20130326.tar.gz
>>
>> 3.  Add more tags.  I know you have the RELEASE-0.9.4-1, etc. tags but you
>> need some additional one for things such as release-candidates,
>> testing-candidates, etc.  For example, maybe the current gtk3.2 branch should
>> have a tag something like gtk3.2-t1 [test 1].
>>
>> Why the above suggestions:
>> 1. From the git-scribe man page:
>>>         The command finds the most recent tag that is reachable from a commit.
>>>         If the tag points to the commit, then only the tag is shown. Otherwise,
>>>         it suffixes the tag name with the number of additional commits on top
>>>         of the tagged object and the abbreviated object name of the most recent
>>>         commit.
>> 2. Use this tag-info to set the virt-manager "version"
>>
>> 3. Bring back a bit of the v-m.spec.in file.  Set the Version as the "date"
>> from one specified on the tarball using "sed".  Then you can add the tag-info
>> as the "release" or "extra-release" using "sed". The output of "sed" should be
>> the .spec file which is NOT managed by git but the spec.in is.
>>
>> 4. When you run "git archive", create the tarball (.tar file) but not
>> compressed (.gz).  Now run "tar -rf" to add the spec file into the tarball.
>> Now you can run gzip to compress the tarball.
>>
>> 5. Skip creating rpms ... source or "binary".  Create a mechanism to produce a
>> snapshot tarball.  To get an rpm, rpmbuild -ts works just fine.
>>
>> I have implemented something like the above as a bash script for another
>> project.  If you want, I will take a shot at implementing this as part of
>> setup.py ... just point me where to look and where to change things for the
>> virt-manager --version.
>>
> I started replying to things individually here but I got confused: what's the
> end goal exactly? 'python setup.py rpm' generates a uniquely named rpm based
> on the current commit? Who does this benefit?
>
> If it's beneficial, I'd rather just make it a new subcommand like 'python
> setup.py gitrpm'. You could likely overwrite the RPM version values with
> rpmbuild options, rather than revive virt-manager.spec.in mangling which I'd
> like to avoid if possible.
>
> But really if this is about using RPMs as part of the development process, I'd
> rather spend effort on making sure running virt-manager from the source tree
> works.
>
> Thanks,
> Cole
>
>
OK, let me try again to explain my position.

1. During development in a git-clone repository I will generally have 
some non-git-managed files such as patch files or directories with patch 
files.  Unfortunately, "python setup.py sdist" includes all of this in 
the tarball it creates.  This is why I am suggesting the use of 
git-archive instead of tar to create the archive image. The resulting 
tarball only includes the "offical" git-managed files which is what I 
believe should be in a distribution (snapshoot or release) tarball.  
Yes, ".gitignore" is included but that is a nit. This tarball should be 
sufficient to be used as a source for "rpmbuild -ts" or, when expanded, 
to build and install the product. If I really wanted a tarball which 
included everything, I can easily create that.

2. The current sdist/rpm tarball uses "0.9.4-1" to version the tarball 
and rpms.  This is useless since this is this same "version" will be 
used for tarballs created in the master branch or the gtk3.2 branch.

3. Alternative version: Use something like "git20130327" for the 
version.  Better, at lease I can differentiate between a tarball 
produced last week and one produced today.  Better, but not really a winner.

4. A really good alternative is to use "git describe --tags" produce the 
version.  Two examples:

       $ git checkout -b gc-0.9.4-1 RELEASE-0.9.4-1
       $ git describe --tags
       RELEASE-0.9.4-1

      $ git checkout gtk3.2
      $ git describe --tags
     RELEASE-0.9.4-1-86-gc61d9af

You can see that you get unique identifiers.  If this is used as the 
"version", then it will not matter when you create the tarball, there 
will be a different version if the contents of the tarball is different 
but the same version if they are the same.  I like this!

Of course, before doing the archive (creating the tarball), you must 
make sure there are no uncommitted changes.

5.  So, now I have produced a good tarball with a version based on 
content (if the content is different, the the version will be 
different). But there are two other things that need to use this new 
version: the value given when the user enters "virt-manager --version" 
and the spec file.

6.  I am not sure where/what to change but the effect result is to use 
the value returned by "virt-manager --version" to be  the value from 
"git describe --tags".  The basic place is line 44 in cliconfig.py but 
the is a cli.cfg referenced and that is likely where this should go.  I 
am just not sure where is is or when this cfg file is created.


7. For the spec file, I want to slightly revert to having a spec.in file 
which will be managed by git and a spec file which will not be managed 
by git.  Then setup.py needs to do "sed" to replace /@VERSION@/ with the 
result of "git describe --tags".  I also suggest that the release be set 
to "0" since most of these will be snapshots.  Once the updated spec 
file is created, it can be added to the tarball (tar -rf) and the new 
tarball compressed with gzip.

Have I explained this any better?  I hope so.

BTW, there is still some clutter left with the remove of autotools which 
should also be deleted (e.g., autobuild.sh).  Do you want me to create 
patches for that or is that already being handled?

Over to you folks.

Gene