[Zope-dev] Egg install bot results

[Chris McDonough]

I've created a variant of the script I used to generate the results  
referenced in my original message within this thread.

http://svn.repoze.org/playground/trunk/chris/zopesvnchecker/releasechecker.py

The new variant attempts to "easy_install" each cheeseshop project  
that has 'zope' in its author metadata (found via the cheeseshop's  
search XML-RPC interface) into a "pristine" virtualenv.    This mimics  
what end-users would see if they attempted to install Zope-related  
packages from the cheeseshop without using zc.buildout.

I'll set this up to run every few days.  I'll attempt to send its  
output to zope-coders.

- C


On Nov 13, 2007, at 4:29 PM, Chris McDonough wrote:

> I've created a bot process (see
> http://svn.repoze.org/playground/trunk/chris/zopesvnchecker/ 
> checker.py)
> that:
>
> - checks out the "trunk" subdir of each top-level directory in
>   svn.zope.org
>
> - if the resulting directory does not have a "setup.py", we skip the
>   directory
>
> - if the resulting directory does have a setup.py, we create a
>   temporary virtualenv with setuptools installed and run the
>   virtualenv's bin/python passing it "setup.py install".
>
> - We capture the results of the install, and print a summary.
>
> The "summary" results are at
> http://www.plope.com/static/misc/checker_results.txt .  The "details"
> pickle generated by the script is at
> http://www.plope.com/static/misc/results.pickle .
>
> My analysis of the results are these:
>
>  - A version conflict exists in a low-level dependency between a
>    requirement for zope.traversing 3.4.0 and a requirement for
>    zope.traversing>=3.5.0a1.dev-r78730 when installing many zope.*
>    eggs using "setup.py install" (and easy_install), making it
>    impractical for non-Zope people to actually install most of the
>    interesting and released zope.* modules.  This conflict needs to
>    get fixed.  Additionally, some modules (like zope.pagetemplate)
>    should not fail with this dependency error in the first place;
>    instead their dependencies need to become less conservative.
>
>  - The only 10 packages (out of 80) in the zope.app namespace can be
>    easy_installed.  All the others fail with the above dependency
>    error.
>
> Suggestions:
>
>  - Find and fix the zope.traversing easy_install conflict.  I'll try
>    to debug this.
>
>  - Institute a policy that all distributions that are released to the
>    cheeseshop should be installable via "easy_install".  IMO, if they
>    are not installable this way, they should not be released to the
>    cheeseshop, given the larger Python community's expectations.
>
>  - Figure out why buildout can (apparently) qsuccesfully install
>    dependencies of currently failing zope.* eggs while easy_install
>    can't.  I probably won't be able to do this.
>
>  - An automated test should ensure that easy_install does something
>    reasonable with the latest release of each distribution.  If the
>    test fails, a new release should be made, or the dependency bug
>    tracked down and fixed.  My checker script could be made to do
>    this, and I'd provide the work if someone told me which
>    distributions to test, and how to maintain the list of
>    distributions to test over time.
>
>  - Ditch the idea of releasing separate distributions for each
>    package in zope.app.  The individual eggs typically can't be used
>    outside of a zope appserver installation (and if they can, they
>    probably shouldn't be in "zope.app", they should be in "zope" or
>    they should be their own top-level package), and the
>    "namespaciness" of zope.app is suspect when it's unlikely that
>    anyone who is not a Zope committer will release a distribution
>    which makes use of that namespace package.  Their current
>    overgranularity makes distributing them as separate eggs and
>    releasing them to the cheeseshop a form of "cheeseshop pollution",
>    especially given that so few of them can actually currently be
>    installed using easy_install or "setup.py install".  If cheeseshop
>    is going to continue to be used as the index, I'd suggest creating
>    a zope.app top-level svn module with a single setup.py in
>    containing all the packages that are meant to go into zope.app.
>    Version the resulting "zope.app" distribution as necessary instead
>    of versioning many more granular "zope.app.*" distros.  It's OK if
>    some people don't use some of the functionality in the resulting
>    egg, just toss everything in.  There is precedent here in the
>    Paste distribution.  It has many submodules and does many things,
>    but it comes in the form of a single egg.  Yes, you lose the
>    ability to make a bugfix in one subpackage and release it, but
>    IIRC the intent is to trim zope.app down anyway, pushing
>    libraryish things out to top-level or zope.* packages.
>
> Issues:
>
> - Who's in charge?  Whomever you might be, to what extent do you
>   agree/disagree with the above suggestions?  If you agree with any,
>   how can I help fix things?
>
> - I'm unsure how anybody is able to install Zope 3 right now using
>   eggs, unless there's some fundamental difference in the way
>   easy_install resolves dependencies vs. buildout.  I have not looked
>   at how Stephan's KGS works yet, though, so I'm sure I'm just
>   missing some magic.
>
> - We should consider fixing setuptools install to detect conflicts
>   before attempting to install anything.  The current regime of "find
>   conflicts halfway through an install" is IMO untenable in the long
>   term for using eggs as a distribution mechanism.  This may mean a
>   very invasive change to both the package index and the client
>   software, though.
>
> _______________________________________________
> Zope-Dev maillist  -  Zope-Dev at zope.org
> http://mail.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
> http://mail.zope.org/mailman/listinfo/zope )
>