[Grok-dev] Autoincluding ZCML for package dependencies

[Jeff Shell]

On Jan 21, 2008 6:24 AM, Ethan Jucovy <ejucovy at openplans.org> wrote:
> Hey Grok community,
>
> My colleague Robert Marianski and I are at the Snow Sprint and we've
> been having a lot of fun looking at Grok.  We've been working with
> Martijn for the past couple of days on a package to automatically
> include ZCML for a project's required dependencies (by looking at
> `install_requires`) to eliminate the need for redundant "<include
> package='...'>"s.  I've just written a blog post about this code and
> our work on it which I thought some of you might be interested in
> (here: http://www.openplans.org/projects/opencore/blog/2008/01/21/automatic-inclusion-of-zcml/
> ).  (I also wrote a shorter post just describing my impressions of
> Grok in general here:
> http://www.openplans.org/projects/opencore/blog/2008/01/21/grok-is-awesome/
> -- but the title says it all: "Grok is Awesome.")
>
> We've got pretty good test coverage and a passing test suite, though
> we haven't yet quite managed to confirm that it works in a "live"
> environment, so we're working on doing that and finishing up the code.
>  If you'd like to take a look, it's currently hosted here:
> https://svn.openplans.org/svn/snowsprint/z3c.autoinclude/trunk/

Sounds like a good (or decent enough) idea. You're basing what to
(potentially) import off of the requirements in setup.py, yes? Do
those stay in the order specified in setup.py? My quick glances at
some eggs seem to indicate so.

Tres raised the alarm of Zope 2's DWIM hell, which is the first thing
I though of as well. One problem with Zope 2's way of
auto-importing-and-registering-things-in-Products is that it was just
alphabetical (probably based on listing the directories in the file
system). This presented problems if you were trying to register
something with, say, Formulator, and your package name started with B.
The behaviour would be different than if your package name started
with J.

Having some kind of explicit control over that order outside of the
more chaotic order of Python imports is the number one killer-awesome
feature of ZCML. But if that order can generally be
controlled/inferred by the Egg's requirements, then I say - awesome.

Ack. I see that you're wrapping the calls in sorted()::

    for dotted_name in sorted(info.get('meta.zcml', [])):

Do you have any particular reason for doing that?

Also - I would beg for some log.debug statements::

    def debug_includes(dist, include_type, dotted_names):
        if not dotted_names:
            return
        log.debug('%s - autoinclude %s: %r', dist.project_name,
include_type, list(dotted_names))

    (inside the auto-include-directives)

    meta_zcml = list(info.get('meta.zcml', []))
    debug_includes(dist, 'meta.zcml', meta_zcml)
    for dotted_name in meta_zcml:
        ...

Sometimes these things go a little bit screwy, based on whatever local
setups/configurations happen to be in place, and it helps to know the
decisions that the software made. Trust me. I've had software try to
do what I mean for years, and for some reason I must mean for it to
piss me off because it can do that astoundingly well.

And/or - the docs for autoinclude should include some other tips such
as how to debug or inspect the choices made via 'debugzope'. Just some
function that wrapped getting the distribution and returning the
IncludeFinder results for a particular package/distro. Hell - such a
function might be nice so that one could use z3c.autoinclude to
generate a static list of includes if they want to take over explicit
control; which could also be usable in buildout to generate a master
site.zcml for production that is something a bit more physical which
system administrators can see and tweak if necessary.

I'll explain more in another message, but I think it's very important
that automagic tools provide a way of seeing into the magic. It
doesn't take much to add some helper functions and/or logging
statements, and I think that part of the Grok philosophy should
include (if it doesn't already) helping people who are stumped by some
unexpected (by the user) behavior.

-- 
Jeff Shell