[Zope3-dev] Re: Autoconfiguring a site.zcml

[Jim Fulton]

On Apr 22, 2007, at 10:32 AM, David Pratt wrote:
...
> In looking at the way I am developing, my goals are package reuse  
> and thin glue in an app package (that is also the app egg) to bind  
> packages to make an application consisting mostly of installation,  
> security, testing and perhaps skinning. That said, I do not really  
> want or care about organizing packages neatly under a single src  
> folder to pull them together into a larger egg. To keep it simple,  
> I just want to list the eggs I use in app part of my buildout.

I wonder what you are trying to contrast here.  Your goal of being  
able to just list eggs sounds reasonable.  Who would disagree with it?

...

> You are penalized for this approach in app buildouts since more  
> small independent eggs (holding some generic functionality) with  
> their own dependencies equals more checks.

What checks are you referring to?

> Being explicit like this

Like what?  I don't understand what the "this" is that you are  
referring to.

> increases the amount of time for the buildout to run, particularly  
> if there are plenty of eggs with many dependencies. Think in  
> numbers like 150 - 200+ eggs with zope and zope app - no exaggeration.

Are you referring to Buildout's checking for new egg versions?
Judicious use of the -N option can cut down on this a lot.

...

> An approach that that leads to shorter app buildouts is to group  
> packages under the main app setup.py. This is currently what fits  
> with the development approach spelled out in the app recipe docs  
> but it has downsides also.

I don't understand what you are referring to. Which app recipe are  
you referring to?

> This is nice for speed in running a buildout but don't like the  
> fact that the main setup has got to capture dependencies for the  
> collection as a whole because you just a bigger egg with multiple  
> packages where you still need to track down dependencies. True,  
> some of these may be packages from external eggs you need to list,  
> but probably more are packages that you have assembled within your  
> app (that may be desirable to re-use) but where you have not been  
> being explicit about the dependencies (since these are determined  
> in an egg not a package).

Could you give some examples of what you're talking about.  I'm  
trying to follow you but can't.  You seem to be discussing whether  
dependencies should be listed in a setup file and suggesting that it  
somehow affects buildout performance.  buildout performance isn't  
affected by *where* dependencies are defined.  If your package has  
dependencies, then list them.  This is othogonal to performance.

> Personally, I don't want to take the time to try to pull the  
> dependencies together in an app that is a collection of packages  
> like this - since it is a chore not worthy of a human.A machine can  
> do this better than I can (and it is welcome to the task!). Also,  
> apps are prone to change - each change equals more time re- 
> examining this over this again and again amidst a constantly  
> changing landscape of package revisions - yuk.

Are you referring to Python dependencies or ZCML dependencies?

If you are the author of a package, you should know what other  
packages you're using.  You should know when this changes.  In  
theory, determining these dependencies could be automated.  No one  
has automated it yet.


> When you add that as a zope3/python developer you are mostly  
> working at a package level, you realize that spelling dependencies  
> with more granularity in an egg is much more attractive and just  
> easier overall than figuring this out for an app with a cluster of  
> packages afterwards (on top of how this all goes together with  
> other external/namespace packages like zope, zope.app, z3c, zc etc,  
> etc).

I wish I knew what you were trying to say here.

Are you saying that if A depends on B and B depends on C and A  
doesn't use C directly that A should not have to list C as a  
dependency?  If so, I don't think anyone would disagree with you.

> I feel the same way about the site.zcml. In the initial docs on the  
> app recipe,

What app recipe. Please be specific.

> a smaller site.zcml is recommended since you can tuck most of the  
> meat in your app configuration. In experimenting, the bottom line  
> is it doesn't really matter where it is - because is really the  
> same size regardless of where the pieces reside.  So I have been  
> considering - what am I really trying to do here - and how much  
> grief do I want to encumber putting it all together. The semantics  
> of having a large site.zcml that takes care of app configuration is  
> not inconsistent with the fact that I am using the buildout to  
> configure and build my app. So in the end I would be happy to  
> consider a larger site.zcml that does the work of organizing  
> includes to all packages in the app. And at the package level being  
> left with just making the includes within it (such as including a  
> browser or an ftest sub package) to complete the configuration tree.

Are you saying that If A depends on B and B depends on C that you  
expect A to only have to include B's ZCML and rely on B's ZCML to  
include C's ZCML?  I think that's a reasonable strategy.

(It would be more reasonable if we filed some holes in ZCML  
overriding, but that's a different topic.)


> That said, I don't relish to time it will take to looking at  
> ordering all of the includes in a site.zcml.

You shouldn't have to.

...

> Overall, it should not matter really as far as I can see. If egg1  
> needs the contents of egg2, it stands to reason that that package  
> includes of egg2 need to be pushed higher in the site.zcml that  
> those of egg1. Further if we adhere to good information in the egg  
> we should not have to worry that imports of packages in eggs will  
> not work. So I tend to believe that some means of scoring,  
> weighting or boosting could be used together with the exploration  
> of each egg for a configuration (*-configure.zcml, *-meta.zcml) to  
> autogenerate the site.zcml as the buildout proceeds.

I don't think this is necessary.  An egg that has ZCML should load  
the ZCML from other eggs it depends on.  This means that these eggs  
have to be designed this way.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org