[Zope-dev] the Zope Framework project

[Martijn Faassen]

Hey there,

Chris McDonough wrote:

> 1)  I'm not in favor of a single steering group for the *entirety* of all Zope
> software.   We've tried a similar thing in the past (via the foundation
> structure); it didn't work and I'm not sure how we'd expect things to turn out
> any differently this time.  Instead, perhaps the focus of groups should be on
> some much smaller subset of Zope-related software (e.g. the
> zope.interface+zope.component group, the zope.schema group, the ZODB group,
> etc).  Could we consider this?

I don't recall we actually did anything within the Foundation structure. 
The structure was there in the bylaws but was never applied, so I don't 
think that can be qualified as a failure. :)

The steering group isn't intended to take a responsibility for the 
entirety of the Zope software. Zope 2, Grok and the Zope 3 app server 
(which would be a distinct entity) would manage themselves and the Zope 
Framework steering group would not have a say over the libraries and 
configuration they add.

I do think the steering group should start worrying about a larger 
amount of the libraries rather than a small set. Part of the reason is 
exactly so we *can* identify subsets and properly delegate their 
management. Delegating their management will require some kind of 
agreement on groups coordinating however. We need to have an overview of 
the whole in order to know how to evolve the parts in many cases. For 
instance, if we move some class due to a dependency cleanup, we need to 
have an effort to update the libraries in the whole to use the new 
imports relations.

I do also think that we should go to a smaller set of libraries. We 
currently share a huge amount of code that only one consumer actually 
uses. For instance, I think all of the zope.app.* packages which contain 
ZMI code can eventually be left to the management of the Zope 3 
developers, meaning they'd not be part of the Zope Framework anymore.

The ZODB is a good example where I'm not sure whether the ZODB should be 
considered part of the Zope Framework at all. I think we should see this 
as an external library that the framework builds on.

The idea that there is a bunch of people who take the responsibility for 
managing the whole doesn't mean they should be obstructing moves to 
improve the parts. Similarly I assume you're taking some form of 
responsibility for "Repoze" and that this is helpful the evolution of 
the parts of it.

My hope is that we'll see more of a catalyst function than anything 
else. I think the answer to proposed changes in the lower layers (which 
is always risky) should be to point out potential risks and say: we can 
make this change, but we also need to make sure we do X. The answer 
should typically not be "no". If it is a "no", we should actively try to 
identify the use cases driving the proposed change and look for 
alternative solutions.

> 2) I'm also not in favor of a giant lockstep set of software versions shared
> between notional releases Zope 3.5, Grok, and Zope 2.12.  I can only see this as
> continuing our mistakes of old by trying to treat some collection of software as
> "Zope" as opposed to letting parts of it survive or die on their own based on
> merit; it'd be more effective to just let each framework use (or disuse!)
> whatever versions of stuff that work best for it.  That's why the software is
> broken out into individual components in the first place; we should encourage
> diversity in component usage.  Instead of trying to legislate and bless some set
> of components as a "version", we should just work to make each piece better and
> worthwhile to use independently; it's value would be in its actual usefulness
> rather than some belief that it works well with the other components in the
> "version".  

I don't understand why you think a list of versions that has release 
numbers that we know works together is a blocker for independent 
evolution for the individual libraries.

I think there are two parallel efforts:

* evolving libraries standalone. We should improve those libraries so we 
can think about them standalone. If a library contains ZMI code, it's 
harder to think about it standalone for instance.

* making sure that there is a list of library versions that people can 
install that isn't broken. That is, run the tests for these libraries in 
isolation and together. If a decision was made to change one library, 
make sure that all the other libraries in the list are updated so they 
actually still work.

I see both efforts as necessary. If you just care about a smaller list 
of libraries, you don't have to worry about a larger list of course, 
though you will have to coordinate with some people who do. You probably 
can do quite well constructing your own list as it's a much smaller one. 
That's fine, and nothing should stop you. But the reality is that many 
people in this group *do* care about a larger list of libraries.

> Could we at least agree that lockstep versioning of a huge set of
> Zope eggs to be shared across many frameworks is not optimal for the long term
> and that it would be better if each framework could pick and choose whatever
> components and versions it actually needed?  Could we also agree that this would
> tend to result in better dependency partitioning ("X depends on Y, I don't need
> Y, I just need X, let's fix that")?

I think the individual frameworks should have the last say on which 
versions they're going to use. That's in the document I wrote:

"""
As a service to the users of the Zope Framework, the Zope developers
also make available lists of version numbers of core libraries that
have been tested to work together as a "Known Good Set". This receives
a version number and is the Zope Framework release.  Users of the Zope
framework can use this list, but may also diverge from it where they
see fit. Other projects (such as the Zope 3 application server and the
Grok project) also have a Known Good Sets that expand on the Zope
framework list (and may diverge from it). Each of these consumer
projects can of course have their own release process, schedule and
version numbering.
"""

It's a service that people do not have to partake in. The Zope Framework 
Steering Group has been explicitly defined as having *no* say over the 
projects directly.

At the same time, I can't agree with the extreme position. I'd prefer 
the frameworks to use mostly the same set of versions, otherwise 
exchanging libraries between them becomes much harder. I don't see the 
point in making 3 groups do the same work while we could come together 
and do it in 1 place. That's what we have this community for, and we 
shouldn't *deny* this community.

I'm very much against us just providing a huge toolbox of random 
libraries and offer no guidance whatsoever about which ones work 
together, or which ones are deprecated, etc. And it's really not true 
you can just pick an arbitrary list of libraries and have them work. 
It's quite a relief to the Grok project we don't need to puzzle out 
which libraries work together ourselves all the time - creating a more 
or less sensible initial list in 2007 cost us quite a bit of time and 
we'd like to share the effort with the wider community.

I also don't believe it's workable to just focus on a single component 
while ignoring all the others that build on it. If you make an 
improvement in one that breaks an API (to clean it up, explicitly 
breaking backwards compatibility), someone will have to update all the 
libraries that use the API. There will at least need to be coordination 
and communication. Someone needs to take responsibility for making sure 
that happens. Sometimes also operations need to take place on a whole 
bunch of libraries at the same time in order to make progress on all of 
them. This is not theory, it's what we needed to do for the dependency 
refactoring project we had a month ago. That's a project with the goal 
to create *more* stand alone libraries.

So while I don't want to get in the way of your use cases, you seem to 
imply there there need be no coordination and cooperation on issues that 
cross the barrier of a single library or a small set of libraries. I 
think we should look at libraries on their own merit, but at the same 
time see them as part of *some* responsible community that takes care of 
them.

I do see that community as useful on the long term.

What I'm trying to do is draw the line around an area outside of which 
we stop caring, at least where the Zope Framework Steering Group stops 
caring. It's a smaller area than the vague line we have now (Zope 3 the 
app server/framework/what is it?). If people want to make sure other 
libraries outside that line aren't broken, they should read the upgrade 
notes and change their code. I want this area to include fewer rather 
than more libraries, but in order to get there we have to care about a 
lot of them to start with.

It may very well be true that in some time we'll develop clusters of 
libraries that can be more or less managed on their own. The ZMI is such 
a cluster that I hope will eventually emerge (and that the Zope 
Framework Steering Group doesn't care about directly). That'd reduce the 
coordination overhead. But sometimes we do need to take coordinated 
action, and I don't see that need disappearing entirely.

Regards,

Martijn