[Zope-dev] "ZTK" futures: one big package?

[Chris McDonough]

A couple of things:

- If it helps to think of this thing as not-the-ZTK, that's fine.  I don't care 
what it's called.  Maybe "the ZTK" is another package that depends on this big 
thing plus all the broken out bits.

- There's very little work to doing this.  You make an SVN directory that has 
externals pointing at each of these packages, write a setup.py that includes 
them all, and ship it.

- I'm not proposing that we "destroy" existing dependency info.  I'm just saying 
that sometimes it doesn't matter.  Some of the bits we're trying to make useful 
independendently don't make sense to make independent of each other: 
zope.app.principalannotation, zope.app.dublincore, zope.app.ftp, zope.site, 
zope.copypastemove, and so forth.  It seems to me very silly to continue 
pretending that these could ever have a meaningful life of their own.

- There's nothing precious about the current packaging structure *except* for 
backwards compatibility.  The original egg-ification of Zope 3 was a largely 
mechanical procedure.  I hope you'd agree that if we had it all to do over 
again, there would be far fewer setuptools distributions; each would provide 
some bit of logical functionality instead of happening to be a directory in SVN.

- I don't disagree that this large distribution has to *depend* on 
zope.component/zope.interface etc.  It's just in a structure where the large 
distribution becomes a set of packages that can't have their dependencies teased 
apart (distributed as a sngle setuptools distribution), it falls out that it's 
"not part of the large distribution", just a dependency of it.

I think at some point it's best to just declare defeat, realize that the current 
packaging structure and distribution strategy is very artificial, realize that 
the granularity that the 100+ egg strategy gets for us is way, way too granular, 
and package up the bits that really don't *need* to have a life of their own 
into some large setuptools distribution.

I don't really care if that thing is called "the ZTK", just that it exists.  If 
later it turns out that some piece of functionality in that distribution *is* 
useful outside, we take it out, make sure it doesn't depend on anything in the 
large distribution, and make the large distribution depend on it.

Another thing is this: even if we're successful in teasing out dependency info 
so we do have a collection of truly independently useful things, after it's all 
over, we're going to get to a point one way or another where we make a big 
package of stuff that just can't have its dependencies teased apart because it 
*really is just one thing*.  Why *not* just do it now?

- C

On 5/11/09 11:11 AM, Martijn Faassen wrote:
> Hey,
>
> Chris McDonough wrote:
>
>   >  Instead, I have argued for promoting packages that have some life of
>   >  their own (independent of the rest of the pile) into subprojects that
>   >  have their own release cycles.
>
>   >  Then outside projects such as Plone and Grok could depend on
>   >  independent versions of such packages, giving them slightly more
>   >  flexibility than requiring a "version of the ZTK".
>
> We already have that flexibility today. To me, the utility of a release
> of version numbers in the ZTK does not at all exclude the potential to
> evolve the packages to more independent sub-projects.
>
>> Given that this suggestion has been met with skepticism, let me try another
>> tact.
>
> I think that's an inaccurate description of the response you got. I'm
> quite positive about trying to give as many packages as possible a life
> of their own. I don't think you got anyone else arguing against this
> point of view.
>
> I'm also quite positive that some packages are:
>
> * useful as independently distributed packages
>
> * only make sense in a Zope 3 or a Grok or a Zope 2 context, i.e. they
> depend on a significant set of Zope packages.
>
> I'd like to get out of this paradigm:
>
> * the Zope packages are independent sub-projects.
>
> * therefore we cannot distribute a list of versions that work best together.
>
> And this one:
>
> * if we distribute a KGS of anything
>
> * packages in that anything aren't independently reusable automatically
> and should be merged into a ball.
>
> I'd also like to get out of the following paradigm:
>
> * the Zope packages are not independently reusable yet
>
> * therefore we should distribute them all together
>
> We're in a grey area. Some package are here, some packages are there,
> some are in between. Some packages build on other packages, but have
> clear dependency structures. Some don't have clear dependency
> structures. Some have better documentation and better focus than others.
>
> If there is to be a merging of code together, then I propose we continue
> the project where the ZMI code is merged into one or a few packages. We
> can also investigate merging 2 or 3 packages together where it seems to
> make sense, or simply moving code between packages (some code needs to
> go down the dependency list, some up).
>
>> Instead of thinking about it this way, could we think about it as
>> *deflating* the current set of zope.* packages that do not currently have a
>> meaningful life of their own into a single setuptools distribution named "ZTK".
>>    This package would include most everything in zope.app*, plus things like
>> zope.server, zope.publisher, and other things that just aren't useful outside of
>> Zope-the-appserver, or which currently just depend on "too much".
>
> -1
>
> This would make it a lot harder to:
>
> * clean up dependency relationships with the goal of creating more
> reusable code. We'd all hide them in a sumo ball again.
>
> * get rid of bits we *don't want anymore*. If I need anything in a sumo
> package I'd need *all* of it.
>
> * override individual packages with newer versions
>
> * we've done a lot of refactoring recently trying to separate the UI
> from packages. This is done by creating a *new* package, leaving the old
> package behind. We can do this, test this and release this
> package-by-package now.
>
>> Over time, we'd tease out the dependencies of packages that live in the ZTK
>> distribution, making them useful outside without any dependency on the ZTK.  The
>> names of these packages could be arbitrary (they wouldn't need to retain their
>> old "zope.*" names).  Some backwards dependency shims would be added to the ZTK
>> to prevent old imports from breaking, and the ZTK distribution would then just
>> have a dependency on the thing that got broken out.
>
> I don't like the attempt to redefine what the ZTK means to a giant ball
> of Python packages. That's implying that, say, zope.component is *not*
> in the ZTK. That's wrong.
>
> Why generate a whole lot of work for ourselves getting from where we are
> now to here? We've learned how to work with the current situation in
> 2007 already.
>
>> I'm thinking that this would simplify things greatly for people who want to be
>> consumers of "truly independent" Zope packages.  There'd be exactly N of them
>> available for download (where N is much less than 100, more like 20), plus the
>> ZTK, which would have the rest of the pile in it over time.
>
> I don't see why a big package would "simplify things greatly" for you or
> anyone else.
>
>> If someone wanted
>> to use a forked version of a package that lived in the ZTK distribution, they'd
>> either do so by teasing out the dependencies and making it "truly independent"
>> or they'd just reroll a custom version of the entire ZTK distribution.
>
> And that's easier than the current situation how? Are you really
> proposing we destroy the dependency information we've already teased out
> and then make people do the work again?
>
>> Does this make any sense?
>
> Not a lot in my book.
>
> I think an important reason why there's so much awareness of dependency
> issues in the Zope world now (and effort spent on it) is precisely
> because we released our separate packages and can see the dependency
> information clearly.
>
> Regards,
>
> Martijn
>
> _______________________________________________
> Zope-Dev maillist  -  Zope-Dev at zope.org
> http://mail.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists -
>   http://mail.zope.org/mailman/listinfo/zope-announce
>   http://mail.zope.org/mailman/listinfo/zope )
>