Hello, it is evident that there is no consensus on the list of packages that are part of the Zope Toolkit. As Gary suggested me, it looks like the concept of ZTK is different for each developer and it is more or less "the packages I use and I care about". We need a policy to define the ZTK in an explicit way, otherwise we will never get a *real* ZTK KGS that can be used to build applications and the whole concept of ZTK will always be fuzzy. Christian prepared a list of packages which is available here: http://docs.zope.org/zopetoolkit/about/packages.html I propose a (very) simple policy, which we can use to improve the current situation: * a package is part of the ZTK if the following criteria are met: - It has at least N zope developers (with commit rights) who explicitly expressed interest in maintaining the package (because they use it in their projects, for example); N > 1, at least; - All its dependencies (excluding testing dependencies) are part of the ZTK; * we have to ensure, using automated testing, that each package: - has no test failures; - does not introduce test failures in any of the other ZTK packages; * the ZTK KGS only includes the packages that are part of the ZTK; we will provide an extended KGS (which uses the ZTK KGS) with more packages if needed. I would like to add a new column to the table in the aforementioned page listing the developers who are interested in maintaining the package. Ideally, in a couple of weeks we will have a better overview of what does the ZTK mean to each of us. Ideas? Comments? -- Fabio Tranchitella http://www.kobold.it Free Software Developer and Consultant http://www.tranchitella.it _____________________________________________________________________ 1024D/7F961564, fpr 5465 6E69 E559 6466 BF3D 9F01 2BF8 EE2B 7F96 1564
On 8/6/09 11:35 , Fabio Tranchitella wrote:
I propose a (very) simple policy, which we can use to improve the current situation:
* a package is part of the ZTK if the following criteria are met:
- It has at least N zope developers (with commit rights) who explicitly expressed interest in maintaining the package (because they use it in their projects, for example); N> 1, at least;
- All its dependencies (excluding testing dependencies) are part of the ZTK;
* we have to ensure, using automated testing, that each package:
- has no test failures;
- does not introduce test failures in any of the other ZTK packages;
* the ZTK KGS only includes the packages that are part of the ZTK; we will provide an extended KGS (which uses the ZTK KGS) with more packages if needed.
How about another one: * the package has to be usable with all of Zope 2, grok and the Zope 3 application server That guarantees the stated goal that ZTK is reusable. Wichert.
* 2009-08-06 15:30, Wichert Akkerman wrote:
How about another one:
* the package has to be usable with all of Zope 2, grok and the Zope 3 application server
Yep, I agree. -- Fabio Tranchitella http://www.kobold.it Free Software Developer and Consultant http://www.tranchitella.it _____________________________________________________________________ 1024D/7F961564, fpr 5465 6E69 E559 6466 BF3D 9F01 2BF8 EE2B 7F96 1564
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Wichert Akkerman wrote:
On 8/6/09 11:35 , Fabio Tranchitella wrote:
I propose a (very) simple policy, which we can use to improve the current situation:
* a package is part of the ZTK if the following criteria are met:
- It has at least N zope developers (with commit rights) who explicitly expressed interest in maintaining the package (because they use it in their projects, for example); N> 1, at least;
- All its dependencies (excluding testing dependencies) are part of the ZTK;
* we have to ensure, using automated testing, that each package:
- has no test failures;
- does not introduce test failures in any of the other ZTK packages;
* the ZTK KGS only includes the packages that are part of the ZTK; we will provide an extended KGS (which uses the ZTK KGS) with more packages if needed.
How about another one:
* the package has to be usable with all of Zope 2, grok and the Zope 3 application server
That guarantees the stated goal that ZTK is reusable.
I would restrict it to the "big toolkit" set which is "used by" those three, rather than "usable with" them. Packages which aren't actually used by those frameworks are objectively less well supported, even if some developers use them for some apps built on top of the frameworks. The difference is that lots of people are going to *notice* breakage in my proposed set. There are smaller subsets of this group (e.g., my "bicycle-seat ZTK") which are also useful to document. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKevCY+gerLs4ltQ4RAo4uAJ9Sntv4GCbcmEAj9v1DrHfjFTQjoQCeJ+Zv 1eNI/JqpgVzeKWPSMLWPiZw= =Mtzz -----END PGP SIGNATURE-----
Hello, * 2009-08-06 17:03, Tres Seaver wrote:
How about another one:
* the package has to be usable with all of Zope 2, grok and the Zope 3 application server
That guarantees the stated goal that ZTK is reusable.
I would restrict it to the "big toolkit" set which is "used by" those three, rather than "usable with" them. Packages which aren't actually used by those frameworks are objectively less well supported, even if some developers use them for some apps built on top of the frameworks.
Am I correct saying that your idea is to restrict the ZTK to the packages defined as the intersection of the dependencies of zope2, zope3 and grok? ZTK = intersection ( zope2-dependencies , zope3-dependencies, grok ) I don't think this definition fits our needs, but my skills on gron and zope2 are too limited to bring counterexamples. -- Fabio Tranchitella http://www.kobold.it Free Software Developer and Consultant http://www.tranchitella.it _____________________________________________________________________ 1024D/7F961564, fpr 5465 6E69 E559 6466 BF3D 9F01 2BF8 EE2B 7F96 1564
2009/8/6 Fabio Tranchitella <kobold@kobold.it>:
Am I correct saying that your idea is to restrict the ZTK to the packages defined as the intersection of the dependencies of zope2, zope3 and grok?
ZTK = intersection ( zope2-dependencies , zope3-dependencies, grok )
That's my understanding of what Tres wrote.
I don't think this definition fits our needs, but my skills on gron and zope2 are too limited to bring counterexamples.
This, however, is more readily achievable, and provides a good foundation for each of the other projects mentioned to build from. This sounds like the right starting point to me. -Fred -- Fred L. Drake, Jr. <fdrake at gmail.com> "Chaos is the score upon which reality is written." --Henry Miller
On 8/6/09 17:44 , Fred Drake wrote:
2009/8/6 Fabio Tranchitella<kobold@kobold.it>:
Am I correct saying that your idea is to restrict the ZTK to the packages defined as the intersection of the dependencies of zope2, zope3 and grok?
ZTK = intersection ( zope2-dependencies , zope3-dependencies, grok )
That's my understanding of what Tres wrote.
I don't think this definition fits our needs, but my skills on gron and zope2 are too limited to bring counterexamples.
This, however, is more readily achievable, and provides a good foundation for each of the other projects mentioned to build from.
This sounds like the right starting point to me.
+1 Wichert.
I've created a policy draft as well as an initial list of packages on a wiki page, which I hope will help us to collaborate on the list: http://wiki.zope.org/zope3/ZTK I've put into the list the packages that I'd consider part of the ZTK and that I use in my applications. I don't know anything about grok and I doN't have a list of ZTK libraries used by zope2 or repoze. I'd like to ask help from everybody who cares about the ZTK to enhance the list adding the packages that they use, their zope.org username in the list of the developers for the package they are willing to maintain and the framework which are consumers of the libraries. IMHO, it is easier to get a real list if we start from an empty one and we add packages instead of starting from a huge list removing packages. Thanks, Fabio * 2009-08-06 23:56, Wichert Akkerman wrote:
This sounds like the right starting point to me.
Fabio Tranchitella wrote:
I've created a policy draft as well as an initial list of packages on a wiki page, which I hope will help us to collaborate on the list:
About the text on the wiki page:
Addition of new packages
A new package can be added to the ZTK in the following cases:
* it has been factored out from existing ZTK packages; * it provides new fundamental blocks or features that change the way the toolkit-based libraries are used.
I like this set of guidelines.
In order to add the package to the ZTK, all criteria defined by the policy must be met.
This line makes little sense to me. If I factor zope.foo out of zope.bar, I don't provide a new fundamental building block or feature that changes the way the libraries are used. zope.foo is still the in the ZTK.
In case consensus can't be reached among Zope developers, the Zope Steering Group has a final word about the addition of the package.
I'll happily reverse this, as the Zope Toolkit Steering Group by definition manages the Zope Toolkit. It always has the final word on the addition of a package, consensus or not. Of course another task of the steering group is to strive for consensus. I'd just say: "The Zope Toolkit Steering Group decides whether a package can be added to the Zope Toolkit." We can defer to the other document we already have about steering group decision making: http://docs.zope.org/zopetoolkit/steeringgroup/decisionmaking.html
Removal of packages
A package can be removed from the ZTK if all the following conditions are true:
I think 'all conditions' is too strict.
the package provides features that are not needed anymore because: * other packages provide compatible features;
compatible -> comparable?
* there are better alternatives. * no package in the ZTK depends on it.
I have more difficulty with this, as I like to start with a more inclusive list. Therefore, I want to remove packages that contain functionality, such as the ZMI implementation, and things like 'code in the ZODB' support that was experimental. I think we can remove packages by the following *guidelines*: * no other package depends on it. * there are better alternatives. * the feature the package defines is deprecated. That's loose, as I don't define how features get deprecated. But that's fine, as we're humans and we can make sensible decisions in specific cases.
I've put into the list the packages that I'd consider part of the ZTK and that I use in my applications. I don't know anything about grok and I doN't have a list of ZTK libraries used by zope2 or repoze.
Okay, I really think we should end this discussion now; it's just not productive. I think the consensus within the steering group trends towards inclusion (where Stephan wants to include way more than the others). So, the list of packages in the ZTK was prepared by Christian Theune and is here: http://docs.zope.org/zopetoolkit/about/packages.html Any other list (such as the one on the wiki page you created) is not the ZTK list. In addition, we are maintaining the documentation on the ZTK here: http://svn.zope.org/zopetoolkit/trunk/ A wiki might serve as a scratchpad for proposals and I'm fine with that, but let's make sure that: * the wiki page marks itself as a proposal clearly, otherwise it's a decoy. It's not the real documentation. * we integrate any such accepted proposals in the official documentation Another way to propose documentation would be to create a branch of the zopetoolkit documentation in SVN.
I'd like to ask help from everybody who cares about the ZTK to enhance the list adding the packages that they use, their zope.org username in the list of the developers for the package they are willing to maintain and the framework which are consumers of the libraries.
I really think this is the wrong way to go about it. In order to produce your list you'll need *everybody* to pay attention and do something. I don't think such general cooperation of everybody is generally successful.
IMHO, it is easier to get a real list if we start from an empty one and we add packages instead of starting from a huge list removing packages.
-1, and I consider this a Zope Framework Steering Group decision. Regards, Martijn
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Fred Drake wrote:
2009/8/6 Fabio Tranchitella <kobold@kobold.it>:
Am I correct saying that your idea is to restrict the ZTK to the packages defined as the intersection of the dependencies of zope2, zope3 and grok?
ZTK = intersection ( zope2-dependencies , zope3-dependencies, grok )
That's my understanding of what Tres wrote.
I'm not actually sure I even mean "zope3-dependencies": I don't know how maintained the thing known as the "Zope 3 Appserver" really is today. In particular, I would exclude any package in the zope.app.* namespace which is not also used by either grok or Zope2. So, my algorithm would be more like this: counts = {} for package_set in zope3, grok, zope2: for package in package_set: current = counts.setdefault(package, 0) counts[package] = current + 1 ztk = [x[0] for x in counts if x[1] > 1]
I don't think this definition fits our needs, but my skills on gron and zope2 are too limited to bring counterexamples.
This, however, is more readily achievable, and provides a good foundation for each of the other projects mentioned to build from.
This sounds like the right starting point to me.
The ZTK is supposed to be a *base* for the other platforms to use, and to represent the portions of the code which are not specific to any single platform. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKe5AH+gerLs4ltQ4RAqPAAKCvSZ9TAK5pz2VvrLi7YaNeTnTW/gCgvse0 6XX/lCMRRNXQXk0z0om3akg= =WcTJ -----END PGP SIGNATURE-----
Hey, Fabio Tranchitella wrote:
it is evident that there is no consensus on the list of packages that are part of the Zope Toolkit. As Gary suggested me, it looks like the concept of ZTK is different for each developer and it is more or less "the packages I use and I care about".
In a way the consensus should be reflected by the list you mention: http://docs.zope.org/zopetoolkit/about/packages.html
We need a policy to define the ZTK in an explicit way, otherwise we will never get a *real* ZTK KGS that can be used to build applications and the whole concept of ZTK will always be fuzzy.
We have a list. I propose that in order to stop long discussions about what should be in this list, we just start with what's in this list, by circular definition. :) We need to get a procedure in place to do compat tests of what's in that list, dependency graph guarding of what's in that list, and locking down a KGS for that list. I think that since we have a list doing all these things is only a matter of work - there's no fundamental questions we need answered before we can do this work. Once the base is there we can expand on it. I think what we need is a policy for adding packages into this list, and retiring packages from the list. Removal: I think an informal show of hands that asks "is this package important?" on the mailing list is useful there. The other is a situation that nothing depends on that package anymore. Once those two are reached, I think it'd be as simple as petitioning the steering group to have a package removed. Addition: this one is much tougher. New packages can get added if they're factored out of existing packages, that's easy. But fundamentally new package? We need to cross that bridge when we get to this. I suspect innovation for the time being will mostly be around the toolkit, not in it, or in the form of changes to existing packages. I think generally candidates for addition are packages that would change the way we arrange toolkit-based libraries themselves - I recall Shane's WSGI discussions as an example. I realize that to build "real" apps everybody will come up with a list of extra packages beyond the ZTK that they feel are needed too. Let a thousand flowers bloom I'd say - we just need a clean, fertile soil that we need to maintain. We already got plants growing in the soil anyway (Zope 2, Grok, bfg, and lots of Zope 3 apps). And of course there's some philosophy behind what's in the list now: it's a set of libraries shared by Zope 2, Zope 3 (whatever that is) and Grok. That's not the only answer and it's not quite the correct answer even, but philosophers spend way more time trying to pin down far more important concepts than the nature of the ZTK. Reality, mind, knowledge, good and evil are some examples. They haven't agreed yet either and they've had thousands of years to argue. In reality we still find those concepts useful. So let it be with the ZTK. Regards, Martijn
On Thu, Aug 6, 2009 at 1:05 PM, Martijn Faassen<faassen@startifact.com> wrote:
Hey,
Fabio Tranchitella wrote:
it is evident that there is no consensus on the list of packages that are part of the Zope Toolkit. As Gary suggested me, it looks like the concept of ZTK is different for each developer and it is more or less "the packages I use and I care about".
In a way the consensus should be reflected by the list you mention:
http://docs.zope.org/zopetoolkit/about/packages.html
> We need a policy to define the ZTK in an explicit way, otherwise we > will never get a *real* ZTK KGS that can be used to build applications > and the whole concept of ZTK will always be fuzzy.
We have a list. I propose that in order to stop long discussions about what should be in this list, we just start with what's in this list, by circular definition. :)
+1 Except that the "under review" list is pretty long. Shall we include everything that's under review as a starting point? We can always lobby to remove some later.
We need to get a procedure in place to do compat tests of what's in that list, dependency graph guarding of what's in that list, and locking down a KGS for that list.
That's what Fabio and others are working on. (Well, the first and third.)
I think that since we have a list doing all these things is only a matter of work - there's no fundamental questions we need answered before we can do this work. Once the base is there we can expand on it.
Sounds good.
I think what we need is a policy for adding packages into this list, and retiring packages from the list.
Yup
Removal: I think an informal show of hands that asks "is this package important?" on the mailing list is useful there. The other is a situation that nothing depends on that package anymore. Once those two are reached, I think it'd be as simple as petitioning the steering group to have a package removed.
Who's on the steering group?
Addition: this one is much tougher. New packages can get added if they're factored out of existing packages, that's easy. But fundamentally new package? We need to cross that bridge when we get to this. I suspect innovation for the time being will mostly be around the toolkit, not in it, or in the form of changes to existing packages. I think generally candidates for addition are packages that would change the way we arrange toolkit-based libraries themselves - I recall Shane's WSGI discussions as an example.
+1
I realize that to build "real" apps everybody will come up with a list of extra packages beyond the ZTK that they feel are needed too. Let a thousand flowers bloom I'd say - we just need a clean, fertile soil that we need to maintain. We already got plants growing in the soil anyway (Zope 2, Grok, bfg, and lots of Zope 3 apps).
+1
And of course there's some philosophy behind what's in the list now: it's a set of libraries shared by Zope 2, Zope 3 (whatever that is) and Grok. That's not the only answer and it's not quite the correct answer even,
It's a good enough start. Jim -- Jim Fulton
Jim Fulton wrote: [snip]
Except that the "under review" list is pretty long. Shall we include everything that's under review as a starting point? We can always lobby to remove some later.
Yes, I'm +1 on being inclusive. I want to be pretty aggressive about removal, but we need to get from here to there, so let's be inclusive to start with. Regards, Martijn
Hello, * 2009-08-06 19:30, Martijn Faassen wrote:
We need to get a procedure in place to do compat tests of what's in that list, dependency graph guarding of what's in that list, and locking down a KGS for that list. I think that since we have a list doing all these things is only a matter of work - there's no fundamental questions we need answered before we can do this work. Once the base is there we can expand on it.
I'm working to improve the KGS for the Zope Toolkit with automated testing and explicit policies. For example, I set up a buildbot instance here: http://buildbot.tranchitella.it/ztk/ For a subset of the KGS, it runs the tests for the current trunk as well as the tests for each of of the most recent released packages from the KGS. The idea is that we can be sure that the current trunk of a given package does not introduce test failures in any package in the KGS. The buildbot uses z3c.recipe.kgs (which is a mock-up you can download from svn.zope.org and not yet uploaded to pypi) and the KGS as defined here: http://svn.zope.org/*checkout*/zope.release/trunk/releases/controlled-packag... Everything seems to work pretty well, and I'm down to three-four failures for my subset.
I think what we need is a policy for adding packages into this list, and retiring packages from the list.
Yep, this was my original question: I just need a list of packages to submit to my buildbot instance. :) I still think that starting from a very small list, allowing zope committers to add packages making an explicit commitment to maintain them would give us a better overview of what does "ZTK" mean for each of us. Best regards, Fabio
Hey, Fabio Tranchitella wrote: [snip]
I still think that starting from a very small list, allowing zope committers to add packages making an explicit commitment to maintain them would give us a better overview of what does "ZTK" mean for each of us.
I'm of the opposite opinion; let's start with a moderately big list, more or less what was in Zope 3, but with anything ZMI related definitely planned for removal (and there's some other stuff). This is to recognize the reality that real apps right now depend on a *lot* of these packages. A lot of this code is not actually being used, which is why we do the refactoring, but I'd like to shrink gradually, instead of grow from a small core. This is because right now we'll have an easier time shrinking stuff down by fairly mechanical reasoning without involving a lot of people. A growing strategy that determines who is interested in maintaining a package sounds like it needs a lot of communication from a lot of people. In general it'd be good to avoid that.. (I can imagine however we could automate this partially by digging through SVN logs - that information might be interesting) Regards, Martijn
On Thu, Aug 6, 2009 at 1:05 PM, Martijn Faassen<faassen@startifact.com> wrote:
Hey,
Fabio Tranchitella wrote:
it is evident that there is no consensus on the list of packages that are part of the Zope Toolkit. As Gary suggested me, it looks like the concept of ZTK is different for each developer and it is more or less "the packages I use and I care about".
In a way the consensus should be reflected by the list you mention:
Do you mean the first list? (Libraries) For clarity, it might be a good idea to move the other 2 lists to a separate page. Jim -- Jim Fulton
Hey, On Tue, Aug 11, 2009 at 10:31 AM, Jim Fulton<jim@zope.com> wrote:
On Thu, Aug 6, 2009 at 1:05 PM, Martijn Faassen<faassen@startifact.com> wrote: [snip]
In a way the consensus should be reflected by the list you mention:
Do you mean the first list? (Libraries)
For clarity, it might be a good idea to move the other 2 lists to a separate page.
The 'under review' list counts as part of the ZTK at the moment too, though of course we'd like to shrink that. We could move the deprecated section to another page, though the heading's pretty clear. Regards, Martijn
On Tue, Aug 11, 2009 at 7:44 AM, Martijn Faassen<faassen@startifact.com> wrote:
Hey,
On Tue, Aug 11, 2009 at 10:31 AM, Jim Fulton<jim@zope.com> wrote:
On Thu, Aug 6, 2009 at 1:05 PM, Martijn Faassen<faassen@startifact.com> wrote: [snip]
In a way the consensus should be reflected by the list you mention:
Do you mean the first list? (Libraries)
For clarity, it might be a good idea to move the other 2 lists to a separate page.
The 'under review' list counts as part of the ZTK at the moment too, though of course we'd like to shrink that.
I think that's best. And I see that it is noted that these are considered in.
We could move the deprecated section to another page, though the heading's pretty clear.
OK. Jim -- Jim Fulton
participants (6)
-
Fabio Tranchitella -
Fred Drake -
Jim Fulton -
Martijn Faassen -
Tres Seaver -
Wichert Akkerman