Re: [Zope-dev] Uses of the ZTK and how it relates to management
* 2010-03-03 21:44, Jim Fulton wrote:
The ZTK was created in part to deal with instability issues arising from people working on parts without testing the whole.
I suppose everybody here agrees that any change to a package which is part of the ZTK *must* be tested against the whole ZTK. It would be easier to nd leading developers for subgroups of packages (eg. bicycle repair kit, rm generation, ...) willing to raise the quality of a specific subset of packages instead of finding a release manager willing to oversee > 60 packages, which he does not really use (because I don't think we have a single developer using *all* of the packages in the ZTK). These specific leading developers could report and synchronize with a ZTK release manager, though. My two euro cents. Fabio
On Thu, Mar 4, 2010 at 1:12 PM, Fabio Tranchitella <kobold@kobold.it> wrote:
* 2010-03-03 21:44, Jim Fulton wrote:
The ZTK was created in part to deal with instability issues arising from people working on parts without testing the whole.
I suppose everybody here agrees that any change to a package which is part of the ZTK *must* be tested against the whole ZTK. It would be easier to nd leading developers for subgroups of packages (eg. bicycle repair kit, rm generation, ...) willing to raise the quality of a specific subset of packages instead of finding a release manager willing to oversee > 60 packages, which he does not really use (because I don't think we have a single developer using *all* of the packages in the ZTK).
These specific leading developers could report and synchronize with a ZTK release manager, though.
I wonder whether we can split ztk.cfg & zopeapp.cfg further logically ? Something like zca.cfg , index.cfg, form.cfg, auth.cfg etc. ? May be we will end up with one cfg file for each package :) Regards, Baiju M
Hello, * 2010-03-04 09:22, Baiju M wrote:
I wonder whether we can split ztk.cfg & zopeapp.cfg further logically ? Something like zca.cfg , index.cfg, form.cfg, auth.cfg etc. ? May be we will end up with one cfg file for each package :)
I don't think we need to split the cfg files: IMHO what we are looking for (or should be looking for) is not a technical solution, but something to solve a management issue. Having people responsible for a subset of packages does not involve any change in our technical infrastructure, maybe just a website where to store the documentation. Anyway, this is my "idea" of fragmentation of the ZTK into self-contained reusable groups of packages, you can like it or not like, it is just an idea. :) Best regards, Fabio
On Thu, Mar 4, 2010 at 2:42 AM, Fabio Tranchitella <kobold@kobold.it> wrote:
* 2010-03-03 21:44, Jim Fulton wrote:
The ZTK was created in part to deal with instability issues arising from people working on parts without testing the whole.
I suppose everybody here agrees that any change to a package which is part of the ZTK *must* be tested against the whole ZTK.
It would be great if that were true. If so, then the recent arguments have been a terrible misunderstanding.
It would be easier to find leading developers for subgroups of packages (eg. bicycle repair kit, rm generation, ...) willing to raise the quality of a specific subset of packages instead of finding a release manager willing to oversee > 60 packages, which he does not really use (because I don't think we have a single developer using *all* of the packages in the ZTK).
These specific leading developers could report and synchronize with a ZTK release manager, though.
There's nothing preventing people from doing this AFAICT. If someone is interested in pursuing a change to a package or collection of packages, they can do so with or without some organizational structure. Problems would arise only if they proposed a backward incompatible change, which isn't to say that backward-incompatible changes are impossible. Jim -- Jim Fulton
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jim Fulton wrote:
On Thu, Mar 4, 2010 at 2:42 AM, Fabio Tranchitella <kobold@kobold.it> wrote:
* 2010-03-03 21:44, Jim Fulton wrote:
The ZTK was created in part to deal with instability issues arising from people working on parts without testing the whole. I suppose everybody here agrees that any change to a package which is part of the ZTK *must* be tested against the whole ZTK.
It would be great if that were true. If so, then the recent arguments have been a terrible misunderstanding.
I think that most of the misunderstanding lies in the nature of "ownership" of the packages in the ZTK: some would like everyone to care equally about all the packages (including those now factored into the zopeapp.cfg set), while others want to give most of their attention to some smaller set of packages which they use every day. We already have some policies in place which recognize that the "bicycle seat toolkit" packages need special handling, becaues they are used outside the Zope ecosystem (one rule is that we attempt to keep Python 2.4 compatibiltiy for those packages). I certainly don't think anybody here would argue against having the big 'compattest' tests get run: even the "splitters" and "whittlers" (I might be called one of those) are willing to help fix things when a change to one package breaks the others. If we could get automated testing of the various compatibility sets established, with automated reporting of failures, I think we would see a huge improvement in the signal here: instead of arguing hypotheticals, we would be focused on fixing "real" breakages.
It would be easier to find leading developers for subgroups of packages (eg. bicycle repair kit, rm generation, ...) willing to raise the quality of a specific subset of packages instead of finding a release manager willing to oversee > 60 packages, which he does not really use (because I don't think we have a single developer using *all* of the packages in the ZTK).
These specific leading developers could report and synchronize with a ZTK release manager, though.
There's nothing preventing people from doing this AFAICT. If someone is interested in pursuing a change to a package or collection of packages, they can do so with or without some organizational structure. Problems would arise only if they proposed a backward incompatible change, which isn't to say that backward-incompatible changes are impossible.
We still mostly treat them as off-limits, even the three years after we started the effort to break the monolith apart. That change should have made it technically feasible to do backward-incompatible changes, but we haven't yet worked out how to make that happen. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkuQC9gACgkQ+gerLs4ltQ4x8QCfXQ2JkuCy4XyJgHDuzrZZjLIc 1wkAn0J7kdM4F4iRcO4OP2EGGsKi+1vB =r+V4 -----END PGP SIGNATURE-----
On Thu, Mar 4, 2010 at 2:37 PM, Tres Seaver <tseaver@palladion.com> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Jim Fulton wrote:
On Thu, Mar 4, 2010 at 2:42 AM, Fabio Tranchitella <kobold@kobold.it> wrote:
* 2010-03-03 21:44, Jim Fulton wrote:
The ZTK was created in part to deal with instability issues arising from people working on parts without testing the whole. I suppose everybody here agrees that any change to a package which is part of the ZTK *must* be tested against the whole ZTK.
It would be great if that were true. If so, then the recent arguments have been a terrible misunderstanding.
I think that most of the misunderstanding lies in the nature of "ownership" of the packages in the ZTK: some would like everyone to care equally about all the packages (including those now factored into the zopeapp.cfg set), while others want to give most of their attention to some smaller set of packages which they use every day. We already have some policies in place which recognize that the "bicycle seat toolkit" packages need special handling, becaues they are used outside the Zope ecosystem (one rule is that we attempt to keep Python 2.4 compatibiltiy for those packages).
I certainly don't think anybody here would argue against having the big 'compattest' tests get run: even the "splitters" and "whittlers" (I might be called one of those) are willing to help fix things when a change to one package breaks the others. If we could get automated testing of the various compatibility sets established, with automated reporting of failures, I think we would see a huge improvement in the signal here: instead of arguing hypotheticals, we would be focused on fixing "real" breakages.
+1 Hopefully the discussions about buildbots will bear fruit.
It would be easier to find leading developers for subgroups of packages (eg. bicycle repair kit, rm generation, ...) willing to raise the quality of a specific subset of packages instead of finding a release manager willing to oversee > 60 packages, which he does not really use (because I don't think we have a single developer using *all* of the packages in the ZTK).
These specific leading developers could report and synchronize with a ZTK release manager, though.
There's nothing preventing people from doing this AFAICT. If someone is interested in pursuing a change to a package or collection of packages, they can do so with or without some organizational structure. Problems would arise only if they proposed a backward incompatible change, which isn't to say that backward-incompatible changes are impossible.
We still mostly treat them as off-limits, even the three years after we started the effort to break the monolith apart. That change should have made it technically feasible to do backward-incompatible changes, but we haven't yet worked out how to make that happen.
I wonder how many situations there are where backward incompatible changes are needed to unblock other work. I don't suppose we have a list of these do we? One thing that makes problems like this really hard is that email is such a terrible tool for much of the needed communication. It would be nice if we could do something more sprinty. I don't want to help if it involves drawn out email discussions, but, FWIW, I'd be willing to allocate some concentrated blocks of time for high-bandwidth discussion and work. Jim -- Jim Fulton
One thing that makes problems like this really hard is that email is such a terrible tool for much of the needed communication. It would be nice if we could do something more sprinty. I don't want to help if it involves drawn out email discussions, but, FWIW, I'd be willing to allocate some concentrated blocks of time for high-bandwidth discussion and work.
Fortunately there are all sorts of free/cheap ways for high bandwidth communication. Maybe a free conference call + screen sharing. If there was an agenda -- I would be more than willing to schedule, ensure technology is working, and coordinate people being there. Some people in the plone community have been doing weekly sprints. They just completed their 27th weekly sprint. They spend 6 hours or so on a set of bugs in the issue tracker. They do it over #irc. By this is still lowbandwidth communications. Nothing better than voice + supporting visuals to increase effectiveness. Again if someone wants to setup agenda I can help w/ logistics and ensure voice/screen sharing will work for, say, 20-40 people. Unfortunately I know very little about ZTK and the dependency issues. I would not be good person to setup an agenda. cheers alan
Hi, On 03/04/2010 10:44 PM, Jim Fulton wrote:
On Thu, Mar 4, 2010 at 2:37 PM, Tres Seaver <tseaver@palladion.com> wrote:
Weird. Tres' mail didn't make it into gmane ... I'll fold my replies to Jim and Tres into this one mail. (Looks like I'm now turning into the one making noise here. I'm already sorry for people having to read so much. :) )
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Jim Fulton wrote:
On Thu, Mar 4, 2010 at 2:42 AM, Fabio Tranchitella <kobold@kobold.it> wrote:
* 2010-03-03 21:44, Jim Fulton wrote:
The ZTK was created in part to deal with instability issues arising from people working on parts without testing the whole. I suppose everybody here agrees that any change to a package which is part of the ZTK *must* be tested against the whole ZTK.
It would be great if that were true. If so, then the recent arguments have been a terrible misunderstanding.
I think that most of the misunderstanding lies in the nature of "ownership" of the packages in the ZTK: some would like everyone to care equally about all the packages (including those now factored into the zopeapp.cfg set), while others want to give most of their attention to some smaller set of packages which they use every day.
Something that appeared to me while reading that: the specific set that a single developer cares for varies over time. Independent of the specific set I care for at any point in time, I still am interested in a healthy larger set because I might need that again tomorrow or next week.
We already
have some policies in place which recognize that the "bicycle seat toolkit" packages need special handling, becaues they are used outside the Zope ecosystem (one rule is that we attempt to keep Python 2.4 compatibiltiy for those packages).
I certainly don't think anybody here would argue against having the big 'compattest' tests get run: even the "splitters" and "whittlers" (I might be called one of those) are willing to help fix things when a change to one package breaks the others. If we could get automated testing of the various compatibility sets established, with automated reporting of failures, I think we would see a huge improvement in the signal here: instead of arguing hypotheticals, we would be focused on fixing "real" breakages.
+1
Hopefully the discussions about buildbots will bear fruit.
Working on that. :)
It would be easier to find leading developers for subgroups of packages (eg. bicycle repair kit, rm generation, ...) willing to raise the quality of a specific subset of packages instead of finding a release manager willing to oversee > 60 packages, which he does not really use (because I don't think we have a single developer using *all* of the packages in the ZTK).
These specific leading developers could report and synchronize with a ZTK release manager, though.
There's nothing preventing people from doing this AFAICT. If someone is interested in pursuing a change to a package or collection of packages, they can do so with or without some organizational structure. Problems would arise only if they proposed a backward incompatible change, which isn't to say that backward-incompatible changes are impossible.
We still mostly treat them as off-limits, even the three years after we started the effort to break the monolith apart. That change should have made it technically feasible to do backward-incompatible changes, but we haven't yet worked out how to make that happen.
I wonder how many situations there are where backward incompatible changes are needed to unblock other work. I don't suppose we have a list of these do we?
One thing that makes problems like this really hard is that email is such a terrible tool for much of the needed communication. It would be nice if we could do something more sprinty. I don't want to help if it involves drawn out email discussions, but, FWIW, I'd be willing to allocate some concentrated blocks of time for high-bandwidth discussion and work.
Here's a "save the date": gocept is organising a summit/sprint close to this year's DZUG conference which will also have sprinting time and space all over. So that's definitely high-bandwidth. :) The summit itself will be more oriented to getting the Zope developer group organies so we can have less of the meta discussions on the mailing list (that would probably be the part Jim is uninterested in ;)). I'd like to keep that focused on a single day though and use the rest of the sprinting time for getting stuff done. So, anyway, here's the dates: 13. September 2010: zope-dev summit in Halle (Saale), Germany 14. September 2010: sprint in Halle (Saale) 15.-17. September 2010: DZUG conference in Dresden, Geramny 18.-19. September 2010: post-conference sprinting days in Dresden I'd love to see as many of you zope-dev guys here in our offices. :) Christian -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development
On 03/04/2010 04:13 PM, Jim Fulton wrote:
On Thu, Mar 4, 2010 at 2:42 AM, Fabio Tranchitella <kobold@kobold.it> wrote:
* 2010-03-03 21:44, Jim Fulton wrote:
The ZTK was created in part to deal with instability issues arising from people working on parts without testing the whole.
I suppose everybody here agrees that any change to a package which is part of the ZTK *must* be tested against the whole ZTK.
It would be great if that were true.
That's been my understanding all the time. I think we currently don't do it because it might be non-obvious to the individual developers at the time of check-in and due to underused tools. We may not require that a single developer run all test suites in all combinations we desire to be compatible, but with the ongoing effort to improve the nightly/automated builds we should get into better shape there. The one issue with delayed testing is that we would have to impose a rule which requires "cool down" time after a checkin before making a release. That's probably not a terrible thing anyway. Christian -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Christian Theune wrote:
On 03/04/2010 04:13 PM, Jim Fulton wrote:
On Thu, Mar 4, 2010 at 2:42 AM, Fabio Tranchitella <kobold@kobold.it> wrote:
* 2010-03-03 21:44, Jim Fulton wrote:
The ZTK was created in part to deal with instability issues arising from people working on parts without testing the whole. I suppose everybody here agrees that any change to a package which is part of the ZTK *must* be tested against the whole ZTK. It would be great if that were true.
That's been my understanding all the time. I think we currently don't do it because it might be non-obvious to the individual developers at the time of check-in and due to underused tools.
We may not require that a single developer run all test suites in all combinations we desire to be compatible, but with the ongoing effort to improve the nightly/automated builds we should get into better shape there.
The one issue with delayed testing is that we would have to impose a rule which requires "cool down" time after a checkin before making a release. That's probably not a terrible thing anyway.
I would think that requiring the buildbots to clear before tagging a new release of the ztk.cfg file is a good "cool down", but we can't require that people not release the included packages: the ZTK isn't really testable without having them released (a chicken-and-egg problem). I would say that updating ztk.cfg to new "major" versions of sub-packages (ones with new features, or potential backward compatibility issues) requires a great deal more care than updating it to use a new "bugfix only" release of a package already in the ZTK. Adding or removing packages from the ZTK should probably also be a "branch only" item, with discussion required here before merging. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkuRVaYACgkQ+gerLs4ltQ5J3gCg0tFeV6g1lxDk9NdxKk9Ze1Is ceIAoIJjcsL/Ya7Ei4H5lBxqE+sdGDsh =epLA -----END PGP SIGNATURE-----
participants (6)
-
Alan Runyan -
Baiju M -
Christian Theune -
Fabio Tranchitella -
Jim Fulton -
Tres Seaver