We need to get the board green
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Too many never-resolve failures in our buildbots makes their output just noise: the amount of effort required to diagnose the cause of a failure seems to have no payoff if we don't get them each cleared up. I'm just about done with trying to diagnose them, myself, which is making me sad, as I *want* to be pleased with the quality of the software our community produces. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2HdRQACgkQ+gerLs4ltQ6zMQCgt2V6zhx/zKRJBmI3KSw5qD75 OgsAoJYLAmLGSMd9p+ZbrMqjZwDk99UU =l0kU -----END PGP SIGNATURE-----
Hello, On Mon, 21 Mar 2011 11:56:05 -0400 you wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Too many never-resolve failures in our buildbots makes their output just noise: the amount of effort required to diagnose the cause of a failure seems to have no payoff if we don't get them each cleared up.
I'm just about done with trying to diagnose them, myself, which is making me sad, as I *want* to be pleased with the quality of the software our community produces.
I'd say revert back to the original procedure. The guilty dev has x days to fix the failure or the change gets reverted. PS: And please please kick the guilty one's a**, not the buildbot maintainer's. It does not make much sense to disable tests just because they fail. -- Best regards, Adam GROSZER -- Quote of the day: If Presidents don't do it to their wives, they do it to the country. - Mel Brooks
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 03/22/2011 03:59 AM, Adam GROSZER wrote:
Hello,
On Mon, 21 Mar 2011 11:56:05 -0400 you wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Too many never-resolve failures in our buildbots makes their output just noise: the amount of effort required to diagnose the cause of a failure seems to have no payoff if we don't get them each cleared up.
I'm just about done with trying to diagnose them, myself, which is making me sad, as I *want* to be pleased with the quality of the software our community produces.
I'd say revert back to the original procedure. The guilty dev has x days to fix the failure or the change gets reverted.
PS: And please please kick the guilty one's a**, not the buildbot maintainer's. It does not make much sense to disable tests just because they fail.
Chameleon is an external dependency, not managed as part of the Zope repository; I presume that the authors / maintainers of the z3c.* packages chose to depend on it with full knowledge of the risks that entails. If a new release of Chameleon breaks z3c.*, then it is the z3c.* maintainers whose tails need to be motivated: they should either get the exteranl dependency fixed, or work around it in their own packages (e.g., by pinning their dependency to a "known good" version, or by updating to use the newer APIs). Leaving the packages as "permanently failing" is obviously not doing anything to motivate those maintainers. Leaving the board red is *de-motivating* to the community at large, who have to wade through failure reports for (apparently) unmaintained packages daily while trying to diagnose stuff which might have been broken by changes made the day before. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2IobQACgkQ+gerLs4ltQ7ZeQCgopO4FTJE9KhgDsXy7OyCtw5V qbcAn3OG56JqlMqKAqOj/YjQRRf0VDoW =68up -----END PGP SIGNATURE-----
* Tres Seaver <tseaver@palladion.com> [2011-03-22 14:18]:
On 03/22/2011 03:59 AM, Adam GROSZER wrote:
And please please kick the guilty one's a**, not the buildbot maintainer's. It does not make much sense to disable tests just because they fail.
Leaving the packages as "permanently failing" is obviously not doing anything to motivate those maintainers. Leaving the board red is *de-motivating* to the community at large, who have to wade through failure reports for (apparently) unmaintained packages daily while trying to diagnose stuff which might have been broken by changes made the day before.
I have to admit, I'm ignoring the buildbots completely at this time, (even though I think that having them in place could be very valuable), since all I perceive from them is failure-noise, and no trend of it getting less, at all, over the last few months. (That's my gut feeling, not based on research. It still makes me very much want to ignore them.) So, I agree with Tres: this situation is quite demotivating. But I also agree with Adam: I think the value of the buildbots is that they run tests, so disabling tests because they fail feels a little like closing our eyes shut because we don't want to see we're going to be falling off a cliff. I don't have much of an idea how to proceed, except maybe explicitly reducing the scope of the buildbots to a set of packages people around here care enough about that they want and will keep them green. Wolfgang
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 03/23/2011 02:40 AM, Wolfgang Schnerring wrote:
* Tres Seaver <tseaver@palladion.com> [2011-03-22 14:18]:
On 03/22/2011 03:59 AM, Adam GROSZER wrote:
And please please kick the guilty one's a**, not the buildbot maintainer's. It does not make much sense to disable tests just because they fail.
Leaving the packages as "permanently failing" is obviously not doing anything to motivate those maintainers. Leaving the board red is *de-motivating* to the community at large, who have to wade through failure reports for (apparently) unmaintained packages daily while trying to diagnose stuff which might have been broken by changes made the day before.
I have to admit, I'm ignoring the buildbots completely at this time, (even though I think that having them in place could be very valuable), since all I perceive from them is failure-noise, and no trend of it getting less, at all, over the last few months. (That's my gut feeling, not based on research. It still makes me very much want to ignore them.) So, I agree with Tres: this situation is quite demotivating.
But I also agree with Adam: I think the value of the buildbots is that they run tests, so disabling tests because they fail feels a little like closing our eyes shut because we don't want to see we're going to be falling off a cliff.
I don't have much of an idea how to proceed, except maybe explicitly reducing the scope of the buildbots to a set of packages people around here care enough about that they want and will keep them green.
Yup. That is why I was asking to suspend the 'z3c.*' packages which are failing: they haven't passed in months, and nobody seems motivated to fix them. Although they are in ragged shape at the moment, I *do* work to diagnose / fix failures in the ZTK and Zope2 tests. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2KBpMACgkQ+gerLs4ltQ4KfQCgka3lt0/wHdpb/lAIG0KJYp4b b7wAoIivluUBo5XFmF/oFIxUfM8nBqNF =G5fZ -----END PGP SIGNATURE-----
On Mon, Mar 21, 2011 at 11:56:05AM -0400, Tres Seaver wrote:
Too many never-resolve failures in our buildbots makes their output just noise: the amount of effort required to diagnose the cause of a failure seems to have no payoff if we don't get them each cleared up.
+100 to any method of reducing that noise. Continuously failing buildbots shouldn't make the system unusable. -- Brian Sutherland
Hello, On Tue, 22 Mar 2011 17:20:05 +0100 you wrote:
On Mon, Mar 21, 2011 at 11:56:05AM -0400, Tres Seaver wrote:
Too many never-resolve failures in our buildbots makes their output just noise: the amount of effort required to diagnose the cause of a failure seems to have no payoff if we don't get them each cleared up.
+100 to any method of reducing that noise. Continuously failing buildbots shouldn't make the system unusable.
Well I did some... Pinned z3c.pt and Chameleon versions. Others are welcome too. -- Best regards, Adam GROSZER -- Quote of the day: I have read Plato and Cicero sayings that are very wise and beautiful; but I never read in either of them: "Come unto me all ye that labour and are heavy laden and I will give thee rest." - St. Augustine
* Tres Seaver <tseaver@palladion.com> [2011-03-21 16:56]:
Too many never-resolve failures in our buildbots makes their output just noise: the amount of effort required to diagnose the cause of a failure seems to have no payoff if we don't get them each cleared up.
On a tangentially-related note, what happened to the proposed new summarizer formatting? Half a year ago (or whenever), I got the impression it was going to be ready "in a few days", but it seems it never happened. What is needed to get this switched? (I'm not saying dressing the continuous failures up nicer is a solution to the problem Tres is presenting, but I do think the noisy format contributes to the confusion about the buildbots) Wolfgang
Hello, On Wed, 23 Mar 2011 07:43:26 +0100 you wrote:
* Tres Seaver<tseaver@palladion.com> [2011-03-21 16:56]:
Too many never-resolve failures in our buildbots makes their output just noise: the amount of effort required to diagnose the cause of a failure seems to have no payoff if we don't get them each cleared up.
On a tangentially-related note, what happened to the proposed new summarizer formatting? Half a year ago (or whenever), I got the impression it was going to be ready "in a few days", but it seems it never happened. What is needed to get this switched?
(I'm not saying dressing the continuous failures up nicer is a solution to the problem Tres is presenting, but I do think the noisy format contributes to the confusion about the buildbots)
You mind walking over to Theuni and asking him? ;-) -- Best regards, Adam GROSZER -- Quote of the day: be a butterfly. - Buckminster Fuller
On 03/23/2011 08:45 AM, Adam GROSZER wrote:
Hello,
On Wed, 23 Mar 2011 07:43:26 +0100 you wrote:
* Tres Seaver<tseaver@palladion.com> [2011-03-21 16:56]:
Too many never-resolve failures in our buildbots makes their output just noise: the amount of effort required to diagnose the cause of a failure seems to have no payoff if we don't get them each cleared up.
On a tangentially-related note, what happened to the proposed new summarizer formatting? Half a year ago (or whenever), I got the impression it was going to be ready "in a few days", but it seems it never happened. What is needed to get this switched?
(I'm not saying dressing the continuous failures up nicer is a solution to the problem Tres is presenting, but I do think the noisy format contributes to the confusion about the buildbots)
You mind walking over to Theuni and asking him? ;-)
Dang! I need to really change my lair now ... -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting, development, hosting, operations
Hello, * Adam GROSZER <agroszer@gmail.com> [2011-03-23 08:45]:
On Wed, 23 Mar 2011 07:43:26 +0100 you wrote:
On a tangentially-related note, what happened to the proposed new summarizer formatting? Half a year ago (or whenever), I got the impression it was going to be ready "in a few days", but it seems it never happened. What is needed to get this switched?
You mind walking over to Theuni and asking him? ;-)
Oh. Right. Sure. *walks over and back* So. Theuni says, the code[1] (based on the current aggregator script) is probably pretty much ready for deployment. This is a script to be run by cron that scrapes the HTML list archives of the zope-test list. I haven't looked at it yet, so I don't know how ready "probably pretty much" actually means. If I understood this correctly, the current aggregator runs courtesy of Stephan Holek. I guess it would be a good idea to get this somewhere closer to the Zope Foundation, so I'd rather we work something out with Jens Vagelpohl how/where to run this than throw it up on one of our boxes here at gocept. Adam, do you want to tackle this? Otherwise, I can do it, but I don't know when I get enough round toits to make it happen. Wolfgang [1] http://zope3.pov.lt/trac/browser/Sandbox/ctheune/testsummarizer -- Wolfgang Schnerring · ws@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development
Hello, On Thu, 24 Mar 2011 10:51:44 +0100 you wrote:
Hello,
* Adam GROSZER<agroszer@gmail.com> [2011-03-23 08:45]:
On Wed, 23 Mar 2011 07:43:26 +0100 you wrote:
On a tangentially-related note, what happened to the proposed new summarizer formatting? Half a year ago (or whenever), I got the impression it was going to be ready "in a few days", but it seems it never happened. What is needed to get this switched?
You mind walking over to Theuni and asking him? ;-)
Oh. Right. Sure.
*walks over and back*
So. Theuni says, the code[1] (based on the current aggregator script) is probably pretty much ready for deployment. This is a script to be run by cron that scrapes the HTML list archives of the zope-test list. I haven't looked at it yet, so I don't know how ready "probably pretty much" actually means.
If I understood this correctly, the current aggregator runs courtesy of Stephan Holek. I guess it would be a good idea to get this somewhere closer to the Zope Foundation, so I'd rather we work something out with Jens Vagelpohl how/where to run this than throw it up on one of our boxes here at gocept.
Adam, do you want to tackle this? Otherwise, I can do it, but I don't know when I get enough round toits to make it happen.
Wolfgang
[1] http://zope3.pov.lt/trac/browser/Sandbox/ctheune/testsummarizer
I'm rather busy nowadays. But it seems like it's about bugging Stephan Holek to stop the current one and bugging Jens to start the new one, or? Unless the script is broken. Could you run that script -- worst case we'll have 2 mails for a day -- for testing? Seems like it has the settings for gocept and I don't really have an SMTP server here handy. -- Best regards, Adam GROSZER -- Quote of the day: Don't tell any big lies today. Small ones can be just as effective.
Hi, * Adam GROSZER <agroszer.ll@gmail.com> [2011-03-29 11:15]:
On Thu, 24 Mar 2011 10:51:44 +0100 you wrote:
So. Theuni says, the code[1] (based on the current aggregator script) is probably pretty much ready for deployment. This is a script to be run by cron that scrapes the HTML list archives of the zope-test list. I haven't looked at it yet, so I don't know how ready "probably pretty much" actually means. [1] http://zope3.pov.lt/trac/browser/Sandbox/ctheune/testsummarizer
If I understood this correctly, the current aggregator runs courtesy of Stephan Holek. I guess it would be a good idea to get this somewhere closer to the Zope Foundation, so I'd rather we work something out with Jens Vagelpohl how/where to run this than throw it up on one of our boxes here at gocept.
Adam, do you want to tackle this? Otherwise, I can do it, but I don't know when I get enough round toits to make it happen.
I'm rather busy nowadays. But it seems like it's about bugging Stephan Holek to stop the current one and bugging Jens to start the new one, or? Unless the script is broken.
Could you run that script -- worst case we'll have 2 mails for a day -- for testing? Seems like it has the settings for gocept and I don't really have an SMTP server here handy.
I'd really rather not duct-tape this... Alright, then I'll try and get it into running shape and bug people to run it. And unfortunately the code looks like it needs some work before it can be deployed properly. Wolfgang -- Wolfgang Schnerring · ws@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 3/29/11 11:15 , Adam GROSZER wrote:
But it seems like it's about bugging Stephan Holek to stop the current one and bugging Jens to start the new one, or? Unless the script is broken.
Could you run that script -- worst case we'll have 2 mails for a day -- for testing? Seems like it has the settings for gocept and I don't really have an SMTP server here handy.
Thanks to Wolfgang's cleanup work the new script is now in place. It's running once a day at 01:00 AM Eastern Standard Time. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAk2YTtAACgkQRAx5nvEhZLJZtACglWfBgjJRhd3r7iIuU7vb09Pf ybYAnjKBhYpGnapLjiotUHTsJIxNLcGT =bE8F -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 4/3/11 12:41 , Jens Vagelpohl wrote:
On 3/29/11 11:15 , Adam GROSZER wrote:
But it seems like it's about bugging Stephan Holek to stop the current one and bugging Jens to start the new one, or? Unless the script is broken.
Could you run that script -- worst case we'll have 2 mails for a day -- for testing? Seems like it has the settings for gocept and I don't really have an SMTP server here handy.
Thanks to Wolfgang's cleanup work the new script is now in place. It's running once a day at 01:00 AM Eastern Standard Time.
@Wolfgang: Something is not working as seen in today's run. If I run the script with "-T 2011-04-02" I am getting correct output. But if I run it with "-T 2011-04-03" or "-T 2011-04-04" I am getting nothing. Can you test this in your sandbox? Thanks! jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAk2ZZxYACgkQRAx5nvEhZLL7aQCgo6h5CtRLQQeWXrLh7zzILb5F BY0AoLLCmHIyJVVflYNih+s7uEF+wMSo =Bm4U -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 4/4/11 08:37 , Jens Vagelpohl wrote:
On 4/3/11 12:41 , Jens Vagelpohl wrote:
On 3/29/11 11:15 , Adam GROSZER wrote:
But it seems like it's about bugging Stephan Holek to stop the current one and bugging Jens to start the new one, or? Unless the script is broken. Could you run that script -- worst case we'll have 2 mails for a day -- for testing? Seems like it has the settings for gocept and I don't really have an SMTP server here handy. Thanks to Wolfgang's cleanup work the new script is now in place. It's running once a day at 01:00 AM Eastern Standard Time.
@Wolfgang: Something is not working as seen in today's run. If I run the script with "-T 2011-04-02" I am getting correct output. But if I run it with "-T 2011-04-03" or "-T 2011-04-04" I am getting nothing. Can you test this in your sandbox?
@Wolfgang: I have checked in some fixes and it works for me now. We'll see the results tomorrow. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (Darwin) Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAk2Zvn0ACgkQRAx5nvEhZLI62ACdHgr8dvtV0K0esDkpHxYiy1Qv 5u8AnjqXOdmtmk0+/TbojuQ6uUJwK1Ln =2Rbj -----END PGP SIGNATURE-----
participants (7)
-
Adam GROSZER -
Adam GROSZER -
Brian Sutherland -
Christian Theune -
Jens Vagelpohl -
Tres Seaver -
Wolfgang Schnerring