[Zope] Re: [Zope3-dev] The Zope Software Certification Program and Common Repository Proposal

[Stephan Richter]

On Tuesday 21 February 2006 05:59, Reinoud van Leeuwen wrote:
> This seems to me a great step forward but I am missing something.
> The quality is measured by a number of metrics, but it seems nowhere is
> actually measured if the software does what it is supposed to do, if it is
> clear how it works and whether it is something you would advise other
> people to use.
> This is one of the major problems I've had in the past with things like
> the Perl CPAN repository: you can find lot's of modules that seem to solve
> your problem, but usually you discover what a module is really about when
> you've invested a lot of time in it.

Right, I hear you. In fact, this is one of the reasons that we decided against 
a name like "Zope Software Quality Program". With the proposed process, we 
cannot guarantee 100% that the package is good. However, there are a couple 
of safe guards:

(1) If you write doctests as a narrative text file, you really have to think 
hard about the functionality your package provides. I cannot stress it 
enough, doctest text files are *key* to the success of the certification 
program.

(2) At least in the Common Repository, people will read check-in messages.

(3) At higher certification levels, other people must support the package. 
This is also not 100% bullet proof, but it is something.

Overall, I also expect that the community has little tolerance for malicious 
attempts to break the system. If someone detects foul play all he has to do 
is complain on the mailing lists.

> Would there be room for a voting or feedback step in the process where
> people that have tried the module could enter a rating?

Ah, rating and feedback. :-) This was discussed in the pre-proposal phase as 
well. The problem with feedback and rating is that to do it right, it 
requires a lot of resources that we do not have. Here is a scenario:

1. A user U trashes a package because an important feature F is missing in 
package P. So far so good. It is his right to do so.

2. The package P authors see the comment and fix it in the code. Very cool, 
the process works.

3. But then the user U of the post, must retract his comment. What if he is 
not available? Not so good. The alternative would be to ask a certification 
manager, who might know nothing about the issue and will need a lot of time 
reviewing the complaint and solution. Not so good.

While rating and feedback is good, to do it right in a process like the ZSCP 
costs a lot of resources.

Having said that, I see the need to address the issue. Here are two 
suggestions:

1. Add a "Future Possibilities" section to the proposal collecting ideas for 
later iterations of the process. This would allow us to address some of the 
common concerns and say: If we have time and resources, this can be done.

2. There is already a provision in the process that a package can receive a 
warning. Currently the ZSCP states that the warning can only be issued when a 
package does not fulfill the quality metrics for a given release.

I could add another provision that a warning can be issued, if X community 
members and 1 certification manager verified a bad package. Each warning 
carries an arbitrary comment that could describe the reason of the warning. 
This way we can use the existing communication channels (mailing list and 
IRC) for feedback and still have a way to formalize feedback. I guess in this 
case we would also need a "resolve" action that could resolve a warning.

What do you think?

Regards,
Stephan
-- 
Stephan Richter
CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student)
Web2k - Web Software Design, Development and Training