On Mon, Feb 20, 2006 at 04:31:03PM -0500, Stephan Richter wrote: Hi Stephan, This seems to me a great step forward but I am missing something. The quality is measured by a number of metrics, but it seems nowhere is actually measured if the software does what it is supposed to do, if it is clear how it works and whether it is something you would advise other people to use. This is one of the major problems I've had in the past with things like the Perl CPAN repository: you can find lot's of modules that seem to solve your problem, but usually you discover what a module is really about when you've invested a lot of time in it. Would there be room for a voting or feedback step in the process where people that have tried the module could enter a rating?
1.2. Goals [...] * Quality Packages
There is a natural desire for any developer to know what they are getting into when they are using a certain package, a baseline of quality that can be expected. While the Zope 3 community has some ideas of what that baseline is for the core, it is not well defined and applied uniformly. This proposal defines clear quality guidelines.
[...]
2.4. Quality Metrics ~~~~~~~~~~~~~~~~~~~~
The certification is meaningless without the precise definition of tasks that have to be accomplished for each certification level. This section provides a list of concrete items that have to be fulfilled for each certification level.
Legend:
- x: A metric is required for the certification level. - A: The metric check can be conducted automatically. - Q: The metric check can be conducted quickly by human inspection. - D: The metric check would be difficult to conduct by human inspection.
+------------------------------------------+-------+------+------+------+------+ | Metric | Check | List | Le 1 | Le 2 | Le 3 | +==========================================+=======+======+======+======+======+ | Package Meta-Information (see sec. 2.5) | A | x | x | x | x | +------------------------------------------+-------+------+------+------+------+ | Test Coverage | A | 0% | >90% | >95% | >95% | +------------------------------------------+-------+------+------+------+------+ | Tests verified by Automated Test Runner | A | x | x | x | x | +------------------------------------------+-------+------+------+------+------+ | Doctest-based Testing | A,Q | | x | x | x | | (or list reason for not using doctests) | | | | | | +------------------------------------------+-------+------+------+------+------+ | Tests pass on all supported platforms | A,Q | | x | x | x | +------------------------------------------+-------+------+------+------+------+ | Minimal Documentation | A,Q | x | x | x | x | | (``README.txt`` file) | | | | | | +------------------------------------------+-------+------+------+------+------+ | Complete Documentation | Q | | x | x | x | | (Text files cover all of API) | | | | | | +------------------------------------------+-------+------+------+------+------+ | Extensive Documentation | D | | | | x | | (lots of samples, API docs, tutorial) | | | | | | +------------------------------------------+-------+------+------+------+------+ | Documentation available online [1] | Q | | | x | x | +------------------------------------------+-------+------+------+------+------+ | Documentation available in Zope's apidoc | Q | | | x | x | +------------------------------------------+-------+------+------+------+------+ | Common package structure | A,Q | x | x | x | x | +------------------------------------------+-------+------+------+------+------+ | Follows Zope Coding Style Guide | A,D | | x | x | x | +------------------------------------------+-------+------+------+------+------+ | Conform to user interface guidelines | D | | | x | x | | (if applicable to package) | | | | | | +------------------------------------------+-------+------+------+------+------+ | Complete dependency list | A | | x | x | x | +------------------------------------------+-------+------+------+------+------+ | Standard installation method | A,Q | | | x | x | +------------------------------------------+-------+------+------+------+------+ | Release(s) with version number | A,Q | | | x | x | +------------------------------------------+-------+------+------+------+------+ | Up-to-date homepage | D | | | | x | +------------------------------------------+-------+------+------+------+------+ | Active support mailing list | D | | | | x | +------------------------------------------+-------+------+------+------+------+ | Released for at least 3 Zope releases | D | | | | x | +------------------------------------------+-------+------+------+------+------+ | Releases state required Zope version | A,Q | | | | x | +------------------------------------------+-------+------+------+------+------+ | Multiple (3) Active Maintainers | | | | x | x | +------------------------------------------+-------+------+------+------+------+ | Data migration claimed | Q | | x | x | x | | (if applicable) | | | | | | +------------------------------------------+-------+------+------+------+------+ | Data migration auto-tested | A | | | x | x | | (if applicable) | | | | | | +------------------------------------------+-------+------+------+------+------+ | Data migration verified | D | | | | x | | (if applicable) | | | | | | +------------------------------------------+-------+------+------+------+------+ | Data migration well-tested | D | | | | | | (if applicable) | | | | | [2] | +------------------------------------------+-------+------+------+------+------+
-- __________________________________________________ "Nothing is as subjective as reality" Reinoud van Leeuwen reinoud.v@n.leeuwen.net http://www.xs4all.nl/~reinoud __________________________________________________