[Zope-dev] Zope2 / Zope3 catalog index fork / divergence

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Christian Theune wrote:
> On Tue, 2009-06-09 at 20:07 -0400, Tres Seaver wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Chris McDonough wrote:
>>> Log message for revision 100775:
>>>   - Port okascore.c optimization used in okapiiindex from Zope2 catalog
>>>     implementation.  This module is compiled conditionally, based on
>>>     whether your environment has a working C compiler.
>>>   
>>>   - Don't use ``len(self._docweight)`` in okapiindex _search_wids method
>>>     (obtaining the length of a BTree is very expensive at scale).
>>>     Instead use self.documentCount().  Also a Zope2 port.
>> Crap!  this set of performance fixes was added *six years ago* to the
>> Zope2 version from which zope.index was forked!
>>
>> Hasn't anybody been using this package?
> 
> We're using this package. The text index I can only remember in a
> not-so-critical piece where performance currently is not an issue. Also,
> following checkins even today wouldn't have me remember that this code
> is shared.

The code which now lives in 'zope.index' is a copy-and-rename-everything
fork of what was in the Zope2 PluggableIndexes / ZCTextIndex products as
of around Christmas, 2002.  The fix I was replying to is *far* from the
first one which didn't get ported from subsequent changes to the Zope2
indexes, which have obviously been in *heavy* production use.

I propose that we consider merging the two implementations, along the
lines of the following:

1. Review the change history of the two codebases since the original
   fork, porting relevant changes from the Z2 indexes to the
   corresponding Z3 implementations.

2. Review / update the tests for the Z3 indexes, with a goal of 100%
   coverage, as well as improving the "quality" of the tests.

3. Re-implement the Z2 indexes as thin wrappers around the Z3 versions,
   ensuring that all existing Z2 tests continue to pass.

This is somewhat the same pattern as Andreas used to merge the ZPT
implementations, I think.  This effort could be the focus of a sprint
team, if there was interest.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKL9sQ+gerLs4ltQ4RAksCAKDWRMu820PO+oRQNun0VC1LrFjvOACfU6pE
uVtYzoliBjQgI5gark3ALTw=
=+WAL
-----END PGP SIGNATURE-----