[Zope-CMF] Bulk Indexing

[Dieter Maurer]

Shane Landrum wrote at 2004-1-23 09:50 -0500:
> ...
>It's quite possible that I'm dealing with a unique situation here.
>We have 7 large Zope servers talking to a very large ZEO server,
>and unusually for lots of Zope installs, we do a lot of database
>writes, more or less all the time. We have an elaborate automatic
>news feed system that's always dumping new content into the database,
>an automatic workflow approval system for some of that new content, 
>and a staff of 2-15 editorial people on the system at any one 
>time, poking and prodding at content.  As a result, it's 
>relatively likely that at least one object out of several hundred 
>thousand is going to be being worked on when the reindexing is 
>done; hence, ConflictErrors. 

We have a similar scenario:

  We use Shane's "QueueCatalog" (an improved version of it)
  to perform indexing in a separate process (reducing conflict
  probability in the global indexing structures)
  and my "No more ReadConflict" patch to get rid of
  "ReadConflictError"s (with Zope 2.7, we will be using
  a pre-implementation of Jeremy's MVCC proposal instead of
  my patch)

    <http://www.dieter.handshake.de/pyprojects/zope>


-- 
Dieter