[Zope-dev] Re: More resilient indexes

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Roché Compaan wrote:
> Hi there
> 
> Are the current indexing solutions that we have in Zope 2 and 3 the best
> that we can have? Is there any way to improve the amount of concurrent
> indexing a field or text index can handle. The Zope 3 implementation
> doesn't look significantly different to the Zope 2 one in that it still
> uses a BTree for forward and reverse index.
> 
> I have a basic stress test where 10 concurrent threads index 10 random
> words in a FieldIndex and get too many conflict errors to even consider
> this a usable scalable indexing solution. I don't really want to index
> objects in another backend. I really would like to make the ZODB work
> for me here.
> 
> Are there solutions here? Can one employ some of the QueueCatalog
> conflict resolution strategies to make indexes more resilient? Or should
> one use some locking strategy instead?

Check out QueueCatalog, which batches up indexing changes for processing
within a single thread.

  svn://svn.zope.org/repos/main/Products.QueueCatalog



Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG7xSB+gerLs4ltQ4RAl+1AJ91FF672IEULtl4wMlSm+eoDIIaSgCfZJfP
vzzs3S+KsfEliNXdWWNos7E=
=3p6D
-----END PGP SIGNATURE-----