-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Jürgen Kartnaller wrote:
We also run into this kind of problems.
The only save way to solve it was to serialize our index updates.
We do this by using one zope instance which runs only a lovely.remotetask task service. This service contains an indexing task. When indexing is needed we just create a new indexing job which is stored in the job list of the remote task service.
That is exactly how QueueCatalog functions: it "batches" expensive / conflict-prone indexing operations into jobs which are handled later, in a serialized way.
This solves two problems :
- conflicts - low speed because of complex index value calculations
Btw the setup for this application contains 18 zope's each zope running one thread. 17 zope's are used to handle browser requests and one is used to handle the remote tasks (not only indexing tasks).
How many of the front-end servers do writes? Or are all your visitors potential writers? Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGYOWr+gerLs4ltQ4RAmpyAJwPxdx+ob46RgvasStZwoPWQQTa8gCdG/+t 05J+ocnTgZVOnf7MvLw3oGw= =kyUn -----END PGP SIGNATURE-----
Tres Seaver wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Jürgen Kartnaller wrote:
We also run into this kind of problems.
The only save way to solve it was to serialize our index updates.
We do this by using one zope instance which runs only a lovely.remotetask task service. This service contains an indexing task. When indexing is needed we just create a new indexing job which is stored in the job list of the remote task service.
That is exactly how QueueCatalog functions: it "batches" expensive / conflict-prone indexing operations into jobs which are handled later, in a serialized way.
This solves two problems :
- conflicts - low speed because of complex index value calculations
Btw the setup for this application contains 18 zope's each zope running one thread. 17 zope's are used to handle browser requests and one is used to handle the remote tasks (not only indexing tasks).
How many of the front-end servers do writes? Or are all your visitors potential writers?
All servers do writes.
Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFGYOWr+gerLs4ltQ4RAmpyAJwPxdx+ob46RgvasStZwoPWQQTa8gCdG/+t 05J+ocnTgZVOnf7MvLw3oGw= =kyUn -----END PGP SIGNATURE----- _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
participants (2)
-
Jürgen Kartnaller -
Tres Seaver