Bait: I am willing to SPEND MONEY to get this bastard solved. Summary of the situation so far: - Zope cannot deal with overlapping modifications to one object - That includes adding children to containers - The retry mechanism doesn't work - The user sees extremely messy errors - This seriously limits Zope's usability for write-intensive tasks like conferencing - Since all of my plans are for write-intensive tasks, that makes me uncomfortable I've spent my last 24 hours of work on Zope trying to figure out where to hack in any kind of method-aware queueing which could -- at its most simple -- simply not attempt to place more than one call to an object at once. From there, I'd be able to hack the routine for more subtlety. So far, no go. I'm just chasing my tail all around Publish, mapply and a few others, getting nowhere particularly fast. Well, I'm sick of whacking in calls to traceback.print_stack(), guys. I'd much rather get someone who knows what they're talking about to point out "oh, there's half a system in blah.py which needs 30 minutes of work to do what you need, and yeah we know retry is fundamentally broken". Or whatever. So, here goes bribery: If you're Australian and first to point out where I need to apply code, I'll buy you lunch next time I'm in your home city. If you're outside Australia, I'll have Amazon ship a copy of Slaughtermatic to your doorstep. Just someone, please, for fuck's sake, help me solve this concurrency problem before I'm forced to abandon the ZODB and/or Zope entirely. Regards, Garth. -- <gtk@well.com>
Do the '404' errors you get contain the text 'the requested Zope resource' or something different? If it doesn't, then uncaught exceptions are propogating back to Medusa (probably from work() in HTTPServer.py).
- This seriously limits Zope's usability for write-intensive tasks like conferencing
Unless you run in single thread mode (*duck*) ___ // Zen (alias Stuart Bishop) Work: zen@cs.rmit.edu.au // E N Senior Systems Alchemist Play: zen@shangri-la.dropbear.id.au //__ Computer Science, RMIT WWW: http://www.cs.rmit.edu.au/~zen
Do the '404' errors you get contain the text 'the requested Zope resource' or something different?
I'd have to check. Mainly, I get spurious-looking "you didn't supply this property" errors, which someone (can't remember who) said was a bug.
- This seriously limits Zope's usability for write-intensive tasks like conferencing
Unless you run in single thread mode (*duck*)
I'm not hugely fond of that idea, no. :) Regards, Garth. -- <gtk@well.com>
At 06:39 AM 11/9/99 +1100, gtk wrote:
Bait: I am willing to SPEND MONEY to get this bastard solved.
Me too. That's why my company is paying DC for some consulting work in this area, although at this point just to discuss possible solutions to this and related subjects.
- The retry mechanism doesn't work - The user sees extremely messy errors
These sounds to me like bug(s), which I'm sure the DC folks will fix once they're sufficiently pinned down.
- This seriously limits Zope's usability for write-intensive tasks like conferencing - Since all of my plans are for write-intensive tasks, that makes me uncomfortable
Ditto. Ty Sarna and I have done some extensive analysis and testing of ZODB and have found pretty much the same. Any write-intensive data structure will have conflict issues. We came up with several ideas for solutions, but they all require changes to the ZODB architecture, either to allow a lock to be placed on a persistent object (which would be held to the end of the transaction, and guarantee that the in-memory version for that thread was up-to-date), to delay certain operations until a different transaction phase, or to allow objects to be given a chance to resolve conflicts by themselves (e.g. serialize writes that don't really conflict.) This is why we're consulting with DC; to find out which approaches are most feasible. (Side note: DC themselves have noted that currently they recommend that people use external databases for write-intensive objects.)
- Zope cannot deal with overlapping modifications to one object - That includes adding children to containers
Yep. This also includes Catalogs, unfortunately.
I've spent my last 24 hours of work on Zope trying to figure out where to hack in any kind of method-aware queueing which could -- at its most simple -- simply not attempt to place more than one call to an object at once. From there, I'd be able to hack the routine for more subtlety.
It's not quite that simple. Technically, you'd need all the fundamental ZODB interfaces (Connection, Database, and Storage) to support a lock() operation, which would store the appropriate information at that level, and be removed upon termination (successful or otherwise) of the transaction. It's also probable that you'd need deadlock checking.
participants (3)
-
gtk -
Phillip J. Eby -
Stuart 'Zen' Bishop