Re: [Zope] externalmount & zeo voodoo
It sounds like you have two issues:
1. you need better write performance. this can be done by minimizing conflicterrors.
2. your zope is apparently unstable (which may have nothing to do with
the
number of conflict errors you're getting)
Issue #1 is addressable by new application level conflict resolution code added to ZODB which should be out in 2.3.1b2 (the release of which is currently pending on new catalog code to pass all of its unit and performance tests). You can read about it at http://www.zope.org/Members/jim/ZODB/ApplicationLevelConflictResolution. Other complementary ways to deal with this are intelligent coding (don't do writes when you don't have to, don't modify objects that you don't need to), appropriate data structuring (use lots of subfolders in objectmanagers, use other partitioned data structures like BTrees when possible), and
can you elaborate more on using lots of subfolders and BTrees knowing
when to punt and use a relational database that uses locking instead of optimistic concurrency like the ZODB. There's no cookie-cutter solution... hopefully when folks get used to the new app-level conflict resolution, they'll make use of it in their products to increase write performance and minimize conflicts. I'm going to try this for the next sessioning release. The current generation of Zope products know nothing about conflict resolution, so it'll probably take a while for it to become mainstream.
Issue #2 is separate... you should turn on request logging (see the -M
i see this happening when i enabled poll. when there's a lot of users clicking, zope restarts. the restarts stop when i took out poll. otherwise, zope has been solid like a rock
switch in z2.py) and use the script at http://www.zope.org/Members/mcdonc/HowTos/DEBUG-LOG (ignore the bit at the top about inserting stuff into z2.py, it's been superseded by the -M switch) to figure out what requests aren't returning from a method call. With that info, you should hopefully be able to establish a pattern and isolate the problem. i'll try and enable poll and start zope with -M
thanks _______________________________________________
Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
can you elaborate more on using lots of subfolders and BTrees
If you're doing lots of writes, the bottom line is that you want to make sure you're not writing to the same object simultaneously in two threads. The Zope objectmanager design underneath a Folder object is a little crappy right now because it needs to rewrite an _objects list attached to the folder whenever something is added to the folder. When two people are adding lots of stuff into a single folder (each in a different thread), conflict errors can occur. So if you're adding a bunch of stuff to your site, you want to minimize the chance that two items will be added to the same folder at the same time. Using *more* folders can help minimize the chance that two people will be adding stuff into a single one at the same time. Most people don't use BTrees directly in their app code, but BTrees have the property of consisting of many separate persistent nodes (especially the new BTree implementation which the yet-to-be-released Zope 2.3.1b2 will ship with). Building data structures on top of it can have the same effect as making sure that you subdivide your data into subfolders, basically. There is an implementation of a folder (the BTree folder) that has this property, and which is presumably better for adding many objects simultaneously. Use of BTrees and other conflict-resistant data structs for standard basic objects (like folders) is an area for improvement in Zope... there's not much you can do about it as a "content manager", but you can try to be smart about it when you're developing apps in Python. If simultaneous writes *have* to be to a single object (such as a counter, or a poll), BTrees or subfolders don't help you. It *would* help to use app-level conflict resolution in this case on the objects which hold the counts. (It wouldn't just help, it would actually subvert the problem entirely!)
i see this happening when i enabled poll. when there's a lot of users clicking, zope restarts. the restarts stop when i took out poll.
otherwise, zope has been solid like a rock
Huh. Well, sounds like you've already narrowed it down some... I think you're on the right track with the debug logging stuff... let us know!
participants (2)
-
Bak ＠ kedai -
Chris McDonough