[Zope] externalmount & zeo voodoo

[Chris McDonough]

> can you elaborate more on using lots of subfolders and BTrees

If you're doing lots of writes, the bottom line is that you want to make
sure you're not writing to the same object simultaneously in two threads.
The Zope objectmanager design underneath a Folder object is a little crappy
right now because it needs to rewrite an _objects list attached to the
folder whenever something is added to the folder.  When two people are
adding lots of stuff into a single folder (each in a different thread),
conflict errors can occur.  So if you're adding a bunch of stuff to your
site, you want to minimize the chance that two items will be added to the
same folder at the same time.  Using *more* folders can help minimize the
chance that two people will be adding stuff into a single one at the same
time.

Most people don't use BTrees directly in their app code, but BTrees have the
property of consisting of many separate persistent nodes (especially the new
BTree implementation which the yet-to-be-released Zope 2.3.1b2 will ship
with).  Building data structures on top of it can have the same effect as
making sure that you subdivide your data into subfolders, basically.  There
is an implementation of a folder (the BTree folder) that has this property,
and which is presumably better for adding many objects simultaneously.  Use
of BTrees and other conflict-resistant data structs for standard basic
objects (like folders) is an area for improvement in Zope... there's not
much you can do about it as a "content manager", but you can try to be smart
about it when you're developing apps in Python.

If simultaneous writes *have* to be to a single object (such as a counter,
or a poll), BTrees or subfolders don't help you.  It *would* help to use
app-level conflict resolution in this case on the objects which hold the
counts.  (It wouldn't just help, it would actually subvert the problem
entirely!)

> i see this happening when i enabled poll.  when there's a lot of users
> clicking, zope restarts.  the restarts stop when i took out poll.
>
> otherwise, zope has been solid like a rock

Huh.  Well, sounds like you've already narrowed it down some... I think
you're on the right track with the debug logging stuff... let us know!