[Zope] externalmount & zeo voodoo

[Chris McDonough]

> >BerkeleyStorage won't help with conflict errors (you will get the same as
> >under filestorage).  It only prevents database growth in comparison with
> >filestorage.
> >
>
> i thought Conflict Errors are inherent to ZODB.  but i guessed i'm wrong.
> it's Zope.

Nope, it's ZODB.  But ZODB != FileStorage.  BerkeleyStorage is at the same
level as FileStorage, and it has no better or worse performance than
FileStorage when dealing with conflict errors.  You can think of ZODB as the
technology that includes FileStorage and BerkeleyStorage as well as all the
other kinds of storages.  It's not incorrect to say that ConflictErrors are
inherent to ZODB, but it is incorrect to say that ConflictErrors are
inherent to FileStorage.

> how can we minimized Conflict Errors?  my site will soon be more write
> intensive; ie more user interactions with polls, comments, etc.  at the
> current rate, zope dies/restarts as and when it feels like.  use SQL as
the
> backend?

It sounds like you have two issues:

 1.  you need better write performance.  this can be done by minimizing
conflicterrors.

 2.  your zope is apparently unstable (which may have nothing to do with the
number of conflict errors you're getting)

Issue #1 is addressable by new application level conflict resolution code
added to ZODB which should be out in 2.3.1b2 (the release of which is
currently pending on new catalog code to pass all of its unit and
performance tests).  You can read about it at
http://www.zope.org/Members/jim/ZODB/ApplicationLevelConflictResolution.
Other complementary ways to deal with this are intelligent coding (don't do
writes when you don't have to, don't modify objects that you don't need to),
appropriate data structuring (use lots of subfolders in objectmanagers, use
other partitioned data structures like BTrees when possible), and knowing
when to punt and use a relational database that uses locking instead of
optimistic concurrency like the ZODB.  There's no cookie-cutter solution...
hopefully when folks get used to the new app-level conflict resolution,
they'll make use of it in their products to increase write performance and
minimize conflicts.  I'm going to try this for the next sessioning release.
The current generation of Zope products know nothing about conflict
resolution, so it'll probably take a while for it to become mainstream.

Issue #2 is separate... you should turn on request logging (see the -M
switch in z2.py) and use the script at
http://www.zope.org/Members/mcdonc/HowTos/DEBUG-LOG (ignore the bit at the
top about inserting stuff into z2.py, it's been superseded by the -M switch)
to figure out what requests aren't returning from a method call.  With that
info, you should hopefully be able to establish a pattern and isolate the
problem.

>
> >Hopefully soon I am going to try to get a new coresessiontracking release
> >out.  I am not working next week, but hopefully sometime in the next
couple
> >weeks.
> have a nice vacation!

Thanks!