[ZODB-Dev] Advice needed

[Roché Compaan]

On Fri, 2006-06-23 at 22:03 +0200, Dieter Maurer wrote:
> Adam Groszer wrote at 2006-6-23 14:27 +0200:
> > ...
> >Some fears they are having and I can't find unambiguous information:
> >- Is ZODB a good choice for this app?
> 
> It depends...
> At least careful design is necessary!
> 
> The most problematic aspects of the ZODB are write conflicts.
> 
>    When two concurrent transactions modify the same object
>    a write conflict will occur unless the object provides
>    application specific conflict resolution (which might
>    resolve some conflicts).
> 
>    This behaviour is especially serious when you have
>    expensive (long running) transactions. The longer
>    the transaction runs the higher is the risk that
>    it interferes with another transaction and
>    the higher is the cost of the resulting conflict.
> 
>    This means that you must carefully design your system
>    to reduce the risk of write conflicts.
> 
>    You can (and should!) for example use workflow to
>    prevent that the same application object (read "document" in your
>    case) can be modified by concurrent transactions.
>    You are interested in this for other reasons as well
>    (you do not want to wipe out the work of a colleague by overwriting
>    his changes).
>    This can clear this side (application objects) of the front.
> 
>    However, there are also global objects which can be
>    modified concurrently. The most prominent example:
>    catalog data structures (speak "indexes").
>    They do use application specific conflict resolution -- but
>    it is often not good enough...
>    For the catalog, you can move out indexing operations to
>    a separate thread (done e.g. by the "QueueCatalog" Zope product).
>    This considerably reduces write conflicts at the cost that
>    indexing operations are no longer inline but lack a bit behind.

I am curious what other strategies besides QueueCatalog you employ? Do
you ever use multiple backends for your apps? How do you decide that
this data belongs in a relational backend? How structured must the data
be, or how many records must be written how often?

I find most data is highly structured (fixed schema), but this doesn't
make me choose an RDMBS - the frequency of writes, concurrency and
record volume does.

-- 
Roché Compaan
Upfront Systems                   http://www.upfrontsystems.co.za