[ZODB-Dev] Advice needed

[Roché Compaan]

On Sat, 2006-06-24 at 20:54 +0200, Dieter Maurer wrote:
> Roché Compaan wrote at 2006-6-24 08:53 +0200:
> > ...
> >I am curious what other strategies besides QueueCatalog you employ?
> 
> Careful design with respect to the granularity and locality
> of persistent objects:
> 
>   Move groups of large and rarely used attributes out
>   into persistent subobjects.

Will this lead to smaller transaction sizes for objects that store large
attributes as persistent subobjects?

>   Move groups of small and often modified attributes out
>   into persistent subobjects.

And this will cause fewer conflict errors because distinct persistent
objects are being modified compared to a single object whose attributes
are being modified.

>   If some attributes are usually used together, keep
>   them in the same persistent subobject.

Which results in fewer persistent subobjects being modified, thus less
conflict errors.

>   Don't group large randomly accessed mass data into the same
>   persistent object (as this stupid ZCatalog metadata implementation
>   does). Use persistent wrappers instead.

This is interesting! ZCatalog metadata is stored in a IOBTree. Are you
saying the values in the IOBTree should be wrapped in a class that
subclasses Persistent, and not stored as raw values.

The ZCatalog and indexing in general seems to be the biggest problem
with ZODB applications. BTrees do not seem to fix the problem. This is a
huge headache for me. As an application developer I don't really want to
get involved with the software infrastructure, but problems like this
forces me too.

Can somebody describe a ZODB indexing implementation where an
application can index new objects concurrently? Given the above, this is
probably not a stupid question.

-- 
Roché Compaan
Upfront Systems                   http://www.upfrontsystems.co.za