[Zope3-Users] zodb objects backup

[Gary Poster]

On Feb 27, 2006, at 12:03 AM, Shaun Cutts wrote:

>
>
>> From: Tom Dossis [mailto:td at yoma.com.au]
>> Shaun Cutts wrote:
>>> But with replication, this issue is taken care of. (Too bad
> replication
>>> isn't part of the core functionality....)
>>
>> Maybe soon..
>>    http://hathawaymix.org/Software/PGStorage
>>
> Great!
>
> Another potential issue on the scalability front I had forgotten about
> earlier:
>
> How do BTrees perform under lots of concurrent updates? (I know  
> this is
> a tricky one, as implementations can get pretty complex to deal with
> this.)

You've got a lot of questions. :-)  I only have time to be quick, and  
hope that it is somewhat helpful.  In that vein:

Pretty well.  It has custom code to try to resolve conflicts.  They  
are designed to handle it pretty well.  That said, even with the  
special resolution code, catalogs (heavy BTree users) are often  
hotspots for conflicts, and sometimes get special treatment to  
serialize updates.

> I note that
> http://www.zope.org/Wikis/ZODB/FrontPage/guide/ 
> node6.html#SECTION0006300
> 00000000000000
>
> Says that "As with a Python dictionary or list, you should not  
> mutate a
> BTree-based data structure while iterating over it".
>
> Does this apply only to thread-local modifications or to any
> modification by anyone else?

thread-local (or more correctly, connection-local)

> Ie, are BTrees "versioned" as the ZODB
> is... if I'm iterating over a BTree in my process (in ZEO, say), and
> another process modifies the BTree, does that sometimes show up in my
> copy, or only after commit?

Only after commit.

> Also, wrt "ConflictError" -- is the BTree considered one object, or  
> are
> the python objects (buckets, tree structure, ...) treated
> separately?....

separately.

> In general, are the BTrees just written "naively" on top of ZODB,  
> or do
> they interact in some special way with the storage?

Hm.  They are certainly not "naive" of the ZODB; they authors have  
deep knowledge of the ZODB, and take advantage of ZODB hooks (such as  
the conflict resolution) and behavior.

Gary