[ZODB-Dev] Re: so your coming from the rdbms world tutorial?

[Roché Compaan]

On Sat, 2008-03-22 at 15:49 -0400, Sean Allen wrote:
> >> Ha! Ok.
> >>
> >> Questions then from there.
> >>
> >> Why not store each object type,
> >>
> >> customer, order etc in their own folders?
> >
> > You could, and I understand that you want to do so coming from a RDBMS
> > world, but if you are working with objects then you have the luxury of
> > organising content in a way that is closer to reality. I think it is
> > rather convenient to store a customer's orders inside it.
> >
> 
> I can see the convenience. Ok, so here is a big question...
> Assuming you did store each object in its own folder and you had
> 
> Customer
>      has many orders
> 
> At what point do you have to hook into zodb so that if you say did this:
> 
> customer = Customer( ... )
> customer.addOrder( Order( ... ) )
> 
> that the order in question when customer is saved, gets saved to its  
> own folder
> and then customer gets a reference to that now saved persistent order
> before it itself is saved.

You first create the order inside the orders folder, then you reference
it from the customer:

order = Order()
orderfolder['order123'] = order
# assuming you have a folder for orders in each customer, the following
# will create a reference since the above code already persisted the
# object.
customer.orders['order123'] = order

I don't see why you need the reference though. Why not just create the
order directly in the order folder for the relevant customer:

order = Order()
customer.orders['order123'] = order

> 
> > There is another good reason - it makes a lot more sense to distribute
> > object creation to different containers in the ZODB to avoid write
> > conflict errors.
> 
> I think I understand this but can you elaborate?
> 
> By 'distribute object creation to different containers' do you mean  
> store each
> object in own folders?

No that would lead to an unnecessary proliferation of folders and it
wouldn't actually remove conflict errors.

Conflicts errors occur when two concurrent transactions modify the same
attribute. When you are putting objects in a folder you are essentially
modifying the same attribute (unless you are using a btree). A good
application design strategy for the ZODB is to ensure that concurrent
transactions are modifying attributes on different objects. If you are
putting all your orders in one folder you are increasing the chances of
conflict errors. 

Like I mentioned above you could use a btree folder since it has a
fairly good conflict resolution strategy. But given a high enough
insertion rate even a btree will not protect you against conflict
errors. I would recommend you create orders inside a btree-based order
folder inside each customer. This will significantly reduce the
likeliness of conflict errors.

> 
> >
> >
> >>
> >> is there a reason to have a folder that contains a more complete  
> >> graph?
> >> what advantages would that have? speed of access?
> >
> > What do you mean with "more complete graph"?
> 
> storing line items inside  orders inside customers type scenario

Access and insertion times for items in folders with fewer items are
faster and like I mentioned above, distributing objects writes reduces
conflict errors 

> 
> >
> >
> >>
> >> --
> >>
> >> the database i am looking at moving over has
> >>
> >> 1.2 million entries in a transactions table
> >> 980,000 orders
> >> 775,000 customers
> >> 1.5 million order items
> >
> > I think these numbers are quite manageable. But think carefully what
> > kind of queries you want to do on the data. You have a very rich query
> > language in SQL that allows the construction of complex queries and it
> > will come naturally to you. To do the same in the ZODB will take  
> > careful
> > planning.
> 
> Are there any mistakes that people usually make when doing this sort
> of mental context switch that you can make me aware of now?
> 
> Any good reading on the subject?
> 
> Either there isnt a ton of information on this on the net or my google  
> skills
> are slipping.

The most valuable resource is the mailing list. Unfortunately nobody has
documented their experience elsewhere before.


-- 
Roché Compaan
Upfront Systems                   http://www.upfrontsystems.co.za