[Zope] Thread-Safety and ZODB Was: How/Where to store user specific data?

[Chris McDonough]

Thomas Guettler wrote:
>>>I am developing a product in python (Multible Choice Test) and want to
>>>store some data for every user (Results of the tests).
>>>
>>>I see two ways: 
>>> Add data to the user object in acl_users 
>>>
>>God, no.  Won't scale, depends on which acl_user is in use.


This could scale, given an efficient datastructure for holding user data.

>>>  or
>>> Store data in a hash with the user-id as key
>>>
>>Won't survive threading.  You could use some sort of Session
>>product, but if this needs to be permanent, that is also the wrong
>>solution.

This isn't true.

> Why? According to the ZopeBook ZODB is thread-safe. But I don't know
> how this thread-safety is implemented. How can I synchronise things
> like this:
> 
> if(self.i<100): i=i+1

Do you mean:

if self.i < 100:
    self.i = self.i + 1

If so, you neednt worry about synchronization because the ZODB handles 
it for you.  So you *can* do this without fear of inconsistency.

However!

1.  The default Zope storage mechanism (FileStorage) may be
     inappropriate for this sort of "counter" object because
     simple counters will tend to cause the Data.fs file to
     grow significantly for every transaction

2.  An object that is a "hotspot" (one that is bound to be
     changed a lot and runs a high risk of being changed
     simultaenously) is at risk of generating lots of
     conflict errors.  The ZODB uses optimistic concurrency
     instead of locks, and conflict errors are a strategy
     for dealing with simultaneous writes.  In Zope,
     these are automatically dealt with by a request retry
     mechanism, but it has the potential to slow the system
     down.  As the counter is a potential hotspot, you need
     to be aware of the ZODB concurrency control strategy
     here.


A reasonable solution to both problems is to use a special class as a 
counter that can perform app-level conflict resolution.  There is such a 
class in Zope already in lib/python/BTrees/Length.py.  You can use it 
like so:

from BTrees.Length import Length
from Acquisition import Implicit
from Persistence import Persistent

class Foo(Persistent, Implicit):
     def __init__(self):
         self.counter = Length()

     def incr(self):
         self.counter.change(1)

The effect that this has follows:

- A call to incr will increment the counter object by one.  This will 
still grow the ZODB, but only by the size of the Length object (not by 
the size of the Foo object).

- A call to incr will never generate a conflict error because the Length 
class has app-level conflict resolution.

For more info on both topics, see 
http://www.zope.org/Documentation/ZDG/Persistence.dtml

> If self.i mustnot be bigger than 100 this could be a problem.

No.

>>This sounds like a job for a database.  
>>
> 
> Up to now I was very impressed by ZODB. Would like to use it.

The ZODB is a database and there's no reason to not use it fior this 
purpose if you understand the constraints.
-- 
Chris McDonough                    Zope Corporation
http://www.zope.org             http://www.zope.com
"Killing hundreds of birds with thousands of stones"