[Zope] Thread-Safety and ZODB Was: How/Where to store user specific data?
Chris McDonough
chrism@zope.com
Wed, 22 Aug 2001 08:09:28 -0400
Thomas Guettler wrote:
>>>I am developing a product in python (Multible Choice Test) and want to
>>>store some data for every user (Results of the tests).
>>>
>>>I see two ways:
>>> Add data to the user object in acl_users
>>>
>>God, no. Won't scale, depends on which acl_user is in use.
This could scale, given an efficient datastructure for holding user data.
>>> or
>>> Store data in a hash with the user-id as key
>>>
>>Won't survive threading. You could use some sort of Session
>>product, but if this needs to be permanent, that is also the wrong
>>solution.
This isn't true.
> Why? According to the ZopeBook ZODB is thread-safe. But I don't know
> how this thread-safety is implemented. How can I synchronise things
> like this:
>
> if(self.i<100): i=i+1
Do you mean:
if self.i < 100:
self.i = self.i + 1
If so, you neednt worry about synchronization because the ZODB handles
it for you. So you *can* do this without fear of inconsistency.
However!
1. The default Zope storage mechanism (FileStorage) may be
inappropriate for this sort of "counter" object because
simple counters will tend to cause the Data.fs file to
grow significantly for every transaction
2. An object that is a "hotspot" (one that is bound to be
changed a lot and runs a high risk of being changed
simultaenously) is at risk of generating lots of
conflict errors. The ZODB uses optimistic concurrency
instead of locks, and conflict errors are a strategy
for dealing with simultaneous writes. In Zope,
these are automatically dealt with by a request retry
mechanism, but it has the potential to slow the system
down. As the counter is a potential hotspot, you need
to be aware of the ZODB concurrency control strategy
here.
A reasonable solution to both problems is to use a special class as a
counter that can perform app-level conflict resolution. There is such a
class in Zope already in lib/python/BTrees/Length.py. You can use it
like so:
from BTrees.Length import Length
from Acquisition import Implicit
from Persistence import Persistent
class Foo(Persistent, Implicit):
def __init__(self):
self.counter = Length()
def incr(self):
self.counter.change(1)
The effect that this has follows:
- A call to incr will increment the counter object by one. This will
still grow the ZODB, but only by the size of the Length object (not by
the size of the Foo object).
- A call to incr will never generate a conflict error because the Length
class has app-level conflict resolution.
For more info on both topics, see
http://www.zope.org/Documentation/ZDG/Persistence.dtml
> If self.i mustnot be bigger than 100 this could be a problem.
No.
>>This sounds like a job for a database.
>>
>
> Up to now I was very impressed by ZODB. Would like to use it.
The ZODB is a database and there's no reason to not use it fior this
purpose if you understand the constraints.
--
Chris McDonough Zope Corporation
http://www.zope.org http://www.zope.com
"Killing hundreds of birds with thousands of stones"