Hi everyone. We are looking for some product to provide a cache shared between all the threads in Zope, and I think we could use the SharedResource (see http://www.dieter.handshake.de/pyprojects/zope/SharedResource.html) module for such a purpose. This cache will mantain objects built from an SQL database. However, we have some doubt regarding Zope transactions and cache management. Indeed, the commit/rollback process is ok with the SQL db: if the sql transaction fails, the whole zope transaction fails and nothing is stored anywhere. However, how could we manage this with SharedResource? Consider that, at first, we are not going to put zope objects inside such a cache. How could we remove such objects at the abort of the transaction? Or should the objects be put in the cache only at the end of the transaction itself? Any insight will be appreciated (and paid in Pizza credits when you will visit Pisa :=). Regards. Paolo -- Paolo Bizzarri - President - Icube S.r.l. Address: Via Ridolfi 15 - 56124 Pisa (PI), Italy E-mail: p.bizzarri@icube.it Web: http://www.icube.it Phone: (+39) 050 97 02 07 Fax: (+39) 050 31 36 588
Paolo Bizzarri wrote at 2004-7-23 16:48 +0200:
... transaction aware cache management ... We are looking for some product to provide a cache shared between all the threads in Zope, and I think we could use the SharedResource (see http://www.dieter.handshake.de/pyprojects/zope/SharedResource.html) module for such a purpose. This cache will mantain objects built from an SQL database.
However, we have some doubt regarding Zope transactions and cache management. ...
Your doubts may be justified. "SharedResource" is not transaction aware. When a transaction puts something into the resource and later aborts, it is is still in the "SharedResource". Moreover, other transactions can see it before the changing transaction commits. For many applications, this is not very problematic -- e.g. when you cache the result of SQL methods. Usually, such SQL methods put only committed changes into the cache (however, it is easy to construct application for which even this is not true). When you really need transaction awareness, it is probably best to put your cache into a mounted ZODB backed by e.g. a RAM storage -- very similar to how Zope manages sessions nowadays. You can even use Zope's "temp_folder" for this purpose (put an appropriate BTree based (!) cache manacher into this folder). -- Dieter
On Friday 23 July 2004 20:52, Dieter Maurer wrote:
Paolo Bizzarri wrote at 2004-7-23 16:48 +0200:
... transaction aware cache management ... We are looking for some product to provide a cache shared between all the threads in Zope, and I think we could use the SharedResource (see http://www.dieter.handshake.de/pyprojects/zope/SharedResource.html) module for such a purpose. This cache will mantain objects built from an SQL database.
However, we have some doubt regarding Zope transactions and cache management. ...
Your doubts may be justified.
"SharedResource" is not transaction aware.
When a transaction puts something into the resource and later aborts, it is is still in the "SharedResource".
Moreover, other transactions can see it before the changing transaction commits.
For many applications, this is not very problematic -- e.g. when you cache the result of SQL methods. Usually, such SQL methods put only committed changes into the cache (however, it is easy to construct application for which even this is not true).
When you really need transaction awareness, it is probably best to put your cache into a mounted ZODB backed by e.g. a RAM storage -- very similar to how Zope manages sessions nowadays. You can even use Zope's "temp_folder" for this purpose (put an appropriate BTree based (!) cache manacher into this folder).
We thought to use an object inside a temp_folder, containing, inside, whatever suitable structure (a dictionary, a BTree, etc.). However, we thought that since temp_folder uses the LowConflictResolution, which does not support the conflict resolution, this was unsuitable. Are we wrong? -- Paolo Bizzarri - President - Icube S.r.l. Address: Via Ridolfi 15 - 56124 Pisa (PI), Italy E-mail: p.bizzarri@icube.it Web: http://www.icube.it Phone: (+39) 050 97 02 07 Fax: (+39) 050 31 36 588
Paolo Bizzarri wrote at 2004-7-27 09:58 +0200:
... We thought to use an object inside a temp_folder, containing, inside, whatever suitable structure (a dictionary, a BTree, etc.).
However, we thought that since temp_folder uses the LowConflictResolution, which does not support the conflict resolution, this was unsuitable.
"temp_folder" no longer uses a "low conflict" connection. However, you are right that it supports only limited conflict resolution: the underlaying RAM storage maintains just two versions per object, the current and the previous version. Thus, you get just one level of conflict resolution (but not several). Sessions work not too bad with "temp_folder". Maybe, your application can live with it, too, when carefully designed to avoid conflicts as much as possible. -- Dieter
participants (2)
-
Dieter Maurer -
Paolo Bizzarri