[Zope-dev] Efficient and flexible object references

[Steve Alexander]

Gary & Karyn wrote:
 >
 >     * moreover, some caching of referenced objects' information, much
 >       like Catalog metadata, will be necessary.

Do you mean keeping a persistent cache of this data along with your objects?

You could consider a per-transaction cache of data, as used in
ZPatterns, so that the data gleaned from related objects is only
calculated once per transaction, but, you don't need to bother with
cache invalidation messages, as the data is discarded at transaction
boundaries.


Whether you want persistent cacheing or per-transaction cacheing depends
on the ratio of writes and updates to reads in your application.

Obviously, if you are doing a lot of persistent caching, you'll need to
do a lot of invalidating on each change.

A third option is to use some sort of in-memory cache that doesn't rely
on _v_ attributes. The RAM Cache from the StandardCacheManagers product
does this by storing the cached data in a dictionary that is in an
attribute of a module.

If you go down this path, you'll need to be careful about read/write
locking of your cache.


As to where to store your objects, and how to relate them to each other:
do you need to store the objects in particular places in your ZODB? If
not, you could use the ZPatterns approach of storing all objects of a
particular type in a Rack. Thus, you'd have racks for Creators, for
Instruments, for Compositions, and so forth. A Rack is a bit like a
database table.

Within a Rack, each object has a string id. This is because when a Rack
stores its data persistently, it uses an OO BTree.

When you store a reference to object B from object A, if the type of
reference is fixed in your application, you need only store the string
id of object B, as the rack to use will be evident from the type of
reference.
Thus, if object A is a Composition, and object B is a Creator, then
object A can have an attribute my_creator which has the value
"uid000023184982". Object A will know that my_creator is to be looked up
in the Creators Rack.

I've been playing with the idea of a new kind of Rack that uses an IO
BTree to store its objects. You'll be able to access the objects with an
integer id, or as normal by string id, using a standard convention that 
applies to every object in the Rack. An example of such a convention is 
"creator-%08d" % id  where an object with id 12 would also have the 
string id "creator-00000012".

Another advantage of the Rack is that it makes it easy to convert your 
application to use an RDBMS at some later point.

If you want to persue this Rack stuff further, you'll also want to know 
about Specialists. When you know about Specialists, you'll want to know 
about SkinScript.

Also, this thread would best be moved to the zpatterns list: 
zpatterns@eby-sarna.com. Take a look at http://eby-sarna.com for signup 
instructions.

--
Steve Alexander
Software Engineer
Cat-Box limited