[Zope-dev] Efficient and flexible object references

[Joachim Werner]

Hi!

some ideas ...

> As the list above stands now, I will either use a standard tree-based URL
> uid and try to build my own optimized caching mechanism (ouch) or go down
> the treacherous and very dangerous path (for the future) of attempting to
> make the ZCatalog do what I want no matter what.  I suppose my first step
> there would be to see if I can find a way to allow reindexing of a catalog
> without changing RIDs...  I can feel the frowns from here...

With the catalog approach, how exactly are planning to do it? If I get it
right, you store the RID of the related object (let's say the Composer) with
your object (let's say the composition) and if you need the information, you
get the related data from the catalog. You do this because you don't want to
reference the object itself. It would have to be woken up to get the
metadata (Name, date of birth whatever) from it.

One thing that comes to my mind is cataloging a unique id (or the physical
path, which is the standard id in the catalog anyway) with the entry and use
that one as the key. I don't know if it is much slower than retrieving the
catalog entry by its RID, and your custom id would not change when the
object is re-cataloged.

> On a simple level, I need the kind of referencing I describe for
connecting
> people objects as creators to other objects; for connecting any object to
> another (particularly topics) in a "describes" relationship; for
connecting
> same-class objects in a parent-child relationship; and other similar
tasks.
> (Obviously, I'm coming from a bit of a RDBM background on this but I'm
> enjoying the better modeling possible with the ZODB, among other things.)

> When a composition object displays, for instance, it needs to both know
the
> name and address of all of it's creators, ideally without waking up the
> creators yet.  Similarly, a person needs to know back links--what objects
> claim me as a creator?  Rather than caching a page or an object, I have
> decided it will be best to cache the relationships and metadata somehow.

All that sounds very much like you maybe need an RDBM. After all, you have a
lot of relations! If you still want to have "real" objects in Zope instead
of tabular data, you might use the DBObjects framework. I have no
benchmarks, but I guess retrieving your data from PostgreSQL should be quite
as efficient as using the catalog. With DBObjects, you normally have real
persistent objects that have their data stored in PostgreSQL but can be used
from Zope like a ZODB object. The good thing in your case is that you could
then write your own methods like "getMyComposers()" that do the hard work in
SQL and return the results as attributes of the object.

I am not really sure what approach is the most efficient, and whether you
really need massive caching or just a very efficient just-in-time retrieval
and then maybe late caching of the rendered pages using the available
caching tools. I guess you'll need to test that with actual data ...

Cheers

Joachim