[ZODB-Dev] Unofficial AdaptableStorage Wiki

[Phillip J. Eby]

At 05:50 PM 2/20/03 -0500, Shane Hathaway wrote:
>Now let's say the keychain (45,) refers to a ZCatalog object.  Normally, 
>objects inside the catalog would get a similar keychain: (46,), (47,), 
>etc.  But we want objects contained by the catalog to be stored in ZODB, 
>not the relational database.  To make this happen, the ZCatalog domain 
>mapper generates two-item tuples for subobject keychains.  The second item 
>would be an 8-byte OID string.  Subobjects of the catalog at (45,) use 
>keychains that start with 45:
>
>(45, '\0\0\0\0\0\0.b')
>(45, '\0\0\0\0\0\0.c')
>(45, '\0\0\0\0\0\0.d')
>
>and so on.  The meaning of each item in the keychain is encapsulated in a 
>domain mapper.
>
>This means that a single database can cross domains and "mount" data 
>sources transparently.  Unfortunately, nothing in AdaptableStorage takes 
>advantage of this yet, except for the domain mapper at the root, which 
>doesn't do much to clarify it.
>
>Another reason you might use a domain mapper is to change classification 
>policies for branches of the database.  You might decide that new .txt 
>files on the filesystem usually map to OFS.Files, but when inside CMF 
>sites, new .txt files map to CMF Documents.  You can do this without 
>changing CMF (since it's probably not part of CMF's mission), and without 
>creating a mount point for every CMF site.
>
>The Zope 2 / filesystem mapper uses paths as the first item in the 
>keychain.  It was tempting to use Zope's physical paths as keychains, but 
>I decided that would dilute the concept of domain mappers.  It would be 
>like calling every container a mount point.

Makes sense.  It would also add unnecessary traversal overhead.

It sounds to me like the principal difference between what you're doing 
here and what I'm doing in PEAK, is that you want all oids (keychains) to 
be de-referenceable through a single point of access.  In PEAK, I assume 
that you give an oid directly to a DM that knows what it is.

So, let's take your example above.  If I have the ZCatalog stored in 
Postgres, its key is 45, accessed from the DM that does that 
processing.  That DM would be responsible for knowing that (for example) 
the BTree objects of the ZCatalog's state should be retrieved from 
ZODB.  ZODB then has its own oid, the 8 byte string, which would have to 
have been stored with the ZCatalog's state.

It seems like the idea in AdaptableStorage is that you want to decouple 
that knowledge, so that a particular serializer doesn't have to know who to 
give a particular reference to.  For my target application realm, this 
isn't particularly useful because in practice, legacy databases won't store 
arbitrary keys anyway.  :)  To put it another way, if I understand 
correctly, the only time a keychain is helpful as an object reference is if 
you can store one that has arbitrary length and contents.  Is that correct?


>>By way of contrast, EntityDM's in PEAK simply use an opaque value for a 
>>_p_oid, and if a particular serialization implementation needs a path, 
>>it's certainly welcome to use, say, a tuple of keys of some kind for that 
>>purpose, but the thing that manages the objects' state and transactions, 
>>doesn't really *care* about the key format.  So I'm curious whether I 
>>should view the keychain idea in AdaptableStorage as a cool enhancement 
>>or as meaningless extra overhead.  ;)  And I won't know that until I 
>>understand what it is and what it's for.  :)
>
>I think keychains are quite valuable, but since I haven't actually made 
>much use of them yet, I don't have evidence to back it up.  I'm very open 
>to better names for "keychain" and "domain mapper".  I'm also open to the 
>idea that keychains and domain mappers should be scrapped. :-)

They sound to me like things that are useful in the context of Zope and 
pickle-ish backends.  That is, if you're using an RDBMS in The Way God 
Intended (i.e. using columns for domain attributes as opposed to shoving 
pickles into it), then it doesn't seem to buy you much.  In order to set up 
the database, you have to know what the keys are going to be, and what 
object will be responsible for dereferencing those keys, so why worry about 
having a dynamic mapping from the key?  This seems much more meaningful in 
the website construction/management sphere (with the occasional jump out to 
an RDBMS for certain kinds of objects) than in the realm of web-based 
applications that are more RDBMS-focused.

But of course it may be that I have missed your point entirely.  :)

Looking at the AdaptableStorage wiki has given me some ideas for possibly 
adding "aspect serializers" to PEAK in some form.  It may be that I have 
not understood what you actually mean by that, but I know what *I* mean by 
that, and it's useful, so it was good to get the idea, even if it's not 
your idea, if you know what I mean.  :)