Il giorno ven, 11-02-2005 alle 14:31 +0100, martin f krafft ha scritto:

I want records created in the database outside of Zope to show up as objects in Zope automatically. The same applies to deletions.

I want objects created through Zope to create appropriate records in the RDB. The same applies to deletions.

I want to be able to edit objects and have the changes percolate to the database. I also want changes in the database to show up in the objects.

Is this what Ape is all about?

Yes, even if the default SQL storage layout doesn't help you on this, it's schema is minimalistic and take cares of storing objects in a tree-oriented way. The main part of the sql schema are: - the "classification" table used by the Classification gateway to store classification data, wich maps objoid (the one you find on the attr _p_oid of every Persistent obj) to a class name and a mapper name which knows how to serialize and deserialize that class; - oid_seq is the generator of all the objoids; - the "folder_items" table contains parent/child tree relationship and child id which reflect the one you see via ZMI; - the "object_data" table contains the value ofd the "main data attribute" for each obj, like the text for pt and dtml, etc.; - the "properies" table contains main properties of a zope object common to all SimpleItem descendants, like id and title. Depending on the mapper used, a class dedicated table is created to store properties found on object of that class (if you create a folder, you will get a "folder_properties"). The mapping between class names and such tables is maintained in the table "property_tables". - Last but not least, the "remainder" table contains base64 encoded pickles of all the members of an object not serialized by "previous" serializers. Remainder serializer has the lowest priority, so it runs at the very end of the serialization /deserialization process; There are other tables used to store user and security stuff. That said , normal serialization involves some operations: 1) The oid-generator is queried, unfortunately without so much "context", so you can't generate table dedicated oids wich matches classic RDB surrogate keys; 2) The classifier classifies the obj and stores this information using the classification gateway; 3) The composite mapper takes care of serializing the obj, delegating the operation to serializers depending their priority; 4) finally a blind serialized is called to check if all has been serialized; Obviously, the deserialization process happens reversed. What is not so obvious (even if it should, really) is that the main parts of the deserialization process are at storage level: - the creation of a stub instance (empty) of the class the classification has found; - the merge operation of that stub instance with the deserialized members; This means that is really possible to control the process of instance creation, and that is possible to create an instance having starting only by classic RDB data and developing deserializers that take care of filling missing parts, remembering that deserialized members really overwrite preexisting (class) members at instance construction.

...

I thought you were asking if you could make a relational table visible as Zope objects using APE, which it can theoretically do, and which I'm VERY interested in ;-)

Yes, this is what I am asking, I think. Are there any examples of this? Ape looks complicated and before diving into the theory, I'd rather explore it in action first.

So I want a way to associate a folderish Archetype (essentially just a folderish CMF type) with a RDB table so that an instance thereof obtains and manages some of its associated data from/in the RDB. At the same time, the instance should also feel like a regular ZODB object, which I can copy/paste, and in which I can store objects for management by the ObjectManager.

Will Ape help me here?

Yes! I had your same idea so i setup a test sandbox with Archetypes 1.3 head Ape, PostgreSQL and ArchGenXML. I've setup an ape mount point under a plone site declaring (in zope.conf) that i wanted to use a Products.CMFPlone.PloneFolder.PloneFolder instance to be used for the mount point, so i can use it normally like a plone folder (unfortunately i wasn't able to instantiate correctly an Archetypes based folder). Then i built ArchGenXML's sample project outline_od.xmi running AGXML with "--ape" on command line to have default AT Ape mappers (wich then make use of stuff in Archetypes.Apexxx.py module) built automatically and i've installed the product in the portal. Default AT serializers use a dedicated table named like the class to store AT fields values. Despite of what the Archetypes.Apexxx module says on the top, i found that the references are working correctly. After that i wanted to seed if thougths on the deserialization process where correct, so instead of creating and Outline instance via zmi, i used normal "psql" command line tool (for postgres). I filled values for all the main tables: classification, folder_items, object_data, properties and outline (all using oids generated by the oid_seq). Leaving ther other untouched I then have flushed the storage cache to keep it in synk with the sql backend After that i browsed to the folder where i attached my sql built instances (in folder_items) and voilà, the instances are there! The first time i edited them, i got the remaining data (reminder, security, ecc) filled on the RDB. This is just a test done with the default sql storage configuration, but it proofs that with a custom developed ape storage one can reach a real tight integration between ZODB and relational databases. However i still need how to answer to some of the problems that arised: - The cataloging machinery nor the uid stuff are correctly initialized. They gets initialized the first time i edited the instances. I still have to find a good way to trigger such inits... - On my SQL data i use one id sequence per table where i use surrogate keys. I need to find a good schema layout to merge such a classic layout with a more zope storage oriented one using maybe oids like "<table_name>id". It would be useful also to better control AT UID creation and attribution so that they meaningful at SQL level. This will permit the automatic "rendering" of table relations using special AT reference fields; The use of unrestricted Python as procedure language on RDB can help on the first issue, triggering XML-RPC or simply http calls on object creation... I hope this helps, i'm sorry for my bad english. I would be useful share our thougths in an appropriate area of some site...:) bye, Alberto