- Zope - Zope lists

Re: [Zope] Re: Ape backwards
by azazel 11 Feb '05

11 Feb '05

Il giorno ven, 11-02-2005 alle 14:31 +0100, martin f krafft ha scritto: > I want records created in the database outside of Zope to show up as objects in Zope automatically. The same applies to deletions. > > I want objects created through Zope to create appropriate records in the RDB. The same applies to deletions. > > I want to be able to edit objects and have the changes percolate to the database. I also want changes in the database to show up in the objects. > > Is this what Ape is all about? > Yes, even if the default SQL storage layout doesn't help you on this, it's schema is minimalistic and take cares of storing objects in a tree-oriented way. The main part of the sql schema are: - the "classification" table used by the Classification gateway to store classification data, wich maps objoid (the one you find on the attr _p_oid of every Persistent obj) to a class name and a mapper name which knows how to serialize and deserialize that class; - oid_seq is the generator of all the objoids; - the "folder_items" table contains parent/child tree relationship and child id which reflect the one you see via ZMI; - the "object_data" table contains the value ofd the "main data attribute" for each obj, like the text for pt and dtml, etc.; - the "properies" table contains main properties of a zope object common to all SimpleItem descendants, like id and title. Depending on the mapper used, a class dedicated table is created to store properties found on object of that class (if you create a folder, you will get a "folder_properties"). The mapping between class names and such tables is maintained in the table "property_tables". - Last but not least, the "remainder" table contains base64 encoded pickles of all the members of an object not serialized by "previous" serializers. Remainder serializer has the lowest priority, so it runs at the very end of the serialization /deserialization process; There are other tables used to store user and security stuff. That said , normal serialization involves some operations: 1) The oid-generator is queried, unfortunately without so much "context", so you can't generate table dedicated oids wich matches classic RDB surrogate keys; 2) The classifier classifies the obj and stores this information using the classification gateway; 3) The composite mapper takes care of serializing the obj, delegating the operation to serializers depending their priority; 4) finally a blind serialized is called to check if all has been serialized; Obviously, the deserialization process happens reversed. What is not so obvious (even if it should, really) is that the main parts of the deserialization process are at storage level: - the creation of a stub instance (empty) of the class the classification has found; - the merge operation of that stub instance with the deserialized members; This means that is really possible to control the process of instance creation, and that is possible to create an instance having starting only by classic RDB data and developing deserializers that take care of filling missing parts, remembering that deserialized members really overwrite preexisting (class) members at instance construction. > > I thought you were asking if you could make a relational table visible as Zope objects using APE, which it can theoretically do, and which I'm VERY interested in ;-) > > Yes, this is what I am asking, I think. Are there any examples of this? Ape looks complicated and before diving into the theory, I'd rather explore it in action first. > So I want a way to associate a folderish Archetype (essentially just a folderish CMF type) with a RDB table so that an instance thereof obtains and manages some of its associated data from/in the RDB. At the same time, the instance should also feel like a regular ZODB object, which I can copy/paste, and in which I can store objects for management by the ObjectManager. > > Will Ape help me here? Yes! I had your same idea so i setup a test sandbox with Archetypes 1.3 head Ape, PostgreSQL and ArchGenXML. I've setup an ape mount point under a plone site declaring (in zope.conf) that i wanted to use a Products.CMFPlone.PloneFolder.PloneFolder instance to be used for the mount point, so i can use it normally like a plone folder (unfortunately i wasn't able to instantiate correctly an Archetypes based folder). Then i built ArchGenXML's sample project outline_od.xmi running AGXML with "--ape" on command line to have default AT Ape mappers (wich then make use of stuff in Archetypes.Apexxx.py module) built automatically and i've installed the product in the portal. Default AT serializers use a dedicated table named like the class to store AT fields values. Despite of what the Archetypes.Apexxx module says on the top, i found that the references are working correctly. After that i wanted to seed if thougths on the deserialization process where correct, so instead of creating and Outline instance via zmi, i used normal "psql" command line tool (for postgres). I filled values for all the main tables: classification, folder_items, object_data, properties and outline (all using oids generated by the oid_seq). Leaving ther other untouched I then have flushed the storage cache to keep it in synk with the sql backend After that i browsed to the folder where i attached my sql built instances (in folder_items) and voilà, the instances are there! The first time i edited them, i got the remaining data (reminder, security, ecc) filled on the RDB. This is just a test done with the default sql storage configuration, but it proofs that with a custom developed ape storage one can reach a real tight integration between ZODB and relational databases. However i still need how to answer to some of the problems that arised: - The cataloging machinery nor the uid stuff are correctly initialized. They gets initialized the first time i edited the instances. I still have to find a good way to trigger such inits... - On my SQL data i use one id sequence per table where i use surrogate keys. I need to find a good schema layout to merge such a classic layout with a more zope storage oriented one using maybe oids like "<table_name>id". It would be useful also to better control AT UID creation and attribution so that they meaningful at SQL level. This will permit the automatic "rendering" of table relations using special AT reference fields; The use of unrestricted Python as procedure language on RDB can help on the first issue, triggering XML-RPC or simply http calls on object creation... I hope this helps, i'm sorry for my bad english. I would be useful share our thougths in an appropriate area of some site...:) bye, Alberto

1 0

POSKeyError
by Kevin Carlson 11 Feb '05

11 Feb '05

Hi all, Whenever I try to reindex an index in the portal_catalog or delete the index, add it back and then reindex, I get the first error below. I ran fstest.py and there were no errors in the Data.fs file yet. I also ran fsrecover.py and there were zero bytes removed from the Data.fs file. Also, When I run fsrefs.py, I get the second bit of output below. You may have seen my recent posts about the BTreeFolder2 instance that stores the vast majority of the content for this application. It has been suggested that the BTreeFolder2 instance may have corrupt BTrees so I rebuilt those using Tres Seaver's method but the behavior stays the same. Any ideas on this? Exception Type POSKeyError Exception Value 00000000000cc49a Traceback (innermost last): * Module ZPublisher.Publish, line 101, in publish * Module ZPublisher.mapply, line 88, in mapply * Module ZPublisher.Publish, line 39, in call_object * Module Products.ZCatalog.ZCatalog, line 499, in manage_reindexIndex * Module Products.ZCatalog.ZCatalog, line 485, in reindexIndex * Module Products.CMFCore.CatalogTool, line 207, in catalog_object * Module Products.CMFCore.WorkflowTool, line 353, in getCatalogVariablesFor * Module Products.CMFCore.WorkflowTool, line 338, in getWorkflowsFor * Module Products.CMFCore.WorkflowTool, line 307, in getChainFor * Module ZODB.Connection, line 559, in setstate * Module ZODB.FileStorage, line 689, in load * Module ZODB.FileStorage, line 657, in _load POSKeyError: 00000000000cc49a -------------------------------- fsrefs.py output oid 0xC8DF8L BTrees.OOBTree.OOBucket last updated: 2005-01-25 09:42:25.200913, tid=0x35AC9C66B861E00L refers to invalid object: oid 0xCC49AL missing: 'Products.CMFDefault.File.File' oid 0xC8E1FL BTrees.OOBTree.OOBucket last updated: 2005-01-25 09:42:25.200913, tid=0x35AC9C66B861E00L refers to invalid object: oid 0xCC49CL missing: 'BTrees.IOBTree.IOBTree' oid 0xD297EL BTrees.OOBTree.OOBucket last updated: 2005-02-10 04:08:47.313263, tid=0x35B2278C9DEB333L refers to invalid object: oid 0xCC49AL missing: 'Products.CMFDefault.File.File'

2 1

dictionary entries keyed by date - get()
by David Siedband 11 Feb '05

11 Feb '05

I've got a python script that maps a bunch of event records (results of a zSQL query) to a python dictionary, keyed by date. Inside the dtml-calendar tag, I'm trying to use the get() method to return the events for the day that the calendar iterator is on. I've tried a bunch of variations on this, but none of them return any results from the dictionary on days when there are definitely corresponding events. <dtml-in "myEvents.get(date.Date(), None)" mapping> <dtml-in "myEvents.get(date, None)" mapping> <dtml-in "myEvents.get(date)" mapping> how do I make this work? thanks -- David Siedband generation-xml.com

2 1

Using AWSTATS with Zope
by Martin Koekenberg 11 Feb '05

11 Feb '05

Hello, I'm using awstats for mij zope websites. In front op zope I use Apache. On this moment the awstats pages are not secure. How can I make these secure with my zope login. I use the extensible user folder. awstats is a perlscript with arguments... awstats.pl?config=............. Who can help me ? Thanks, Martin Koekenberg * <http://zope.org/Members/TheJester/exUserFolder> *

3 2

ZInformixDA on Zope2.7.3 problem
by Nico Saiz 11 Feb '05

11 Feb '05

Hi ! I'm trying to install ZInformixDA on Zope2.7.3, but it needs a package named informixdb-1.3, and when I execute ./configure it returns a mistake I have python2.3 installed, but I've seen that this package works with python1.5 It's possible that it is an obsolete package? I want to connect Zope2.7.3 under a Knoppix3.7 to a Informix Database with the informix database adapter, and I have this tar.gz: ZInformixDA-0.2.tar.gz.tgz and informixdb-1.3.tar.gz Can you help me? Thanks, Nico

1 0

[french] consultation Zope/CPS
by didier.georgieff 11 Feb '05

11 Feb '05

Hello, For the french speaking companies we have a call for moving our site from CPS1 to CPS3. All informations online : http://www.bas-rhin.sit.gouv.fr/Groups/sit679604/didsit/sit-cps3/view -- didier.georgieff <didier.georgieff(a)agriculture.gouv.fr> DIDSIT du Bas-Rhin

1 0

BTreeFolder2 problem
by Kevin Carlson 11 Feb '05

11 Feb '05

I have a lot of objects in a BTreeFolder2 instance. When I select any object and click on edit, I get an error message that says "Attribute error: __call__". This was not happening just a few weeks ago. Could this indicate that the BTreeFolder2 instance is damaged or could it be that the BTreeFolder2 product is damaged? Any thoughts? Thanks.

3 5

None Object and None String
by Asad Habib 11 Feb '05

11 Feb '05

I have a Zope session variable that is set to None. When I update my MySQL database and set the value of one of my fields to the value of this session variable, the value is set to the string 'None'. Anyone encountered a similar problem? I would think that the None object would translate to the NULL database constant and not the string with value 'None'. Any help would be appreciated. Thanks. - Asad

3 4

webhosting
by boricua＠pepino.is-a-geek.org 10 Feb '05

10 Feb '05

what is the friendyliest webhosting servers in the USA for zope

1 0

chk zope is running
by boricua Orman 10 Feb '05

10 Feb '05

how can one check from a linux box command line if the server is running?

5 5