- Zope-Dev - Zope lists

Re: [Zope-dev] Experiments with ORMapping
by Shane Hathaway 15 May '01

15 May '01

"Phillip J. Eby" wrote: > > At 05:42 PM 5/11/01 -0400, Shane Hathaway wrote: > >"Phillip J. Eby" wrote: > >> I'm not quite clear on how exactly you suggest mapping from RDMBS -> > >> ZODB. There's a *significant* (IMHO) impedance mismatch between ZODB's > >> arbitrarily identified variably structured single records and SQL's > >> content-identified fixed-structure record sets. This is what application > >> frameworks/toolkits (such as your own DBAPI product) are needed for. > > > >If you implement this at the Storage level, yes, there is a major > >mismatch. But at the Connection level it makes a lot of sense. > >Connection basically exposes a pile of pickles as objects; an OR mapping > >exposes a complex schema as objects. > > > >I think that understanding will change the rest of your response. :-) > > > > Nope. As far as I can see, all that does is leverage ZODB Connections as a > crude sort of Rack that only provides a mapping-like interface for > retrieving objects, without helping any of the higher-order needs of an > application. I guess if you define O-R mapping as "can you store and > retrieve an object's properties from a database", then I guess you've > succeeded at that point. But if that was all my O-R apps needed, there > would've been no reason to use the RDBMS; I could've just used ZODB and a > BTree with less bother. I'm telling you there's a lot more you can do with the code that makes up ZODB. The mapping interface is not the key to the way Connection does its work. OIDs don't have to be strings. If we just use cPersistent, cPickleCache (which is misnamed), and Transaction to implement an OR mapping, here's what we get for free: - A transparent, transactional database. - Potential interoperability with everything written for ZODB. - Robust, tested code. - Optimizations in C. Since you challenged me :-) I decided to put together a prototype. What I came up with was a database connection that sets a different _p_jar for each persistent object. The _p_jar is an object that manages the storage for only one persistent object, making it possible to customize the storage. There was only one hurdle in this approach but I hacked around it: the tpc_* messages get sent to each _p_jar. Then I wrote a very minimal test that doesn't connect to an RDBMS but stores and retrieves simple objects in a dictionary. OIDs are still necessary to support multithreaded invalidation messages. But the OIDs in the prototype are a tuple consisting of a schema ID and a record ID. That way the record IDs only need to be unique within an RDBMS table (or combination of tables.) I didn't think anything like this was possible before I got to know Jim. I still didn't understand when he presented the idea months ago. But now I see the idea really is workable. The advantage is that it lets us completely isolate RDBMS storage details from the application. The next thing to do is to write a fishbowl proposal. Shane

6 17

xmlrpc: Syntax error at line 34: illegal character in content --> Look at what I found out
by Ulrich Eck 15 May '01

15 May '01

Hi chris, hi dieter, I also have this problem with using xmlrpc from a python client with zope. > > RuntimeError: Syntax error at line 34: illegal character in content > > This was the bane of one of our projects to the point where Andy re-wrote > ZMailIn to no-longer use XML-RPC. > > It's to do with stuff-which-should-probably-be-entities but we never tracked > down what wasn't encoding something when it should have done. > I'd blame the marshalling code in xmlrpclib.py myself. > > If you ever fix this, do let us know :-) > I have a Zope 2.3.1 server running as well as a Zope 2.3.2. I developed an Service-Framework on the first and then tried to install it on the 2.3.2 .. it failed I printed out the raw data of the transmission and they differ: When I call the function on Zope-2.3.1 i get this response: --------------------------------------------------------- <?xml version='1.0'?> <methodResponse> <params> <param> <value><string>OK</string></value> </param> </params> </methodResponse> --------------------------------------------------------- when I call the same function on the same backend on Zope-2.3.2 I get the following: --------------------------------------------------------- X-Powered-By: Zope (www.zope.org) Python (www.python.org) Content-Length: 128 Connection: close Content-Type: text/xml <?xml version='1.0'?> <methodResponse> <params> <param> <value><string>OK</string></value> </param> </params> </methodResponse> --------------------------------------------------------- There is some header-information in the body (that perhaps shoudn't be there) Where there any changes in Zope 2.3.1 -> 2.3.2 in the Publishing part ?? With the second response the xmllib-parser won't work as we see. any suggestions to that ?? did you guy's updated to Zope-2.3.2 recently ?? thanks for any response and hope this helps a bit .. Ulrich Eck net-labs

1 0

zope early win32 binaries
by Sin Hang Kin 15 May '01

15 May '01

Plesase consider release the binaries for python2(.1) for win32 asap. As my understanding, the current c extensions were pretty stable for zope. what have been under major re-construction is purely in python. Would DC care to release an PYD (DLL) only sub-package for win32 users to test the up-comming zope? I hope all would be pleased. Rgs, Kent Sin --------------------------------- kentsin.weblogs.com kentsin.imeme.net

1 0

Proposal: AuthenticatedRole
by Chris McDonough 14 May '01

14 May '01

Hi all, If you're concerned or curious, please review this proposal on the fishbowl regarding adding a default role to Zope named 'Authenticated': http://dev.zope.org/Wikis/DevSite/Proposals/AuthenticatedRole

1 0

Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments withORMapping
by Karl Anderson 14 May '01

14 May '01

Casey Duncan <cduncan(a)kaivo.com> writes: > I am not arguing necessarily for SQL as a query language for the ZODB. > Although it is an accepted standard, but not a perfect one by any means > especially for OODBs. Its appeal lies mainly in the high level of > community familiarity and the plethora of SQL software to borrow from. Does anyone have an opinion on the possible usefulness of XPath, XQuery, and other XML standards for this? Someone suggested (on the zope-xml wiki) that it would be nice to be able to drop in a cataloger that supported a presumably standard and presumably well-known XML query syntax, and which would work throughout the database because Zope objects would support DOM. This is all speculation, and I personally don't know much right now about XML database interfaces and how finished or well-regarded they are. -- Karl Anderson karl(a)digicool.com

2 1

Disabling anonymous webdav access
by Ivo van der Wijk 14 May '01

14 May '01

Hi All, As someone pointed out on #zope, it is possible to view folder contents using a webdav client as an anonymous user. I.e. download cadaver (http://www.webdav.org/cadaver/) open yourzopeserver:8080 and do ls. Then decide if you want anyone to be able to access this. Eventhough hiding this information may be security by obscurity, there are some things you just don't want everyone to see. This allows you to see, for example, the installed products on the server. A hacker might use this knowledge to exploit some known bug in a zope product if one exists. Most people (like me) probably think it's harmless to let old objects, documents etc linger around as you can't view them in listings through ftp or http. They don't realize webdav is running by default. Actually, it can't even be disabled! (z2.py -X -w80 won't do the trick!) Personally I'd rather see this secured. It's not possible to disable 'view contents information' for anonymous users in zope, as this will ruin your entire site (all anonymous access will then be disabled), so the solution would be to create a new permission for access contents through webdav. And that's what the following (trivial) patch does. After applying you'll get a new permission in your security tab, which is set to manager by default. To get the old behaviour back, just set the permission back to anonymous. Apply it using patch -p1 ../webdav.patch in your SOFTWARE_HOME (i.e. the Zope-2.3.2-src dir). Or just edit lib/python/webdav/Resource.py by hand :) I've tested it with Zope 2.3.2, I can't guarantee it will work with other versions (use at your own risk anyway). -- cut here -- *** Zope-2.3.2-orig/lib/python/webdav/Resource.py Tue Mar 27 21:50:37 2001 --- Zope-2.3.2-src/lib/python/webdav/Resource.py Mon May 14 19:16:46 2001 *************** *** 109,115 **** __ac_permissions__=( ('View', ('HEAD',)), ! ('Access contents information', ('PROPFIND',)), ('Manage properties', ('PROPPATCH',)), ('Delete objects', ('DELETE',)), ) --- 109,115 ---- __ac_permissions__=( ('View', ('HEAD',)), ! ('Access contents information through WebDav', ('PROPFIND',)), ('Manage properties', ('PROPPATCH',)), ('Delete objects', ('DELETE',)), ) -- cut here -- Cheers, Ivo -- Drs. I.R. van der Wijk -=- Brouwersgracht 132 Amaze Internet Services V.O.F. 1013 HA Amsterdam -=- Tel: +31-20-4688336 Linux/Web/Zope/SQL Fax: +31-20-4688337 Network Solutions Web: http://www.amaze.nl/ Consultancy Email: ivo(a)amaze.nl -=-

1 0

Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments with ORMapping
by Joachim Werner 14 May '01

14 May '01

> Probably I'm daft because it is Friday night, but AFAIK ZODB and most OODB's > store an object only once, keyed by its object id. The rest is just references > through that oid, so objects that belong to more than one container can be > added to all these containers and n:m relations are implemented by having a > list of objects on both sides. Yes, but these references (let's call them "symlinks" ...) are not in the standard Zope. Of course it should be easy to do these things, but except for this from Shane I haven't seen anything so far: http://www.zope.org/Members/hathawsh/Symlink/index_html P.S.: Shane, have you developed Symlinks any further? I think they could be extremely useful. I tried out the initial release and liked it, except for the fact that the symlinks looked EXACTLY like real ones, so they can be very irritating ... Joachim

3 2

Experiments with ORMapping
by Shane Hathaway 14 May '01

14 May '01

Kapil (and others), Although very sketchy and I can't guarantee anything works, my experiments with object-relational mapping in Zope are found at http://www.zope.org/Members/hathawsh/ormapping.tar.gz . If ORMapping.py is in the ZODB directory, you can use the following custom_zodb.py to run Zope on top of Postgres using PoPy: import ZODB from ZODB.ORStorage import ORStorage import PoPy connection = PoPy.connect('user=zodb dbname=zodb') storage = ORStorage('pgsql!', connection, PoPy, create=1) DB = ZODB.DB(storage) The initial approach turned out to be rather problematic, since by default everything is stored in Postgres as a large binary object. Zope creates literally thousands of objects when starting for the first time and a few hundred each time thereafter. Postgres does not deal well with large binary objects so start time is something on the order of 10 minutes this way and hard drive space gets eaten up faster than Jim can code. :-) But storage of binary pickles was never the intention anyway. I created a little interface that would allow you to store different classes in different PostgreSQL tables. Before I got to implementing anything, though, I had to move to another project. But the file "sketch" shows (I hope) what I had in mind for making this work: programmers would call a bunch of functions that would put together a relational mapping tree. Now, in our discussion yesterday we decided ORStorage wasn't the right way to achieve relational mapping because there is no way for the database storage layer to have any context just given an OID. If we instead consider writing a replacement for the stuff that's mainly in Connection.py, life gets a lot simpler. At this level it *is* possible to know the parent OID of an object, though the current codebase does not use this opportunity. Also, we wouldn't have to pickle and unpickle unnecessarily. So, if we have context, the relational mapping tree can work. It would turn Zope into a purely relational application server, which a lot of folks apparently want. ;-) Shane

10 34

python scripts
by Tom Deprez 14 May '01

14 May '01

Hi, This weekend I did some work on python scripts and found some things which suprised me, can someone enlighten me on this why these 'strange' things are needed? : 1. When calling a method from a python script you've to do this : (assume x is the value to pass and name is a parameter of mymethod) context.mymethod(name=x) The following doesn't works : context.mymethod(x) (this gives an error) Why???? 2. Assume you've an sqlmethod which expects 2 parametes, but one is optional : eg select * from invoice <dtml-sqlgroup where required> <dtml-sqlgroup> <dtml-sqltest customersid column=customers_id op=eq type=int> <dtml-and> <dtml-sqltest productsid column=products_id op=eq type=int optional> </dtml-sqlgroup> </dtml-sqlgroup> So, you could run this sql with or without the productsid No, calling from python script doesn't works since it expects: context.mymethod(customerid=..., productsid=...) while context.mymethod(customerid=...) doesn't work.. Now, this implies that I've to write 2 sqlmethods (one with 2 parameters and one with 1), which is pretty stupid, since sqlmethods perfect allow to ignore a parameter when not passed... Can somebody tell me more on this? Why this is the case and perhaps a solution for (2) Thanks in advance, Tom Deprez

1 0

RE: [Zope-dev] CoreSessionTracking 0.8 strangeness
by Bjorn Stabell 14 May '01

14 May '01

Yes, it is entirely with cookies, and we are using frames, although the there's only one sub-frame accessing the session object (that's where we show the shopping cart). -----Original Message----- From: Chris McDonough [mailto:chrism@digicool.com] Sent: Saturday, May 12, 2001 23:18 To: Bjorn Stabell Cc: zope-dev(a)zope.org; Exoweb Subject: Re: [Zope-dev] CoreSessionTracking 0.8 strangeness Bjorn, Is this entirely with cookies? Or are you using url-encoding anywhere? Does your application make use of frames or multiple windows? - C Bjorn Stabell wrote: > > Chris, > > It is definately not what you expect, although it was a good guess :) > Here's an example of what happens: > > 1. User adds Burger to shopping cart > -> Shopping cart shows Burger > 2. User adds Coke to shopping cart > -> Shopping carts shows Burger and Coke > 3. User adds Fries to shopping cart > -> Shoppping cart is empty(!) > 4. User adds Fries to shopping cart > -> Shopping cart only shows Fries > 4. User adds Apple Pie to shopping cart > -> Shopping cart shows Fries and Apple Pie > 5. User adds Ice Cream to shopping cart > -> Shopping cart shows Burger, Coke, and Ice Cream(!) > ...and so on... > > Sometimes, clicking links that don't even touch the shopping cart will > make it disappear, e.g., viewing a new product category in the catalog. > It seems pretty random, and it is hard to reproduce reliably. The > shopping cart is displayed on the same page as the catalog. Any ideas? > > PS! I have noticed that in other cases Zope is a acting a little bit > random too, e.g., when it comes to which character set is used; it will > not obey our commands to set it to gb2312 about 1 out of 10 times. That > could be an IE bug, but I doubt it. > > Bye, > -- > Bjorn

1 0