May 2001 - Zope-Dev - Zope lists

Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments with ORMapping
by Joachim Werner 14 May '01

14 May '01

> Probably I'm daft because it is Friday night, but AFAIK ZODB and most OODB's > store an object only once, keyed by its object id. The rest is just references > through that oid, so objects that belong to more than one container can be > added to all these containers and n:m relations are implemented by having a > list of objects on both sides. Yes, but these references (let's call them "symlinks" ...) are not in the standard Zope. Of course it should be easy to do these things, but except for this from Shane I haven't seen anything so far: http://www.zope.org/Members/hathawsh/Symlink/index_html P.S.: Shane, have you developed Symlinks any further? I think they could be extremely useful. I tried out the initial release and liked it, except for the fact that the symlinks looked EXACTLY like real ones, so they can be very irritating ... Joachim

3 2

Experiments with ORMapping
by Shane Hathaway 14 May '01

14 May '01

Kapil (and others), Although very sketchy and I can't guarantee anything works, my experiments with object-relational mapping in Zope are found at http://www.zope.org/Members/hathawsh/ormapping.tar.gz . If ORMapping.py is in the ZODB directory, you can use the following custom_zodb.py to run Zope on top of Postgres using PoPy: import ZODB from ZODB.ORStorage import ORStorage import PoPy connection = PoPy.connect('user=zodb dbname=zodb') storage = ORStorage('pgsql!', connection, PoPy, create=1) DB = ZODB.DB(storage) The initial approach turned out to be rather problematic, since by default everything is stored in Postgres as a large binary object. Zope creates literally thousands of objects when starting for the first time and a few hundred each time thereafter. Postgres does not deal well with large binary objects so start time is something on the order of 10 minutes this way and hard drive space gets eaten up faster than Jim can code. :-) But storage of binary pickles was never the intention anyway. I created a little interface that would allow you to store different classes in different PostgreSQL tables. Before I got to implementing anything, though, I had to move to another project. But the file "sketch" shows (I hope) what I had in mind for making this work: programmers would call a bunch of functions that would put together a relational mapping tree. Now, in our discussion yesterday we decided ORStorage wasn't the right way to achieve relational mapping because there is no way for the database storage layer to have any context just given an OID. If we instead consider writing a replacement for the stuff that's mainly in Connection.py, life gets a lot simpler. At this level it *is* possible to know the parent OID of an object, though the current codebase does not use this opportunity. Also, we wouldn't have to pickle and unpickle unnecessarily. So, if we have context, the relational mapping tree can work. It would turn Zope into a purely relational application server, which a lot of folks apparently want. ;-) Shane

10 34

python scripts
by Tom Deprez 14 May '01

14 May '01

Hi, This weekend I did some work on python scripts and found some things which suprised me, can someone enlighten me on this why these 'strange' things are needed? : 1. When calling a method from a python script you've to do this : (assume x is the value to pass and name is a parameter of mymethod) context.mymethod(name=x) The following doesn't works : context.mymethod(x) (this gives an error) Why???? 2. Assume you've an sqlmethod which expects 2 parametes, but one is optional : eg select * from invoice <dtml-sqlgroup where required> <dtml-sqlgroup> <dtml-sqltest customersid column=customers_id op=eq type=int> <dtml-and> <dtml-sqltest productsid column=products_id op=eq type=int optional> </dtml-sqlgroup> </dtml-sqlgroup> So, you could run this sql with or without the productsid No, calling from python script doesn't works since it expects: context.mymethod(customerid=..., productsid=...) while context.mymethod(customerid=...) doesn't work.. Now, this implies that I've to write 2 sqlmethods (one with 2 parameters and one with 1), which is pretty stupid, since sqlmethods perfect allow to ignore a parameter when not passed... Can somebody tell me more on this? Why this is the case and perhaps a solution for (2) Thanks in advance, Tom Deprez

1 0

RE: [Zope-dev] CoreSessionTracking 0.8 strangeness
by Bjorn Stabell 14 May '01

14 May '01

Yes, it is entirely with cookies, and we are using frames, although the there's only one sub-frame accessing the session object (that's where we show the shopping cart). -----Original Message----- From: Chris McDonough [mailto:chrism@digicool.com] Sent: Saturday, May 12, 2001 23:18 To: Bjorn Stabell Cc: zope-dev(a)zope.org; Exoweb Subject: Re: [Zope-dev] CoreSessionTracking 0.8 strangeness Bjorn, Is this entirely with cookies? Or are you using url-encoding anywhere? Does your application make use of frames or multiple windows? - C Bjorn Stabell wrote: > > Chris, > > It is definately not what you expect, although it was a good guess :) > Here's an example of what happens: > > 1. User adds Burger to shopping cart > -> Shopping cart shows Burger > 2. User adds Coke to shopping cart > -> Shopping carts shows Burger and Coke > 3. User adds Fries to shopping cart > -> Shoppping cart is empty(!) > 4. User adds Fries to shopping cart > -> Shopping cart only shows Fries > 4. User adds Apple Pie to shopping cart > -> Shopping cart shows Fries and Apple Pie > 5. User adds Ice Cream to shopping cart > -> Shopping cart shows Burger, Coke, and Ice Cream(!) > ...and so on... > > Sometimes, clicking links that don't even touch the shopping cart will > make it disappear, e.g., viewing a new product category in the catalog. > It seems pretty random, and it is hard to reproduce reliably. The > shopping cart is displayed on the same page as the catalog. Any ideas? > > PS! I have noticed that in other cases Zope is a acting a little bit > random too, e.g., when it comes to which character set is used; it will > not obey our commands to set it to gb2312 about 1 out of 10 times. That > could be an IE bug, but I doubt it. > > Bye, > -- > Bjorn

1 0

Re: oodb philosophics ;) was: Re: [Zope-dev] Experiments with ORMapping
by Ken Manheimer 13 May '01

13 May '01

On friday, 11 May, Joachim Warner wrote: > > The other motivations for an RDBMS are (1) people have existing schemas > > and want Zope to access the same data as their existing apps, and they > > want it to be transparent, and (2) tables with millions of entries are > > easily stored in Zope but the perception is that the catalog isn't as > > fast as a database index. No one has done any tests AFAIK. > > Then we should do these tests. E.g. I'd like to see: > > - 20 GB of Word and PDF documents stored in the ZODB and full-text + > metadata indexed in ZCatalog > - the complete zope(a)zope.org mailing list archive in the ZODB > > If Zope can handle those without the help of external tools (RDBMS etc.), > I'll use it for all our Document Management ... As a matter of fact, we did a quick CMF demo that has the content of the zope list, zope-dev, and many of the other zope.org lists, and the comp.lang.python list for the past few years. The catalog searches are very very fast, i can't recall if the demo was set up with some interesting canned CMF topics, but the things works well. The picture isn't altogether rosy - the process of loading the objects was arduous. On the other hand, the exercise (actually, a subsequent one with simpler article objects) served as the basis for tuning the cataloging process, and may have helped it get a lot better. If i have time next week, i'll see if we have the corpus online somewhere. (The lists were complete up to a few months ago.) Eventually we'd like to be incrementally stuffing new messages into the database as they arrive. The catalog has required some substantial work investment, but from my viewpoint (particularly since i haven't had to work on it!-), it's well worth it. > > Actually OracleStorage and bsddbstorage, recently released, are designed > > to address concerns about performance and reliability, and they do an > > excellent job at it. And I consider ZODB as "real" an OODB as anything > > else. (In some ways it's the best out there IMHO.) > > Agreed. ZODB has a much longer proven history of success than most other > OODBs. It is quite useful! I have no experience programming with other object databases, and very little with relational databases, so i have no basis for comparison. What i do know, as a python and zope programmer, is that ZODB is spectacularly useful as a persistent store - as flexible *and* as powerful as i could want. The addition of ZEO manages to significantly increase that usefulness! The work we/pythonlabs (and andrew kuchling, etc) is doing to enable use of it as an independent entity can only help improve it's usefulness for everyone. Ken Manheimer klm(a)digicool.com

4 3

New IOBTrees have no 'map' method
by Chris Withers 13 May '01

13 May '01

Hi, I was going to have the next version of Squishdot use the new BTrees, but it appears that the new objects don't have the same methods as the old ones. In particular, the following line from Squishdot: return map(lambda x, p=self: x.__of__(p), self.data.map(self.site_id_list(currtime))) ...fails with an attribute error on map. self.data is an IOBTree. Any ideas? cheers, Chris

3 10

ZPivotTable doesnt save its data dictionary
by florian_reiser＠gmx.de 13 May '01

13 May '01

Hi folks, I've started to program a pivot table. Therefore I've created three classes: ZPivotTable, Row and Cell. The ZPivotTable class contains a set of Row classes, stored in a dictionary, as well as the Row classes contain a set of Cell classes stored in a dictionary. Now I am debugging the product and can't find the reason why the table doesn't store the rows in its dictionary. Could you please help me finding this bug. The source is stored under the following address: http://www.zope.org/Members/freiser/ZPivotTable/ZPivotTable-0.0.tar.gz Thanks Florian Reiser -- ----------------------------------------------------------------------- | Florian Reiser PGP-Key:0x5368CF3F | | PGP-Fingerprint: 84C7 24BC F3A4 487F CBC7 692C 05DB 6E11 5368 CF3F | -----------------------------------------------------------------------

2 4

zope.org Product Release
by richard＠bizarsoftware.com.au 13 May '01

13 May '01

I'm trying to upload a .zexp to a product release ("Add Product Release") and it's not working. The export is only 57k, but all I get is zope.org not responding... Any hints? Should I upload something other than a zexp? Richard -- Richard Jones richard(a)bizarsoftware.com.au Senior Software Developer, Bizar Software (www.bizarsoftware.com.au)

2 1

RE: [Zope-dev] CoreSessionTracking 0.8 strangeness
by Bjorn Stabell 12 May '01

12 May '01

Chris, It is definately not what you expect, although it was a good guess :) Here's an example of what happens: 1. User adds Burger to shopping cart -> Shopping cart shows Burger 2. User adds Coke to shopping cart -> Shopping carts shows Burger and Coke 3. User adds Fries to shopping cart -> Shoppping cart is empty(!) 4. User adds Fries to shopping cart -> Shopping cart only shows Fries 4. User adds Apple Pie to shopping cart -> Shopping cart shows Fries and Apple Pie 5. User adds Ice Cream to shopping cart -> Shopping cart shows Burger, Coke, and Ice Cream(!) ...and so on... Sometimes, clicking links that don't even touch the shopping cart will make it disappear, e.g., viewing a new product category in the catalog. It seems pretty random, and it is hard to reproduce reliably. The shopping cart is displayed on the same page as the catalog. Any ideas? PS! I have noticed that in other cases Zope is a acting a little bit random too, e.g., when it comes to which character set is used; it will not obey our commands to set it to gb2312 about 1 out of 10 times. That could be an IE bug, but I doubt it. Bye, -- Bjorn -----Original Message----- From: Chris McDonough [mailto:chrism@digicool.com] Sent: Friday, May 11, 2001 20:47 To: Bjorn Stabell Cc: zope-dev(a)zope.org; Exoweb Subject: Re: [Zope-dev] CoreSessionTracking 0.8 strangeness This is very odd, as cst depends on ZODB locking just like everything else in Zope, and uses the same transaction facilities and semantics. Now this may *be* the problem... some other folks have explained a problem where newly-created session data "disappears" if the long-running request that created it is aborted... this may make sense, because the request never gets the chance to finish, and the session data object never gets committed to the database because the ZODB transaction is aborted. It may be the case (especially if you're using frames or multiple windows) that simultaneous concurrent initial requests for a session data object from the same client might cause one of two newly-created data objects to be lost. But I can't imagine a case where a session data object exists for a while (a few minutes), and then a client comes in with the same sessionid, and a new session data object is created for him, not *replacing* the old one, but in addition to the old one... the only possibility for something like this that I can see is on initial request, or if a particularly long-running method creates a session data object, and a user comes in directly after it... ah, wait. I think I see how this can happen. 1. User hits /longrunningmethod. 2. /longrunningmethod creates a session data object as part of its operation. 3. /longrunning method requires 1 minute to process completely. 4. In the meantime, the user (in a separate window or frame) visits /shortrunningmethod. 5. /shortrunningmethod creates a session data object as part of its operation. 6. /shortrunningmethod completes and the session data object it created is committed to the ZODB. 7. User visits /shortrunningmethod2, which modifies the existing session data object, /shortunningmethod3, which also does this, etc. 8. In the meantime, /longrunningmethod finishes and replaces the session data object which has stuff in it from the shortrunningmethods with the one it created. I'm not completely 100% sure about this, but at least it's something to test. Could this be what's happening to you? I may have been overzealous when dealing with confict error problems here... basically, the conflict error detection stuff is disabled for session data objects, which makes this sort of thing possible and likely. Conflict error detection is the ZODB equivalent of record-locking... and when it's ignored, you can have some of the same problems that happen in a nontransactional system (like the problem I outlined above). Sigh. - C Bjorn Stabell wrote: > > Hi, > > We're developing a shopping cart using the CoreSessionTracking product > v0.8, but we've run into a strange problem; once in a while > getSessionData() will create a new session data object (the token value > remains the same), and us that for a while. Now we'll end up having two > different shopping cart objects, with getSessionData() randomly > returning one of them them. We're running Zope 2.3.1. Could it be that > some threads can't find the shared session data object and creates their > own? The only thing changing between each HTTP request is the thread > serving the request, AFAIK. > > Regards, > -- > Bjorn Stabell <bjorn(a)exoweb.net> > > _______________________________________________ > Zope-Dev maillist - Zope-Dev(a)zope.org > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope )

2 1

Re: [Zope-dev] Question about import code
by Dieter Maurer 11 May '01

11 May '01

Shane Hathaway writes: > Fred Wilson Horch wrote: > > > Importing multiple objects simultaneously could result in a fair amount > > > of confusion... > > > > Can you elaborate? When you import a folder, for example, it imports > > the folder and all objects in it. > > If there are some objects in the folder that have conflicting names and > others that don't, what do you do? Zope already solves this problem for "paste". Although, I am not sure that I like this solution, "import" should behave the same, for consistency. > Also, if you're thinking about > providing this functionality, what about spanning multiple folders? What > if some folders already exist? Again, it should behave like "paste", for consistency. Dieter

1 0