June 2000 - Zope-Dev

Re: [Zope-dev] transaction support for ordering transactions.
by Anthony Baxter 06 Jun '00

06 Jun '00

>>> "Phillip J. Eby" wrote > There is an error in your patch, however. _prepend should use > self._objects.insert(0,object), as the way you're doing it now will break > the _append method. (Because _append is a reference to _objects.append, > and you're replacing the old _objects with a new list. Subsequent calls to > _append will append the object to the old _objects list, not the new one > you've just added.) Darn. That'll teach me to make a quick patch like this. Still, getting the feedback is the key thing - I wasn't sure if the approach I was taking was the right one. > There is also a *second* problem with your patch, which is more serious and > less correctable. If a _prepend takes place *during* a commit operation, > the current implementation of the commit operation will be blown all to Ugh. Ok, how about then just doing something like not allowing a prepend once the committings already started? > Anyway... I'm very much in favor of solving the same problem that you > are... But it requires a rather more complex patch (so that > transaction.commit() could deal with prepends during commit) and first we'd > need to convince Jim that it's the right thing to do. :) (Not only that, > but he might come up with a better way to do it than prepending...) He does tend to do that. I'd very much like a way to fix this - at the moment, the only workaround for me is for SQLSessions to have an entirely different database connection to themselves. > Ironically, I'm not sure your problem actually *needs* this fix to be > solved. You probably need to tie your session object's behavior to an > earlier transaction event (the "_p_jar.commit()" operation rather than > tpc_finish()), and you can then be guaranteed that the SQL connection has > not yet seen a tpc_finish() either. I'm not sure I understand this - are you suggesting hooking it to the database connector's commit? > *My* problem, on the other hand, is that even if I do this, ZODB does not > allow objects to be _p_jar.commit()ed twice; if you modify an object which > has already been _p_jar.commit()ed in the same transaction, you will get an > unresolvable ConflictError (because the ZODB thinks another transaction > modified the object before you). Ugh. Nasty nasty. But hey, isn't the whole point of ZPatterns to expose all the nasty corners of the Zope innards to light? > Thus, I need to ensure that DataManagers > which are saving things up 'till a _p_jar.commit() operation, get > registered ahead of any modification to an object that they might need to > modify again later. So a fully functional "prepend" capability would be of > great benefit to ZPatterns, but I'm not sure that your problem actually > needs it, and your patch, unfortunately, does not achieve it. Thanks for the feedback - time for me to hit the source code again. :) Anthony

1 0

BerkeleyStorage.py is broken?
by nw_moriarty 06 Jun '00

06 Jun '00

There are many admonitions to use BerkeleyStorage.py if you don't require versions in a database. However, I have tried without success to get it to work with TCS.bsddb.db and BerkeleyDB 2.7.7 in a stand alone application. How ANYBODY got it to work? Nigel --== Sent via Deja.com http://www.deja.com/ ==-- Before you buy.

2 1

transaction support for ordering transactions.
by Anthony Baxter 06 Jun '00

06 Jun '00

As part of making SQLSession transaction-sane, I'd like to be able to specify that something can register itself to be committed or aborted earlier in the list of objects rather than just being put on the end of the list. The reason for this: I want SQLSession to just use a normal DB connection, and have the commits &c to "just work". The new code now only does the SQL statements in SQLSessionObj's _finish() method - unfortunately, by that time, the database's '_finish()' method has sometimes already been called (if there's a different database call before the Session object is created, then the database adaptor will be registered first). I thought about various ways of dealing with this, such as being able to say 'register me _before_ that object', but ended up going simply with a flag to say 'put me on the front of the list, rather than the back.' This way, anything that needs to say 'no, put me before the standard ones' can do so. Thoughts? Alternatives? Anthony Patch against current-cvs follows: --- lib/python/ZODB/Transaction.py 2000/05/30 19:03:27 1.22 +++ lib/python/ZODB/Transaction.py 2000/06/06 04:01:51 @@ -117,6 +117,9 @@ for c in self._connections.values(): c.close() del self._connections + def _prepend(self, object): + self._objects = [object] + self._objects + def sub(self): # Create a manually managed subtransaction for internal use r=self.__class__() @@ -315,9 +318,12 @@ del objects[:] # clear registered if not subtransaction and self._id is not None: free_transaction() - def register(self,object): + def register(self,object,early=0): 'Register the given object for transaction control.' - self._append(object) + if early: + self._prepend(object) + else: + self._append(object) def note(self, text): if self.description: -- Anthony Baxter <anthony(a)interlink.com.au> It's never too late to have a happy childhood.

2 1

zpatterns-0.4 ?
by Jephte CLAIN 06 Jun '00

06 Jun '00

Hello, I have used ZPatterns-0.3.0 to complete a project that I shipped yesterday. Although I had to write a CatalogAwareRack/Specialist to allow me to search for rackmountables, I can say that ZPatterns has saved my life! I would like to thanks Philip for this great contribution. (too bad I no longer remember the url of that site where you can send so-called "virtual beers" to fellows on the internet :-)) Now, for a project I have to ship by the end of june, I would like to use zpatterns-0.4.0. I especially want to use the new search interface. Does Philip know when ZPatterns-0.4.0 is out? If it is until the middle of june, I can certainly wait. In the other case, I wonder if the search plugin (in the case it has already be written) can be used with zpatterns-0.3. In that case, I can do the work of adapting the code to zpatterns-0.3 Also, for that project, I will have to write a sql attribute provider. If one can give me advices on how Ty wants to implement SQL Attribute Provides after he's done with LDAP Attribute Provider, perhaps I can start the work (since I have a use for it) and let Ty continue the work when he has more time / have finished LDAP Attribute Provider. I already have some ideas on how to do it, but I lack to view how far the concept have to be pushed. Without any help, I will end up writing a SQL Attribute Provider that work only for me... regards, jephte clain minf7(a)educ.univ-reunion.fr

2 1

ReiserStorage 0.01 alpha
by Petru Paler 06 Jun '00

06 Jun '00

Hello all, Attached you can find the code for a preliminary ZODB storage manager which is designed to take adventage of the reiserfs filesystem (http://devlinux.com/namesys) While it is usable on other filesystems too, the performance will decrease as the number of objects in your database increases. Comments: 1. This is as alpha as it can be. While it is usable to do some testing on a spare Zope, you must be insane to even try to use this on a production system. 2. A generic DirectoryStorage is comming soon, with the diference being splitting files by directory (and, of course, being slower than the reiserfs version). 3. As you can see in the source, I/O errors in critical points can corrupt the storage. 4. It currently does not support versioning and undo. I'm working first on getting it stable and fast, and then add these features. Feedback is not only welcome, but wanted :) -Petru

1 0

ZPatterns ProperySheet usage example?
by Mike 06 Jun '00

06 Jun '00

Hello people, could anybode give an example (in code) on subject? Thanks, Mike

3 3

RE: [Zope] highly available Zope thread; our hanging problem
by Marcus Collins 06 Jun '00

06 Jun '00

Hi, I'd like to comment on this, and summarise some references below. Much of this discussion took place on the zope-dev list (see references), so I'm cc'ing the zope-dev list. You might also wish to add to the Wiki: http://www.zope.org/Members/tseaver/Projects/HighlyAvailableZope/. > -----Original Message----- > From: Brian Takashi Hooper [mailto:brian@garage.co.jp] > Sent: 06 June 2000 12:11 > To: zope(a)zope.org > Subject: [Zope] highly available Zope thread; our hanging problem > > Hi all - > > I was looking at the discussion from April that was posted on the > HighlyAvailableZope Wiki about problems with Zope hanging; we had a > similar situation here at Digital Garage which seemed to be alleviated > by changing the zombie_timeout to be really short (like, 1 minute). > Before changing the zombie_timeout, the server would periodically hang > and not give any responses to requests, sometimes recovering after a > short time. Some questions at this point: 1. Were you running with multiple threads, and if so, how many? 2. If you were using multiple threads, would *all* the threads periodically hang, or was the hanging isolated to a single thread at a time? 3. Could you possibly comment on the operating system used? 4. Which zombie_timeout did you twiddle -- the one in the zhttp_channel in ZServer.py, or that in http_channel in medusa/http_server.py? > At this point, I don't have anything more than just an empirical > observation - changing this parameter seemed to help our server. Has > anyone else noticed anything similar, or can explain this observation? Concerning the zombie_timeout suggestion, here are some references when I posed the question of whether reducing the value would be beneficial: Amos Lattier wrote in http://lists.zope.org/pipermail/zope-dev/2000-April/004194.html: > The ZServer zombie stuff is to get rid of zombie client > connections, not zombie publishing threads. These are quite > different beasts. Michel Pelletier wrote in http://lists.zope.org/pipermail/zope-dev/2000-April/004229.html: > What the Zombie timeout means is that after a publishing thread gets > done answering a request, the socket may not go away. This many for a a > number of reasons, the client 'hung' and is not 'putting down the phone > after the converstation is over' (so to speak) or network troubles may > prevent the connection from closing properly. This means that there is > a 'zombie' connection laying around. This zombie will probably end up > going away on its own, but if not, ZServer will kill it after a period > of time. > > The only reasorce laying around during the life of a Zombie is an tiny > little unused open socket, the Mack truck of a Zope thread that served > the request for the zombie socket does not 'hang' for that entire period > of time, but goes on after it has completed the request to serve other > requests. > > Amos is correct in that these problems are almost always at the > Application level, and not at the ZServer level. The fact that Pavlos > can prevent hanging by inserting a print statement in the asyncore loop[*] > is suspicious, but we do not have enough information yet to point > fingers anywhere. [* references http://lists.zope.org/pipermail/zope/2000-April/023697.html] I'd be _very_ interested in hearing more on this! Our Zope installation has been pretty stable of late (isn't it strange that, when you want to find out what's causing things to break, they play nice?), with uptime of thirty-something days, but I'm still very keen to get to the bottom of this, since I don't believe it was some ephemeral problem. hth, and thanks again! -- Marcus

2 1

Re: Medusa and FTP PUT
by Chimezie Thomas-Ogbuji 05 Jun '00

05 Jun '00

Eventually, I found that NullResource is the top level class that handles this first (Objectmanager doesn't inherit off of it -- it inherits off of Resource, which would explain why I couldn't override the PUT method). It has a block of logic similar to ObjectManager.PUTer that checks the mime-types and instanciates DTMLDocuments, Files, Images appropriately. The docstring for NullResource says: Null resources are used to handle HTTP method calls on objects which do not yet exist in the url namespace so I'm assuming my overidden PUT method will never be called by the Medusa FTP server if I try to put a nonexisting file on the server. However, I tried to change the content of an existing file (an XMLDocument instance) using angle-ftp and xemacs and my overidden PUT was 'still' never invoked and a DTMLDocument with the new content was put in it's place. Go figure Chimezie Thomas-Ogbuji wrote: > > I tried hacking the ObjectManager.py code (specifically the PUTer helper > class), > however it seemed to have no effect when I upload a .xml file (it > creates > a DTMLDocument). I even tried removing the logic that creates > a DTMLDocument, but regardless, it still creates a DTMLDocument when I > upload a .xml file. I tried deleting the .pyc problem in case it was > caching the module, but to no avail. I even added a print statement > to the PUT method (which usually gives a display on stdout) but nothing > happened when I tried to put a new .xml file anywhere in the Zope > heirarchy. > > Do you have any idea? How is the content type determined anyways? Is > this determined > by the medusa server core, or is this set by the client (I don't think > it's the latter > because most ftp clients only allow you to set the file type as ascii or > binary only). > > -- > Chimezie Thomas-Ogbuji > Consultant > Fourthought Inc. > (303) 583 9900 ext 104 > cogbuji(a)fourthought.com -- Chimezie Thomas-Ogbuji Consultant Fourthought Inc. (303) 583 9900 ext 104 cogbuji(a)fourthought.com

1 0

Import from upload?
by Sin Hang Kin 05 Jun '00

05 Jun '00

Hi, I have just come acroos this idea of having zope accept the import file from an upload form instead of having it from the server directory. In web hosting environment like imeme.net, It is painful and unsecure for all to share the import directory for updating. Rgs, Kent Sin --------------------------------- kentsin.weblogs.com kentsin.imeme.net

5 5

Re: [Zope-dev] The future of Zope with XML( XSLT revisited )
by Josh Zeidner 05 Jun '00

05 Jun '00

Hi, It is great stuff! ( similar idea to Apache Cocoon )... now all we need is someone to put in an XSLT( 4XSLT ) processor into the core Zope code- are there any immediate plans for this? note: i realize this has been discussed before :) . -thanks, Josh Z >Hi, Josh > > Thanks for the insight! I also just noticed that there is a whole lot >of >good information already up there in the Wiki. This does look like great >stuff. I see how it complements Zope's current data storage capabilities. >It >provides good ways to move XML data in and out of the ZODB via XSLT and to >store and manipulate XML data efficiently using XML Documents. These are >excellent capabilities indeed. > > Have a great weekend! > >Kevin > >----- Original Message ----- >From: "Josh Zeidner" <jmz_phylogenic(a)hotmail.com> >To: <kid(a)kendermedia.com>; <zope-dev(a)zope.org> >Sent: Friday, June 02, 2000 10:54 AM >Subject: Re: [Zope-dev] The future of Zope with XML( what does ZDOM do for >us? ) > > > > > > Hi Kevin, > > > > What FourThought/DC are currently doing with Zope will allow for fast > > transformations to memory-resident structured data( ZDOM or whatever ) >to > > exportable XML documents( which are simply strings of characters ). >This > > introduces( to the Zope world ) cutting edge metaphors in >web-application > > design. All data to be served to a client is some form of XML: >including > > HTML ( a better term: structured data ). The internal structure, or > > representation( or "deep structure" if you are into Chomsky ), is the > > data/knowledge as the web site managers see it- and most likely the most > > accurate and flexible when dealing with the particular >ormation-domain( > > this is what theyre paid for :) ). Each user has custom needs- and >wants to > > see this information the way they( or the applications working on thier > > behalf ) are accustomed to. So, architectually speaking, we need a way >to > > easily and quickly construct XML documents from some meta-type( our >internal > > structure ). This is what ZDOM is accomplishing for us. > > > > Parsing is a expensive endeavor. It is not practical to parse >character > > streams every time we want to get at our data. So the solution is to > > represent the data as something more computationally digestible. Again >: > > this is >what ZDOM is accomplishing for us. > > > > Does this answer your question? > > > > have a good weekend, > > > > Josh Zeidner > > > > > > > > -"zippa-dee-doo-da" , uncle remus > > > > >Hello, > > > > > > Thanks for getting this document going. It's good to see where XML >is > > >going in Zope. > > > > > > I do have a question, though. For me, the main value in XML has >been > > >in > > >its usefulness as a flexible interchange format. Right now, I can store > > >data > > >as objects (either straight Python or ZClasses) in Zope and transform >those > > >into XML if I need to share it with the outside world (eg RSS). >Similarly, > > >though I haven't done this, I believe I can take XML from the outside > > >world, > > >parse it and store it in objects in the database. Are there advantages >to > > >storing the data in Zope as XML vs. storing my data as objects? > > > > > >Thanks, > > >Kevin > > > > > >----- Original Message ----- > > >From: Glenn Gasmen > > >To: zope-dev(a)zope.org > > >Cc: consultants(a)fourthought.com > > >Sent: Thursday, June 01, 2000 11:05 PM > > >Subject: [Zope-dev] The future of Zope with XML > > > > > > > > >http://www.zope.org/Wikis/zope-xml/Unified%20Vision > > > >_______________________________________________ >Zope-Dev maillist - Zope-Dev(a)zope.org >http://lists.zope.org/mailman/listinfo/zope-dev >** No cross posts or HTML encoding! ** >(Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) > ________________________________________________________________________ Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com

2 1