- Zope-Dev - Zope lists

still segfaults, this time no ZMySQLDA
by Leonardo Rochael Almeida 28 Feb '02

28 Feb '02

Hi, It's me and my instability problems again :-) The system: Zope-2.4.4b1 (binary linux-x86 from zope.org) Python-2.1.2 (see above) Products (excluding Zope defaults and ZClass based ones): LocalFS Pagina0_Base (Hiperlógica product) Pagina1_Base (Hiperlógica product) Renderable exUserFolder It has an 800+MiB Data.fs (when packed) that is crammed full of python scripts. I'm still looking for a way to recompile them all without sending the machine (1GiB physical RAM) into a swap storm which would ultimately crash Zope or the machine and keep it from commiting the transaction that caused all the trouble in the first place (one of these days I'll refactor the subtransaction code out of the ZCatalog and into a ZopeFind that will accept an action parameter, along with a subtransaction one, to perform these kinds of global maintenance in a reasonable way). None of the products mentioned above use C compiled modules. I disabled all the use of mysql on this instance so that I could remove ZMySQLDA and faked the authentication system so that it's usage pattern remained the same even without mysql. After that I reenabled the cycle-gc. The system started crashing about every 5 minutes. As much as I'd have liked pointing a finger at ZMySQLDA, start the process of migration to PostgreSQL and consider this problem solved once and for all, I have finally come to the conclusion that there is no one else to blame but Zope and Python themselves. I have since reenabled ZMySQLDA use by the site and disabled the cycle-gc. The options I think I have left: * Try the new ExtensionClass.h * Run Zope under a memory analiser such as Valgrind (which might not be feasible if the runtimes are too slow): http://devel-home.kde.org/~sewardj/ * Try migrating to Zope-2.5.1b1 The situation is starting to look desperate. We now have to restart the server twice a day to keep the machine from swapping. This can't go on any longer. Any suggestion is apreciated. -- Ideas don't stay in some minds very long because they don't like solitary confinement.

4 11

FieldIndex unindex_object could not remove... This should not happen.
by Florent Guillaume 28 Feb '02

28 Feb '02

Playing with workflow switching in CMF, I sometimes have this message: 2002-02-28T18:33:06 ERROR(200) FieldIndex unindex_object could not remove documentId -862940561 from index review_state. This should not happen. Traceback (innermost last): File /home/zope/test-zope/lib/python/Products/PluginIndexes/common/UnIndex.py, line 168, in removeForwardIndexEntry (Object: review_state) KeyError: -862940561 I gather that this happens when I do self.unindexObject() on an object that had a review_state attribute (actually a workflow variable made available to the catalog through a wrapper) in the past (being in a reviewing workflow) but at the time of the unindexing this variable does not existe anymore because the current workflow followed by the object doesn't have it. Does this ring any bell ? Thanks, Florent -- Florent Guillaume, Nuxeo (Paris, France) +33 1 40 33 79 10 http://nuxeo.com mailto:fg@nuxeo.com

2 5

Help Securing new ftpFolder product.
by Tim Ansell 28 Feb '02

28 Feb '02

Hi I'm currently developing a product called "ftpFolder". This product is designed to make Zope easier to use with webDAV and ftp. It tackles a few common problems that occur when using these tools to edit Zope. Firstly Zope prefers to have no extension on the files it stores, this breaks alot of Windows programs (such as Web Folder and Dreamweaver). The ftpFolder tackles this problem by providing a "virtual mapping" onto the real zope tree. In the virtual mapping the objects gain extensions. This mapping is accessed through 'cd'ing into a virtualFTP object found in the FtpFolder. For example FtpFolder | + DTMLDocument + DTMLMethod + Image | + virtualFTP | + DTMLDocument.ddtml <- This is the above DTMLDocument object + DTMLMethod.ddtml <- This is the above DTMLMethod object + Image.jpeg <- This is the above Image object Extensions are provided using a mapping table and "plugins" which can dynamic create the extension depending on contents. (ie the Image would have a plugin to tell if it was a .jpeg or .gif or .png etc) It does this by providing "virtualObjects" which map properties onto a real object. I need a way to define the security so that the virtualObject takes on the security of the real object it is wrapping. One way that was suggested to me was to "copy" the security properties from the real object to the virtualObject in __init__. Would this work? If so what would i have to copy? Another problem the ftpFolder solves is the lack of the ability to edit Zope properties. It does this py providing a "virtualProperties" object which acts as xml representation of the objects properties. This objects appear in the virtualFTP folder as the <document>.prop and can be uploaded and downloaded to access and change the documents properties. This works well but i need help to make sure this doesn't open up any security holes allowing unprivledged people from changing properties. The final and most complex problem is that the ftpFolder allows you to ftp into "versions objects". In the virtualFTP tree version objects become "folderish", this allows people to 'cd' into them, and access the the tree as if they where in that version. How this works is in 3 steps a "versionFolder" object is created to "hold" the version. When anybody accesses the versionFolder is xml-rpc calls a "do" method on the local zope with the correct headers to cause the version to be entered. The "do" method then calls the correct method on the object (because the methods don't have doc strings and can't be called directly - plus REQUEST isn't picklable so can't be send across xml-rpc). versionFolder <- XML-rpc -> virtualObject.do <- function call with "mapply" -> virtualObject.<function> This might seem a horrible way to do it but it's the only way i have found to do this. It works suprisingly well, the problem is that it's horribly unsecure, i need help to secure this and the other problems. The source can be found on sourceforge cvs -z3 -d:pserver:anonymous@cvs.iiiserver.sourceforge.net:/cvsroot/iiiserver co ftpFolder If you are intrested in helping me, or can offer some suggestions could you please email me (as well as the list if you feel it's approiate). Hope to hear from you all soon. Mithro PS BEWARNED the source isn't exactly the kindest :(

1 0

Possible Bug in Helpsys
by Andre Schubert 28 Feb '02

28 Feb '02

Hi all, These days i worked on a Help for my ZClass-based Product and found out, that the HelpSys-Topics are unsorted in menu.dtml. If have changed <dtml-tree> to <dtml-tree sort=id>, so it is possible to sort the HelpTopics in a logical way. Should this be changed in further versions of zope or not ?? -- Andre Schubert EMail: Andre.Schubert(a)km3.de Tel: 03774 6625-78 km3 teledienst GmbH Fax: 03774 6625-79

2 2

Re: [Zope-dev] still segfaults, this time no ZMySQLDA
by Anthony Baxter 28 Feb '02

28 Feb '02

>>> Leonardo Rochael Almeida wrote > So, what should I try now? I've been unable to keep up with email lately (no decent net connection) but can I ask if you've got a core dump at all? If so, where was it? Inside the garbage collector, I'm guessing... If so, what was the object _before_ the corrupted object in memory? (Had I thought to look at this when I had similar problems, I would have picked up the stupid stacksize bug much earlier...) If you haven't recompiled the scripts after upgrading to a working PythonScripts, you'll still see crashes. When you run with '-D', do you get a lot of "Script (Python) foo needs to be recompiled"? If the problem is that you can't recompile the scripts, why not shut it all down, manually mount the ZODB, and recompile 10 scripts at a time? (This is a not-particularly difficult task - I can give a sample bit of source if it would help...) Anthony -- Anthony Baxter <anthony(a)interlink.com.au> It's never to late to have a happy childhood.

1 0

how to reach Zpublisher without passing by ZServer ?
by Olivier Moïses 27 Feb '02

27 Feb '02

Hello, Does anyone know how to reach Zpublisher without passing by Zserver from any client ? I explain my question : I would like to send XML Fragments from a client (Mozilla) to Zope. But, in HTTPRequest.py, I read that when the content-type of the request is text/xml and the method is post (exactly like in most of the cases I process), automatically, Zope process the request as XML-RPC. The XML-RPC only support ASCII (7-bits characters encoding) in the specs, and I want to encode at least UTF-8 in request and response. I could base64 encode all my data, but I would like to avoid the overloading of this encoding. Is there a (simple) way to add another protocol than XML-RPC, Webdav, HTTP, and FTP to Zope ? Is there a cool way to extend Zserver, or directly access the ZPublisher from another source ? Regards Olivier Moises

1 0

Re: Zope vs. Cocoon
by Craeg K. Strong 27 Feb '02

27 Feb '02

Hello: I am the author of this paper (mea culpa! :-) and I thought I would respond to your comments. First off, thanks for thorough and thoughtful review. My responses are embedded below: Stefano Mazzocchi wrote: >First, kudos to Gianugo for finding the ask-slashdot thread about 'zope >vs. cocoon', one user suggested to take a look at > > >http://www.arielpartners.com/arielpartners/content/public/topics/technology… > >which I think is a good review of the current status of the two >projects. > >Yes, I've installed and used Zope in the past, even if I never created >something that worked with it. > >Here I will try to address the points of the article. > > - o - > >>The paper concludes that, despite a few annoying limitations, Zope >>is a much more powerful, mature, and well-documented environment >>and probably holds a one to two year lead over Cocoon and other >>similar Java-based publishing environments. >> > >This quote says it all and there are a few things that I have to admit >myself: > >1) well-documented: cocoon documentation is currently very poor compare >to Zope's or compared to other successful technologies (say PHP). It is >clear that without good docs, there is no good product. > >But Cocoon has been released final 3 months ago and it's growth rate is >clearly impressive, even for my apache experience. > >Anyway, I would venture to forecast that in a year, Cocoon will *easily* >cover this difference, also because many books are planned to appear. > This is good news. Zope now has four (English language) books out, plus two more in the works. I see there is a Cocoon book to be published in June, but from what I understand it covers the Cocoon1 architecture ? >2) mature: granted, you can't have a 3 months-old technology and >consider it 'mature'. At the same time, we've been working on this for >two years and we know where we're going. There is very little doubt that >Cocoon will mature and solidify soon. > Based on experiences with Cocoon1 and the traffic on cocoon-dev, I would agree. >3) powerful: on the technological side of things I dare to disagree with >them. Those 'few annoying limitations' they describe, I found them 'very >deep architectural mistakes'. > >They cite XSLT support, but that's nothing compared to the fact that >there is no way for you to create a Zope site without the user 'knowing >it'. No matter how hard you try, it shows. It's 'zopy', they all look >the same, much like 'this is a manila-site' > >Some don't think this 'cloning restriction' a severe limitation, I think >this is not a annoyance, but the *first* rule. > I agree that this is a very important consideration. However, I cannot agree with your observation. Zope powers many more sites than those of which you may be aware. Unfortunately, I don't know too many of them personally, but here are a few: http://www.activestate.com http://www.homegain.com http://www.arielpartners.com (I couldn't resist :-) According to news at the latest Python conference (http://www.python10.org) Zope is now powering the websites for one of the major US television networks (ABC? for the life of me, I can't remember which). Hopefully one of the Zopistas out there can help me out... >But this hits a very serious point: comparing Zope and Cocoon is like >comparing apples with oranges, technologically. > >Boths are fruits, as both publish web content, but Zope is a 'publishing >environment' while Cocoon is a 'publishing framework'. > >An 'environment' is an application that you customize, a 'framework' is >the foundation of your own application. > I might define those words differently, but be that as it may. Here is some information about the Zope architecture: - both Zope and Cocoon2 come with an example/tutorial that can be customized. Just because Zope's tutorial is more fully fledged does not mean that it cannot be as radically changed. - Zope consists of an integrated object-oriented database (ZODB) and a webserver (ZServer, based on Medusa). It is possible to use ZODB with Apache in lieu of ZServer. - Zope provides a 100% object-oriented abstraction for web publishing. URLs are turned into method calls on objects. The objects are stored in ZODB. Of course, you can create your own classes that obtain their content from the file system, rather than the ZODB (almost like a symbolic link or alias). - Any kind of python object can be stored in ZODB. You can define your own objects, or use one of their predefined classes. Therefore, you immediately have access to all the python libraries, such as the XML libraries. - Because of acquisition, you can add behavior to objects without changing their class definitions. - Because (much of) Zope content is contained in the ZODB, it is convenient to use ZServer because it provides access to objects contained in ZODB via FTP/webDAV. This makes it possible, for example, to edit the contents of any Zope object (provided you have sufficient permissions) using Emacs-FTP (see http://www.cs.indiana.edu/elisp/ange/ange-ftp_toc.html) as if it were a local file. - The latest version of Zope (2.5.1beta1) is now using the "standalone" version of ZODB. Both ZServer and ZODB are now modular pieces that can be used separately or together, as desired. Doubtless, many people will continue to use them in their integrated form. >This is why I liked Zope as an environment, but didn't like Zope as a >framework: as an environment, all sites look similar, as a framework, >all sites might be totally different. > >Just ask you this simple question: could you power cnn.com with zope >without noticing any difference? could you with Cocoon? > 100% yes for both. > >This tells you all. > > - o - > >If you take a look at the table of feature comparison, you'll see that >Cocoon 'can' provide the functionality that Zope already has, but has a >module, an extension, or, as they say 'with lots of work'. > >This is a good thing! > >I believe that Zope is mis-placed architecturally, it's an hybrid >between a CMS and a publishing framework. And does some of everything, >but both poorly, compared to specialized solutions. > Actually, there is a CMS available for Zope: the Zope Content Management Framework (see http://cmf.zope.org) It is a layer on top of Zope that provides additional functionality such as workflow, syndication, "portlets", and reskinning. Apache Jetspeed provides some similar functionality to Zope CMF (though not all). We chose not to use the Zope CMF because of its architecture: it is not based on standard XML technologies and, in our opinion, brings us too far into the "proprietary language land." That was an important consideration for us. In fact, I can tell you for certain that, if Zope were to disappear tomorrow (god forbid!), we could rehost our entire website in Cocoon2 without too much trouble because >90% of it uses XML/XSLT technologies. Having said that, we are quite happy to have all of Zope's functionality available to us. >This is why I'd love to see Cocoon web applications modular: so that >people will start adding 'out-of-the-box functional modules' that zope >currently ships and people expect. > >Forrest was born to provide such a module for information publishing of >OSS communities. > >But you could have a 'webmail module', a wiki module, a weblog module, a >calendar module and so on. > >But let's look at the features that Zope provides out of the box and >Cocoon doesn't: > >1) Integrated Object-oriented database with support for full graphical >editing of all objects > >Do you really want this? I don't. > Perhaps I did not explain it well enough. Many of the threads on cocoon-dev lately have been about sitemaps and flowmaps. They quickly become complex. We want a simple but powerful language to express content objects and flows. We might have intermediate pieces of content created by flows that then get processed differently by different sub-pipelines (like the UNIX "tee"). Zope provides a unique solution to this problem. Imagine a folder-like tree representation of your site. It consists of Manager (Folder) objects and ManagerItem objects. Managers are containers for ManagerItems. Managers are represented as folders with a familiar MS Windows explorer-style graphical presentation. How would you express pipelines? That is where acquisition comes in. One way to explain it is that it features a more recursive style, while Cocoon is more iterative. For example, imagine your ZODB contains the following: /content/index_html /content/topics/body /content/topics/swimming/intro /content/topics/sailing/intro In the example above, index_html is a template for a web page. It includes a "hole," or variable called body. If you call http://content/index_html directly, it will complain of an unresolved variable, namely "body." Similarly, /content/topics/body has an "intro" variable. Therefore, if you call the "index_html" method on the object whose URL (read UUID) is /content/topics/swimming, it will "acquire" the index_html method from /content/index_html. This is called acquisition by containment. The index_html method then acquires "body" from /content/topics/body, and finally the swimming intro. Recall that in Python, functions are actually first-class objects-- so for example you can think of /content/index_html as a "method" object that is contained within the /content/ folder object. For Java programmers, think of the GOF Strategy pattern on steriods :-) >2) acquisition: Zope features dynamic run-time inheritance which enables >"context-oriented" programming. This means that objects change their >behavior based on their current context. These changes include not only >different implementations of methods, but also whole new sets of >methods. This provides many of the benefits of dynamic reclassification, >but with arguably more flexibility. Cocoon does not offer such a >feature. > >I can't really tell you what this is, so they might have a point. > (see above) >3) Integrated FTP, WebDav, and HTTP access to all content > >There is no way of making this possible out-of-the-box in cocoon, it's >too application specific. This is a choice. It's possible to provide >webdav support on top of cocoon today! I think I have to provide a >sample to show people how. > This gets back to the "Environment vs Framework" debate. Is an operating system an environment? Just because bourne shell is provided, does that mean I have to use it? Not if I have bash handy! On the other hand, on win2K, you might be stuck with the built-in windows commandline... Zope is more like UNIX in this regard. There are many tools provided, but you only use what you want to use. Of course, integrated FTP and webDAV access is very handy. Can you edit the underlying XML content of your Cocoon-powered site from an airport kiosk? With Zope you can, because every object can be edited directly using the web browser. Again-- this is not the normal way of doing business-- hey, we _live_ in Emacs. But the capability is very handy for those quick fixes and is a major selling point for our customers. An editor certainly makes life easier, but it is not required. This is in keeping with Python's "batteries included" philosophy :-) >4) Seamless support for adding arbitrary metadata to all objects in the >database. > >XIndice will provide this. Cocoon should not have built-in databases, it >mixes concerns. > Agreed. However, if you had a set of integration classes, tutorials, and documentation handy, and provided a separate download that combined the two products with no extra hassle required, how many people do you think would download it? >5) Integration with role-based authentication/authorization security, >all manageable from the web > >Recent code donations have removed this, even if there is lots of >cleanup work to do. > >Then the Cocoon strong points: > >1) Integration with Source Code Control System > >Zope is not file based, it's entirely database based. So CVS doesn't >work on it. > We have made our first baby steps toward solving this problem: http://www.zope.org/Members/arielpartners/CVSFile >They say this is a non-issue with file-based cocoon, but they are wrong: >revisioning is an issue with an eventual CMS on top of cocoon... but >this shows just a more balanced architecture. > This is the never-ending debate between Version Control Systems and Databases. Which do you want? Of course, the answer is both. It is probably only a matter of time before someone introduces a way to integrate CVS and Xindice, for example. (obviously, you could batch-export the whole bloody thing from Xindice and version it, but more granular versioning would be nice) >2) Integration with J2EE and other Java-based business logic > >Cocoon is a servlet, thus we get it for free. They find themselves >completely detached from the rest of the world, even if they could >easily use web-services to glue things. This is a clear marketing plus >for us. > You are absolutely correct in this regard IMHO. In fact, this might be the biggest long term problem for Zope. - Zope is mostly written in Python, but some is (still) in C. This is a pain, because it makes it impossible to run Zope under Jython. - Python VMs are _way_ slower than Java VMs. Python just does not have the equivalent to the hordes of senior scientists working away at Sun's java labs on the "hotspot" technology. Obviously, both VMs will continue to improve, and VM performance is secondary to good architecture and design, but ... - If Zope could be made to run under Jython (http://www.jython.org) integration with J2EE would be virtually a no-brainer, b/c you are already inside a Java VM. >3) Support for XML and XSLT Processing pipelines Not provided. Must >roll your own. This is a lot of work. Cocoon does it out of the box > >What else can I say :) > >No seriously: Zope is a more mature effort, but i strongly doubt that it >can stand the evolution rates that this community exhibits. > I cannot agree with you here. Zope Corporation is the primary driving force behind Zope, so there are actually people out there who get paid to enhance Zope every day :-) I think their release schedule speaks for itself: 2.5.1beta1 2/22/2002 2.4.4beta1 1/28/2002 2.5.0 1/25/2002 2.5.0beta4 1/15/2002 2.5.0beta3 12/23/2001 2.5.0beta2 12/05/2001 2.5.0beta1 11/28/2001 2.4.3 11/12/2001 2.4.3beta1 11/05/2001 That is just since November. In general, they have put out point releases every month and minor releases every six months. Zope 3.0 is currently under development. >Moreover, there is no indication of internal modularity and >extensibility, SoC-based design, IoC design, data storage abstraction... >and no indication on caching strategies, scalability and performance >issues. > Actually, there is a RAM Cache Manager object built in to Zope. This object, like all other Zope objects, can be extended in Python (for example to change the caching strategy or whatnot). Zope Enterprise Objects (http://www.zope.org/Products/ZEO) is an optional add-on product that turns Zope into a distributed architecture, allowing multiple processors, machines, and networks to act as one website. Zope _is_ too difficult to develop for, and it is not very easy to create objects that can be used both inside and outside the Zope environment (i.e. the Zopish aspects are not modular enough). That is what the Zope 3.0 effort is all about. The Zope community has learned from the Java/J2EE architecture: they are refactoring the internals of Zope so that add-ons as well as Zope itself will be component-based. (see: http://www.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/FrontPage ) However, having said that, Zope _is_ amazingly extensible. Consider these statistics: When we first became interested in Zope (Sept/Oct 2001), there were about 300 add-on products available for Zope at the zope.org site. Today, there are over 500 add-ons available. This tells me two things: Zope has an extremely active community, Zope is extensible >My impression is that they use Zope because it gives them the >functionality they need. > >Cocoon is different: it will give you the 'substrate' you need to make >your functionalities grow... and give you a way to modularize these >functionalities to share add compose with others. > >I think we have very little to learn from Zope that we didn't know >already. > After reading my reply, do you still feel that way? > > >But my point is: comparing Cocoon and Zope is an serious architectural >mistake. > I agree they were created to solve different problems. However, there are many reasons why developers may want to compare the two technologies: - both are leading open-source platforms for building websites - other major open source alternatives may no longer be as attractive (recent demise of arsdigita, enhydra's problems, ...) - XSLT is becoming more mainstream (why Zopers might want to consider Cocoon) - Inclusion of Lucene + Xindice with Cocoon narrows the gap In general, based on my assessment of the architectural directions of both projects, they are likely to provide more and more similar functionality as time goes on. When it comes right down to it, I believe the slash-dotters were correct when they said that the main issue may be choice of programming language. If you have a development shop full of Java programmers, are you really going to expect them to learn Python and Zope? (Of course, that is precisely what ArielPartners did :-) Curious to hear your response, --Craeg -- Craeg K Strong, General Partner Ariel Partners LLC http://www.arielpartners.com voice 781-647-2425 fax 781-647-9690 --------------------------------------------------------------------- To unsubscribe, e-mail: cocoon-dev-unsubscribe(a)xml.apache.org For additional commands, email: cocoon-dev-help(a)xml.apache.org

10 12

Mapping non-Zope Files
by Villemin, Remi 27 Feb '02

27 Feb '02

Hi, I want to create a catalog of Zope objects that reflects the structure of an out-of-Zope directory. For all the files in the out-of-Zope directory I would like to have a Zope object containing a link to the actual file. Is there a product that allows that? Thanks a lot, Remi

2 1

problem adding ZClass instances via DTML
by Mark James Adams 26 Feb '02

26 Feb '02

I'm following the instructions in "Adding ZClass Instances Programmatically" (http://www.zope.org/Members/tazzzzz/addZClasses) but Zope always returns an error. Using the code from a DTMLMethod in another ZClass in the same product: <dtml-with "manage_addProduct['MyProduct']"> <dtml-call "MyZClass_add(_.None, NoRedir=1)"> </dtml-with> I get the error: File /usr/local/Zope/lib/python/DocumentTemplate/DT_Util.py, line 339, in eval (Object: MyZClass.createInObjectManager(REQUEST['id'], REQUEST)) (Info: REQUEST) File , line 0, in ? NameError: MyZClass So this is clearly having a problem finding MyZClass from the MyZClass_add method. I then noticed that, according to the HowTo, if you are instantiating a ZClass from within another ZClass, you only need the code: <dtml-call "MyZClass_add(_.None, NoRedir=1)"> However, this gives the error: File /usr/local/Zope/lib/python/DocumentTemplate/DT_Util.py, line 339, in eval (Object: MyZClass_add(_.None, NoRedir=1)) (Info: _) File , line 0, in ? NameError: MyZClass_add Any suggestions? Thanks. -- Mark James Adams mark(a)raysend.com | mja27(a)cornell.edu

3 2

sre and import_class
by Matt Behrens 26 Feb '02

26 Feb '02

re seems to prefer sre over pre these days, so the following bit of Products/PythonScripts/module_access_examples.py no longer works: # from re import RegexObject, MatchObject # allow_class(RegexObject) # allow_class(MatchObject) The reason seems to be that sre doesn't define RegexObject and MatchObject, preferring instead to define types SRE_Pattern and SRE_Match, which can't be imported. Arguably this is probably a deficiency in sre, but as it stands, it seems to be currently impossible to use re inside of python scripts. I'm using an EM. I'll file a Collector issue on this one, unless someone has a solution, or someone wants to just take the 're' section out of module_access_examples.py.

1 0