December 2001 - Zope-Dev

Integrating mailman with Zope?
by Fred Wilson Horch 06 Dec '01

06 Dec '01

Hi everybody, After a hiatus on our project to integrate Mailman and Zope 2.3.x here at EcoAccess, we're about to tackle it again. Has anyone out there tried this already? Last we heard, Mailman was a standalone CGI program, and in order to make it work with Zope you had to run both Mailman and Zope behind another web server (such as Apache). What we're planning to do is write a CGI product for Zope 2.3.3 so that we can run CGI programs with ZServer under Zope, eliminating the need to set up and configure Apache or any other web server. If you've already written a CGI Zope product (i.e., a product that lets you run CGI programs from any Zope folder), please let me know! Otherwise, I'll continue the experiments. ;-) Best regards, Fred -- Fred Wilson Horch mailto:fhorch@ecoaccess.org Executive Director, EcoAccess http://ecoaccess.org/ P.O. Box 2823, Durham, NC 27715-2823 phone: 919.419-8567

3 3

getPhysicalPath differs for function vs. __getattr__
by John Ziniti 06 Dec '01

06 Dec '01

Sorry the subject is not so discriptive, but here is the issue I am dealing with. I wonder whether this has to do with the way Zope deals with objects or has something to do with Python: I have an Python Product class "Object". With the following bits of code, I get different results from a <dtml-var>, depending on what I use. class Object(PropertyManager, Implicit, Item): def get_path(self): """Return a path to myself""" path=string.join(self.getPhysicalPath()[1:], '/') return path def __getattr(self, attr): """Return a path to myself""" if attr == 'path': path=string.join(self.getPhysicalPath()[1:], '/') return path else: raise AttributeError, attr Now the following two dtml tags yield: <dtml-var "get_path()"> => /path/to/object <dtml-var "path"> => object Hopefully you see what I mean. Why does this happen? Can I put it in the collector as a bug? TIA, Ziniti

2 1

reduce() doesn't work in Python Scripts
by Brad Clements 06 Dec '01

06 Dec '01

In Zope 2.4.3 binary on Linux import operator reduce(operator.add,[0,1,2,3,4]) gives me Error Value: global name 'reduce' is not defined Is this a faq? Brad Clements, bkc(a)murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax netmeeting: ils://ils.murkworks.com AOL-IM: BKClements

2 5

ZGDChart-0.6.2.babel restarts my Zope
by Romain Slootmaekers 06 Dec '01

06 Dec '01

Yo, I installed ZGDChart and was playing around with it. cool stuff... But I came across this problem: at first, everything worked fine, but know, whenever I try to view a chart, my zope server restarts. no messages what so ever. I tried the test.py that came with ZGDChart , and that seems to work fine. I don't have a clue on how to tackle this problem except for adding print 'cucu' lines all over the ZGDChart code :( I have a 2.4.0b3 built from source and a 2.1 python (if this might help) Anyone any ideas ?? TIA, Sloot.

2 1

RE: [Zope-dev] Zope server speed
by MANOTTI Alessandro 06 Dec '01

06 Dec '01

Thank you for your suggestions (thanks to Matt). I will try to make smaller but more requests (1000 requests of 1000 records). I will even try to "bypass" Zope and write an "external method" in Python (I know Pytho very well). About this solution: can you tell me what I have to do to use ZOracle directly in Python (import ... ? ) ? Thanks to everybody! -----Original Message----- From: Matthew T. Kromer [mailto:matt@zope.com] Sent: Thursday, December 06, 2001 15:27 To: MANOTTI Alessandro; zope-dev(a)zope.org Subject: Re: [Zope-dev] Zope server speed Hi Alessandro, Zope and Python combined aren't too bad if you write some wrappers that help you manipulate data; the problem comes about when you try to do large naive queries -- there is a LOT of overhead in taking Oracle data, encapsulating it in Python objects, and then wrapping that in the Zope machinery. I have a sample database in Oracle of US Zip codes -- it has about 45,000 records in it. I can pull this into DCOracle2 on the C adapter side (dco2.so) in less than a second. However, wrapping each result cell in a python object blows that up into about 9 seconds. If I were to iterate over the whole thing in Zope (which I haven't timed) it would get a LOT longer, because Zope is enforcing policy on every object accessed. So, one of the ways to make that go faster is to use python modules 'outside' of Zope (ie external methods) to perform large calculations which takes a ZOracleDA connection object to get a cursor object on the database, does the query directly, and formats and returns the results. I realize this isn't ideal, but Zope is not optimized for the case of "extract thousands of records from the database and format them in a huge table." Zope likes "extract hundreds of records from the database and format them in a big table" instead. With each release, Zope has more features that enable you to program through the web, but when you're doing performance intensive calculations, it's usually time to write a python module and plug it in to zope (via an external method, or by writing a product). If you have the time to learn, I suggest the product route as being more satisfying, but it can be more frustrating as well. ----- Original Message ----- From: "MANOTTI Alessandro" <alessandro.manotti(a)bfeurope.com> To: <zope-dev(a)zope.org> Sent: Thursday, December 06, 2001 8:52 AM Subject: [Zope-dev] Zope server speed > > > HI all, > > > > I have a problem: > > > > my company is looking for an application server. I know Zope (I presented > > it), and a friend of mine presented Macromedia ColdFusion. > > > > We made some tests about the speed of the systems, arghh!!! Zope is VERY > > slow compared to ColdFusion! > > > > Some examples: > > > > I used Zope with DCOracle2 drivers to connect and extract 257000 (yes! it > > is a test... two hundred fiftyseven thousand records!) and Zope took more > > than 10 minutes, since I stopped it after that time (call SQL and print > > data via <dtml-var ... > TAG). > > > > The same PC, the same server, ColdFusion took about 2 minutes to complete > > the operation!!! > > > > Coldfusion is more than 5 times faster than Zope!!! > > > > PHASE 2: > > > > I said: ok, maybe Zope is slower that Coldfusion in TAG formatting... and > > I tried to get the records and perform a calculation, without no output... > > > > Sigh! the results are the same! > > > > I am depressed, since I talk about Zope very well in my company, but > > now... > > > > Coldfusion and Zope are similar, so we shall select ColdFusion to develop > > web applications. > > > > SIGH !!! > > > > Can I try something to increase Zope power?! > > > > > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > Alessandro Manotti > > Presidente dell'Associazione "RIUSA" > > > > Sito web: http://riusa.apritisesamo.net > > email: riusa(a)email.it > > mailing-list: riusa-ml(a)yahoogroups.com > > > > > > _______________________________________________ > Zope-Dev maillist - Zope-Dev(a)zope.org > http://lists.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) >

2 1

Zope server speed
by MANOTTI Alessandro 06 Dec '01

06 Dec '01

> HI all, > > I have a problem: > > my company is looking for an application server. I know Zope (I presented > it), and a friend of mine presented Macromedia ColdFusion. > > We made some tests about the speed of the systems, arghh!!! Zope is VERY > slow compared to ColdFusion! > > Some examples: > > I used Zope with DCOracle2 drivers to connect and extract 257000 (yes! it > is a test... two hundred fiftyseven thousand records!) and Zope took more > than 10 minutes, since I stopped it after that time (call SQL and print > data via <dtml-var ... > TAG). > > The same PC, the same server, ColdFusion took about 2 minutes to complete > the operation!!! > > Coldfusion is more than 5 times faster than Zope!!! > > PHASE 2: > > I said: ok, maybe Zope is slower that Coldfusion in TAG formatting... and > I tried to get the records and perform a calculation, without no output... > > Sigh! the results are the same! > > I am depressed, since I talk about Zope very well in my company, but > now... > > Coldfusion and Zope are similar, so we shall select ColdFusion to develop > web applications. > > SIGH !!! > > Can I try something to increase Zope power?! > > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > Alessandro Manotti > Presidente dell'Associazione "RIUSA" > > Sito web: http://riusa.apritisesamo.net > email: riusa(a)email.it > mailing-list: riusa-ml(a)yahoogroups.com > >

4 3

Re : Open Letters and Zope 3
by Godefroid Chapelle 06 Dec '01

06 Dec '01

Q: What is Zope 3X? A: Zope 3X is Zope rebuilt from the ground up, applying the lessons learned from Zope 2 and CMF. <snip> Q: Will Zope 3X support DTML? A: Probably not. Q: Will Zope 3 support DTML? A: Yes. I would like to get a better idea of what 'Zope rebuilt from the ground up includes'. For me, Zope is also used to produce non Markup language (for ex emails) which you cannot use ZPT for. So would the new Zope been limited to ML ? -- Godefroid Chapelle BubbleNet sprl rue Victor Horta, 18 / 202 1348 Louvain-la-Neuve Belgium Tel + 32 (10) 459901 Mob + 32 (477) 363942 TVA 467 093 008 RC Niv 49849

9 13

Re: [Zope-dev] More signal 11 restarts....
by Gilles Lavaux 06 Dec '01

06 Dec '01

Hello Which config/tool make that your zope restart automatically??? Gilles -----Original Message----- From: Leonardo Rochael Almeida <leo(a)hiper.com.br> To: zope-dev <zope-dev(a)zope.org> Date: Wednesday, December 05, 2001 20:07 Subject: Re: [Zope-dev] More signal 11 restarts.... > > >Hi Harald, > >On Wed, 2001-12-05 at 14:32, Harald Koschinski wrote: >> Daniel Duclos wrote: >> >> > I have a zope that is dumping signal 11 every 40 minutes or so. I have >tried >> >recompile python 2.1.1 with-threads without-pymalloc, recompile Zope with it, >> >recompile ZPAtterns, recompile and instal MYSQL for Python 0.9.1, upgraded to >> >Zope 2.4.3, all this on a Debian Linux box. >> > Nothing changed... still restarting... >> > Anybody, please, has any ideia on this matter? >> > Please, let me know if there's any relevant info that I forgot to mention >> >abot my case! >> > Thanks in advance!! >> > >> >> I have the same problem since we are live :-((( >> >> I have the same versions running like you and I tried the same to fix - >> but no change. >> >> What is your state with this problem. Fixed ? How? > >It's not fixed, but we managed to make it bearable so as not to lose the >client. > >by replacing LoginManager with exUserFolder we managed to bring down the >zope restart time from 5 min (we have a HUGE Data.fs) to 20 secs, and by >installing ZEO we brought down the restart time to between 2 and 8 secs. >By increasing the caching of requests we managed to increase the time >between restart from 15 min to 2 hours. That and a nicely formated >Apache error page for Proxy Errors, for the lucky bastards who happen to >hit the server in the exactly 2 seconds of restart, managed to calm down >the client enough for us to breath. > >I'll try the requestprofiler tip Chris gave, but I don't have much hope, >since we use mysql for authentication and the site is authenticated >mostly everywhere. > >We are seriously considering droping mysql for postgresql. > > Cheers, Leo > >-- >Ideas don't stay in some minds very long because they don't like >solitary confinement. > > > >_______________________________________________ >Zope-Dev maillist - Zope-Dev(a)zope.org >http://lists.zope.org/mailman/listinfo/zope-dev >** No cross posts or HTML encoding! ** >(Related lists - > http://lists.zope.org/mailman/listinfo/zope-announce > http://lists.zope.org/mailman/listinfo/zope ) > > >

2 1

Re: [Zope-dev] Searching/Indexing/ZODB/SQL/BerkleyDB
by Chris Withers 06 Dec '01

06 Dec '01

Casey Duncan wrote: > > I'm not sure I want to store the indexes in the ZODB, just index ZODB data at > a low level. Ah, okay, and yes, in that case, I am in complete agreement ;-) (the level I'm aiming at is just to be able to index python objects, I'll leave plugging that into the ZODB architecture up to someone who understands it better...) > Yup, I think I have a solution, but it'll involve some coding ;^) Ooo...care to explain? :-) > I would rather avoid having to use a relational database unless I have to. > Perhaps the index pluggability could be made to support different backends > (like FileStorage et al does). Yeah, unfortunately, the difficult bit is combining queries: gimme the results where index1=='fish' and index2 is between 2 and 5kg. if index1 is in SQL and index2 is in ZODB, for example, how would you go about efficiently combining results? > > That said, I wasn't aware of Matt's work up until very recently. I'd love > > to see an Indexer that didn't require an RDB (or BerkleyDB :-P) and scaled > > to GigaBytes of Data... > > Yup, me too. Well, I'm just purchasing my copy of Managing Gigabytess now ;-) > OK, I'm available all this week, but I'm not as available the next two weeks. > Lets find a good time. I'm available any time and date, just as long as I get a coupla days notice... cheers, Chris

6 19

corrupted data.fs
by Chris Kratz 05 Dec '01

05 Dec '01

Hello all, My apologies if this is not appropriate on this list and my apologies for this "book". But, we have been having more and more problems with Zope and it is getting quite frustrating. I thought since I had posted on the zope list and no one had any suggestions that I would try this list next. The behavior we are seeing is as follows. We are using versions heavily. There are two of us primary developers, soon to be three working heavily within the same data.fs. We use versions, often running several at the same time. When the code is ready, we save the version incorporating the changes to the live site. Ever since we upgraded to Zope 2.4.1, saving the version rarely actually does that. It will save some objects, but usually only a few. We have to then go to each object and save it again (sometimes making changes) and continue saving the version until we get all the locked objects unlocked and committed. This is very annoying, but not a show stopper. What is starting to scare me is that I wonder if this problem with the version commits is somehow causing the corruption of the database. Now, I say all this because it may or may not be related to the other problems we continue to have. We periodically, get folders that are no longer visible within the ZMI and produce a keyerror in filestorage.py (traceback at end of msg). But, the same folder is still happily being served externally (ie it generates the web pages based within that folder without error to our customers). This tells me that it is still in the ZODB, it is just unreachable via the ZODB. Since we cannot access it from within zope any longer, we have to load the data.fs from backup and restore the offending folder losing any work done since the previous night. It never seems to effect any other parts of the website and it has happened in several different areas. This is becoming more and more of a showstopper especially when we lose code. 1. I have packed the database. 2. I have run fsrecover.py over the data.fs. - Nothing to recover. So, if we aren't corrupted, why can we no longer work with that folder? 3. I have exported and imported the offending folder without effect (XML and regular). 4. We were packing the zope database nightly via a cron job to control ZODB bloat. I have turned that off for now on the theory that somehow mixing overnight packs with versions is causing problems. Help!!! This continues to happen and it is really scary when your database continues to get corrupted underneath you on a live site when you are in heavy development. Can anyone give any ideas on how to track down this problem? Could it be with the versioning code? Is there a better way to work with a production site and still get the benefits of the versioning? Is versioning just dangerous and shouldn't be used? Pardon my tone, but I am a little frustrated. I will say that the last time this happened, thankfully, that particular folder was being worked on in a version and even though the folder was inaccessible outside of the version, when working within the version, it was still there. So, when that version was cut in, we were again able to work with that folder. This time however, the folder in question wasn't begin worked on in a version. Thank you for your time, Sincerely -Chris ------------- **Traceback** Error Type: KeyError Error Value: ?f Traceback Traceback (innermost last): File /usr/local/zope2.4/Zope-2.4.0-linux2-x86/lib/python/ZPublisher/Publish.py, line 171, in publish File /usr/local/zope2.4/Zope-2.4.0-linux2-x86/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: manage_main) File /usr/local/zope2.4/Zope-2.4.0-linux2-x86/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: manage_main) File /usr/local/zope2.4/Zope-2.4.0-linux2-x86/lib/python/Shared/DC/Scripts/Bindings.py, line 324, in __call__ (Object: manage_main) File /usr/local/zope2.4/Zope-2.4.0-linux2-x86/lib/python/Shared/DC/Scripts/Bindings.py, line 354, in _bindAndExec (Object: manage_main) File /usr/local/zope/lib/python/App/special_dtml.py, line 244, in _exec (Object: manage_main) File /usr/local/zope/lib/python/DocumentTemplate/DT_In.py, line 711, in renderwob (Object: objectItems) File /usr/local/zope/lib/python/DocumentTemplate/DT_In.py, line 839, in sort_sequence (Object: objectItems) File /usr/local/zope2.4/Zope-2.4.0-linux2-x86/lib/python/ZODB/Connection.py, line 519, in setstate File lib/python/ZODB/FileStorage.py, line 588, in load (Object: /usr/local/zope/var/Data.fs) File lib/python/ZODB/FileStorage.py, line 564, in _load (Object: /usr/local/zope/var/Data.fs) KeyError: ?f ---------------- **Instalation Info** Platform: RH 7.2 Zope Version: (Zope 2.4.1 (binary release, python 2.1, linux2-x86), python 2.1.1, linux2) Python Version: 2.1.1 (#1, Aug 8 2001, 11:57:42) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] System Platform: linux2 Products installed: eCrumbler (Installed product CookieCrumbler (CookieCrumbler-0.3)) ExternalMethod (Installed product ExternalMethod (External Method-1-0-0)) MIMETools (Installed product MIMETools) MailHost (Installed product MailHost (MailHost-1-2-0)) OFSP (Installed product OFSP (OFSP-1-0-0)) PlugIns (Installed product PlugIns (PlugIns-0-4-3p1)) PluginIndexes (Installed product PluginIndexes) PythonScripts (Installed product PythonScripts (PythonScripts-2-0-0)) Renderable (Installed product Renderable) SiteAccess (Installed product SiteAccess (SiteAccess-2-0-0)) Squishdot (Installed product Squishdot (Squishdot-1-2-1)) StandardCacheManagers (Installed product StandardCacheManagers (StandardCacheManagers-1-1-0)) ThreadSafeCounter (Installed product ThreadSafeCounter (0.0.3)) TinyTablePlus (Installed product TinyTablePlus (TinyTablePlus-0-9)) UpdateSupport (Installed product UpdateSupport (UpdateSupport-1_0_0)) WarpFramework (Installed product WarpFramework (2001-05-02-1222)) ZCatalog (Installed product ZCatalog (ZCatalog-2-2-0)) ZGDChart (Installed product ZGDChart (ZGDChart 0.5.1)) ZGadflyDA (Installed product ZGadflyDA) ZPatterns (Installed product ZPatterns (ZPatterns-0-3-0)) ZPoPyDA (Installed product ZPoPyDA) ZPsycopgDA (Installed product ZPsycopgDA) ZSQLMethods (Installed product ZSQLMethods) ZSyncer (Installed product ZSyncer) ZWiki (Installed product ZWiki (ZWiki-0-9-6)) ZopeTutorial (Installed product ZopeTutorial (Zope Tutorial 1.0)) iTrack (Installed product iTrack (0.2.0)) -- Systems Analyst/Programmer VistaShare LLC chris.kratz-no-spam-please-vistashare.com (replace -no-spam-please- by @) www.vistashare.com

2 1