- Zope - Zope lists

Problem: performance of retrieving properties from objects...
by Antwan Reijnen 06 Apr '02

06 Apr '02

Hi all, Due to a deadline in one week (yeah, I know, shame on stupid me!!!) and some problems I ran into, combined with my still too low Zope-experience (learning fast!), I could use some experience in datastorage matters in Zope... Of course, I would like to experiment myself as much as possible, but well, time is running out... I am experimenting with adding and retrieving lots of properties on propertysheet objects, within ZClasses. I find the performance of retrieving large sets of properties (say 400 props, strings, lines etcetera) in this way, way too slow for my application (on my slow pc (300 MHz) it takes 30 seconds). Also, while holding the number of retrieved properties constant, the retrieval-proces gets slower when the propertysheet contains more properties ;-( For the sake of the discussion, suppose I use propertyItems() on a propertysheet to retrieve the data. So now I am forced to experiment further with another way to store my data. So maybe one or two experienced ZopeGuru's can shed a little light on the following possibilities (so I have to experiment a little less in my free (whoohaha) weekend, myself ;-): 1) Suppose I build a new python baseclass in which the bulk of my properties are stored. My slow ZClass object (actually a CMF object) uses this as a base class, and I write my own data-retrieval methods. How much will the increase in performance be? Would it be worth the while? Will I still be able to use the portal_catalog to catalog some interesting properties (I guess yes). 2) Suppose I have to switch to a database (there goes my weekend and free evenings next week). I want to retrieve the properties as one record then. I think the number of 400 props will decrease in this matter, too. I assume the database will retrieve these results a LOT faster. How much will the overhead be, however, to send such a dataset back to Zope...? How about using this database in combination with an existing catalog? 3) Suppose I build a CMF product entirely in python (may god help me), and I store all these properties as mentioned in (1), would the performance be higher than in case (1)? Pfff, back to my application and coffee now. Any help and insight will be greatly appreciated... When I have time, I'll write a summary or a How-To on how to handle these kind of storage problems... Greetings, TIA, Antwan.

2 1

zope, fastcgi, and large files
by alt 06 Apr '02

06 Apr '02

I'm currently using zope 2.3.3, apache 1.3.20, and fastcgi. So far it works quite well. However, I'm trying to support large transfers, protected via a acl_user (passwords, expiration, etc.). Each download/transfer could be 20-50mb. Initial implementation is done by setting the response header to be an binary attachment, then use the RESPONSE.write() command to stream out the data. However, it appears FCGIServer buffers the *entire* 20mb before sending it out to the apache mod_fastcgi. This won't work for me, as it's a huge memory hog, plus there's a noticeable delay before the buffer is flushed out to apache. The FCGIChannel sendDataRecord() and push() shows that buffering is turned on, due to some thread related issue. Forcing it off indeed shows problems when there are multiple connections If I didn't have to deal with passwords/expirations, I could have done this outside of zope. Is there a better way to handle this? How does www.zope.org deal with this? Thanks, Albert Ting (alt(a)sonic.net)

1 0

Re: ZGDChart refresh
by Ayhan Ergul 06 Apr '02

06 Apr '02

In my experience, IE was particularly persistent about caching images, even with all sorts of http headers (pragma, etc.) in place. I ended up referencing images with an additional URL parameter that has a randomly generated value, eg: http://www.foo.com/charts/mychart?param1=12&param2=34 becomes http://www.foo.com/charts/mychart?param1=12&param2=34*anticache=99582223529 this fools the browser into thinking it's a different chart every time since the URL looks different (assuming your RNG is sufficiently good). Obviously the side effect is that now every image gets to be downloaded every time even if it hasn't changed. This was the behavior I needed (with realtime data), so no problem. If your data is determined solely by parameters instead, then you might construct your URL to reflect those parameters, so the same chart gets loaded as long as the parameters are the same (good for static/quasi-static data). Ayhan Ergul --- In zope@y..., Derek Simkowiak <dereks@r...> wrote: > -> Control/Shift when reload or something like that > > No, no, no... he needs a solution for his *customers*. He's > looking for a "pragma: no-cache" header or some such. > > How does one tell Zope to send no-caching HTTP headers? > __________________________________________________ Do You Yahoo!? Yahoo! Tax Center - online filing with TurboTax http://taxes.yahoo.com/

1 0

performance concern
by Bill Ort 05 Apr '02

05 Apr '02

Hi, We are new to Zope and are trying to create a form- and db-intensive web application using Zope 2.4.0 (though I'm testing in 2.5 also). I've noticed that our complex forms (which use nested dtml-in loops) are running extremely slow, taking more than 30 seconds on the server side (450MHz Pentium II w/ linux). So I created a snippet benchmark and noticed something peculiar, in that running a simple loop for the first time is often very slow, however in subsequent refreshes of the document the loop runs almost 8-10 times faster. I turned off caching as much as I know how (in the browser and zope). Here is the benchmark I'm using: <dtml-var standard_html_header> <dtml-comment>Note: My standard header turns off browser caching.</dtml-comment> <dtml-call "REQUEST.set('starttime',_.DateTime().millis())"> Start Time (ms): <dtml-var starttime> <dtml-call "REQUEST.set('testlist',['1','2','3','4','5','6','7','8','9','10'])"> <dtml-in "testlist"> <dtml-var sequence-item>  </dtml-in> <dtml-call "REQUEST.set('endtime',_.DateTime().millis())"> End Time (ms): <dtml-var endtime> <hr size=1> Delay (ms) = <dtml-var "endtime - starttime"> <dtml-var standard_html_footer> Any help would be extremely welcome...as is, the site is too slow to demo to potential clients. Thanks, Bill Ort billort(a)billort.com

8 8

create instance: global Class name not defined
by florian.konnertz＠web.de 05 Apr '02

05 Apr '02

Hi to you ! I installed a new product and its first ZClass (the example in the book from beehive). When I wanted to add my first 'buch' object, I got the following strange error: Let's see, do you have an idea? cu, Florian Traceback (innermost last): File /usr/local/dc/Zope-2.5.0/lib/python/ZPublisher/Publish.py, line 150, in publish_module File /usr/local/dc/Zope-2.5.0/lib/python/ZPublisher/Publish.py, line 114, in publish File /usr/local/dc/Zope-2.5.0/lib/python/Zope/__init__.py, line 158, in zpublisher_exception_hook (Object: BuchClass_factory) File /usr/local/dc/Zope-2.5.0/lib/python/ZPublisher/Publish.py, line 98, in publish File /usr/local/dc/Zope-2.5.0/lib/python/ZPublisher/mapply.py, line 88, in mapply (Object: BuchClass_add) File /usr/local/dc/Zope-2.5.0/lib/python/ZPublisher/Publish.py, line 39, in call_object (Object: BuchClass_add) File /usr/local/dc/Zope-2.5.0/lib/python/OFS/DTMLMethod.py, line 127, in __call__ (Object: BuchClass_add) File /usr/local/dc/Zope-2.5.0/lib/python/DocumentTemplate/DT_String.py, line 473, in __call__ (Object: BuchClass_add) File /usr/local/dc/Zope-2.5.0/lib/python/DocumentTemplate/DT_With.py, line 61, in render (Object: BuchClass.createInObjectManager(REQUEST['ArtNr'], REQUEST)) File /usr/local/dc/Zope-2.5.0/lib/python/DocumentTemplate/DT_Util.py, line 159, in eval (Object: BuchClass.createInObjectManager(REQUEST['ArtNr'], REQUEST)) (Info: REQUEST) File <string>, line 2, in f NameError: global name 'BuchClass' is not defined thanks, Florian P.S. I'm sorry I forgot the subject in my previous posting.. -- I'm very tired... ______________________________________________________________________________ Darf es ein bisschen mehr sein? Mehr Speicher, mehr Mail, mehr Erlebnis, mehr Leistung, mehr Prämie unter http://club.web.de/?mc=021102

1 0

Modal behaviour wanted!
by Luca Manini 05 Apr '02

05 Apr '02

Hi, I'looking for a standard and flexible way (a Zope idiom) to do in Zope what's really easy to do in a normal GUI (no web) for a very common problem. Suppose I have an index_html that lists some items. Each item has a link that allows to edit it. After edit I want to come back to the list [see 1]. The edit process will probably require: 1) a form (edit_html) to be displayed (to collect data), 2) a PythonScript (edit_script) to do the actual job, 3) a form (edit_ack) to be shown (to say OK or ERROR) If everything goes fine I will (probably) like to go back to index_html, if there is an error I will like to (at least once) go back to edit_html. Of course I don't want the edit_html to know (hard-coded) that its "backlink" is index_html (it will not always be the case). I can accept edit_script and edit_ack to know who call them (it will be probably be edit_html), but also in edit_ack it would be nice to have a way to go back to the original caller (with a Cancel button). When doing everything from a Python Product, things are somewhat easier because you are always in the context of the edited/added object and you can mantain some "editing state" and return the appropriate form; but how to do it just with DTML Methods and Python Scripts? Any hint? TIA, Luca. PS: Almost all the "small" Zope examples I've seen do not help a lot in this regard because quite often they use the FORM action and a single button (too simple for real cases), or they use a single kitchen-sink DTML with <dtml-if> and some trick to hold state (does not scale well). [1] Note: that's the same like adding an Folder to a Folder, but: 1) after adding the folder, you're sent back to the "right" place because everything is done by manage_addFolder that is called in the context of the contaning Folder and has the "backlink" (self.manage_main) hard-coded in its python code. So this is not a general "solution". 2) If everything goes fine you do not get any confirmation (ok you get the list and you see the new Folder), 3) If there is an error (ex: duplicated id) you are left with a Zope traceback with NO LINK to the original place.

4 4

Error running fsrecover.py
by Oliver Erlewein 05 Apr '02

05 Apr '02

Hi all When trying to run fsrecover.py I get the following error: ~/.../var>python /.../lib/python/ZODB/fsrecover.py Data.fs.old Traceback (most recent call last): File "/.../lib/python/ZODB/fsrecover.py", line 92, in ? import FileStorage File "/.../lib/python/ZODB/FileStorage.py", line 194, in ? from TimeStamp import TimeStamp ImportError: No module named ExtensionClass What am I doing wrong? Regards Oliver Erlewein -- GMX - Die Kommunikationsplattform im Internet. http://www.gmx.net

3 2

urgent contact
by Uman Garuba 05 Apr '02

05 Apr '02

I am Uman Garuba, an Engineer with the federal ministry of petroleum resources, I gotyour contact through a friend whom I told I needed an assistance on some information, although I did not disclose thenature of business to him, since it> demands absolute secrecy. The Tiny republic of Benin a country in west>Africa discovered crude oil, blackgold in Portnovo>and>since they're not members of organization of Petroleum Exporting Country (OPEC) they requested>for our assistance through our ministry whose subsidiary>is the NNPC, we were able to assist in drilling and> excavation of this black gold and also spot lifting.>As a> result of the spot lifting an excess of $48.5million>US> dollars was recorded and it is in ourfavor, my> colleagues then decided we shouldlook for an>honest and sincere person who could claim>responsibility>of this amount because the law of our country does>not allow civil servants to operate foreign account,> talk less of owing one. Presently the m! oney is>safely> kept in an escrow account secure from all eyes. We> shall be meeting with you if you accept to assist>us,> secondly it would also afford us to know what>percentage>that will be given to you,for your assistance. It>will> take seven working daysto actualize thisproject and> safely transfer into your bank account on our behalf>and> yourself. I will appreciate your quick response>through my email I amopen to further question you may wish to ask in respect of this business transaction. Sincerely yours, Uman Garuba

1 0

<no subject>
by florian.konnertz＠web.de 05 Apr '02

05 Apr '02

Hi to you ! I installed a new product and its first ZClass (the example in the book from beehive). When I wanted to add my first 'buch' object, I got the following strange error: Let's see, if you have an idea!? cu, Florian Traceback (innermost last): File /usr/local/dc/Zope-2.5.0/lib/python/ZPublisher/Publish.py, line 150, in publish_module File /usr/local/dc/Zope-2.5.0/lib/python/ZPublisher/Publish.py, line 114, in publish File /usr/local/dc/Zope-2.5.0/lib/python/Zope/__init__.py, line 158, in zpublisher_exception_hook (Object: BuchClass_factory) File /usr/local/dc/Zope-2.5.0/lib/python/ZPublisher/Publish.py, line 98, in publish File /usr/local/dc/Zope-2.5.0/lib/python/ZPublisher/mapply.py, line 88, in mapply (Object: BuchClass_add) File /usr/local/dc/Zope-2.5.0/lib/python/ZPublisher/Publish.py, line 39, in call_object (Object: BuchClass_add) File /usr/local/dc/Zope-2.5.0/lib/python/OFS/DTMLMethod.py, line 127, in __call__ (Object: BuchClass_add) File /usr/local/dc/Zope-2.5.0/lib/python/DocumentTemplate/DT_String.py, line 473, in __call__ (Object: BuchClass_add) File /usr/local/dc/Zope-2.5.0/lib/python/DocumentTemplate/DT_With.py, line 61, in render (Object: BuchClass.createInObjectManager(REQUEST['ArtNr'], REQUEST)) File /usr/local/dc/Zope-2.5.0/lib/python/DocumentTemplate/DT_Util.py, line 159, in eval (Object: BuchClass.createInObjectManager(REQUEST['ArtNr'], REQUEST)) (Info: REQUEST) File <string>, line 2, in f NameError: global name 'BuchClass' is not defined (-- it is!!) cu Florian ______________________________________________________________________________ Für alle, die nicht genug WEB.DE bekommen können. 100 MB Speicher, 01212-Nummer und mehr E-Mail unter http://club.web.de/?mc=021101

1 0

property to method and back again
by Mark James Adams 05 Apr '02

05 Apr '02

Hello: I seem to have broken a ZClass that I'm developing. The ZClass represents a foreign language video of people speaking different words that sound similar. The instructor wanted a way to have the phrase associated with the video (which is also the video's title) come up with colored markings for the part of the phrase being emphasized in the video. For example, one video/phrase is 'vliezig-vlezig', where the 'ie' in the first word and the 'e' in the second word are highlighted. I wrote some code that, given a marked-up phrase like 'vl$ie$ig-vl$e$ig', return a HTML/CSS marked-up version, color-coding the letters between the '$'-signs. However, I still wanted to title to come up without any kind of markup, so I moved the marked-up title (with the dollar signs) to a new property, and deleted the title property. I then created a Python method called 'title' in the ZClass, which would return a non-marked up title. But now I get "KeyError: title" everytime a try to view the methods in the ZClass. I wrote a script to examine the ZClass, and placed it in the ZClass's product's folder: <dtml-in "MyZClass.propertysheets.methods.objectIds()"> <dtml-var sequence-item> </dtml-in> This script returns the following: method1 method2 etc.. title So the 'title' python script is obviously in the ZClass, but if I try to access: /Control_Panel/Products/MyProduct/MyZClass/propertysheets/methods/title/manage_main I get, "Site Error: Resource not found". Any ideas on what the problem is, and how I can correct it? If worse comes to worse, I'm able to directly access all of the methods in my ZClass (except for title), so I can always delete the ZClass and then recreate it. Thanks. -- Mark James Adams mark(a)raysend.com

2 1