March 2000 - Zope - Zope lists

Performance Testing / Expections
by Ben Leslie 27 Mar '00

27 Mar '00

Ok, I know it has been asked a couple of times before on the list, but there hasn't really been any absolute conclusions on the type of performance to expect from zope and/or how to tweak it for the best kind of performance. Over the last two days I have been doing some testing and would like to hear on opinions if these results are similar to other peoples. Firstly the machine specs are: PIII 450Mhz 128Mb IDE Harddrive Running Debian Potato Zope is running through Apache with PCGI. All tests were run using ab. First I tested apache to see how fast it was as a benchmark. 1000 requests, 25 Concurrent. /usr/sbin/ab -c 25 -n 1000 "http://www.eminecorp.com.au/docs/testemc.html" Results: 384 requests per seconds (rps). Avg wait: 54ms. Max Wait. 83ms. Throughput: 20MB/s 1000 requests, 1 at a time. /usr/sbin/ab -c 1 -n 1000 "http://www.eminecorp.com.au/docs/testemc.html" 425 rps AvgWait: 2ms MaxWait: 3ms Throughput: 21MB/s The I tested the same document servered statically through zope. (From zope's root) /usr/sbin/ab -c 25 -n 100 "http://www.eminecorp.com.au/testemc" 42 rps Avg Wait: 514ms MaxWait: 743ms Throughput: 2MB/s /usr/sbin/ab -c 1 -n 100 "http://www.eminecorp.com.au/testemc" 45 rps AvgWait: 21ms MaxWait: 29ms Throughput 2.3MB/s Next on line was a moderately dynamic page. Dtml only, no DB or Ext. Methods. /usr/sbin/ab -c 25 -n 100 "http://www.eminecorp.com.au/eminecorp/ecom2/sesgroup?cu rrentid=0&cur_page=http%3a//www.eminecorp.com.au/eminecorp/ecom2/siteinfo/index_html?" 4.97 rps AvWait: 4.4s MaxWait: 5.6s Throughput 21kb/s /usr/sbin/ab -c 1 -n 100 "http://www.eminecorp.com.au/eminecorp/ ecom2/sesgroup?currentid=0&cur_page=http%3a//www.eminecorp.com.au/eminecorp/ecom2/ siteinfo/index_html?" 5.09 rps AvWait: 195ms MaxWait: 198ms Throughput 22kb/s Then finally testing a page with the same amount of DTML complexity but some external methods which call a database using pygresql /usr/sbin/ab -c 25 -n 100 "http://www.eminecorp.com.au/eminecorp/ecom2/products/?s tartitem=1&currentid=22" 1.3rps Avwait 16s MaxWait 20s Throughput 19.51 kb/s /usr/sbin/ab -c 1 -n 100 "http://www.eminecorp.com.au/eminecorp/ecom2/products/?s tartitem=1&currentid=22" 1.3rps Avwait 768s MacWait 789 Throughput 19.55 kb/s In summary serving straight of apache is very fast, and doesn't suffer if multiple people access the site at once. Static Zope (which you wouldn't really do anyway) was acceptable speed and didn't really suffer too badly if there were multiple accesses. Dynamic Zope was fair in speed, but proably still fast enough, at least for my site anyway. It started to suffer in a bad way with multiple accesses, up to 5 sec response time. Highly dynamic Zope was very bad in rps and hopeless more multiple access. Now I have only been in this game I a while so I don't know how critical the multiple access thing is going to be. I am expecting at most 1000 visitors each day (say a 10 hour period). So 100 visitor each hour shouldn't cuase me any problems at the moment. I guess how I would like to know if anyone out there knows how to improve this performance at all. Can I throw memory and processors at it and make it go away? Obviously there are some things in the highly dynamic bit which I will need to speed up, however how can I make just normal dtml execute faster? The prospect of ZEO has certainly helped in this area but surely I must be able to sqeeze some more performance out of this machine. Thanks in advance. Benno

6 11

Re: Indirect calling of extensions
by Glenn Rogers 27 Mar '00

27 Mar '00

I wrote: > > I want to call this in a manner similar to: > > > >  > >  > >  > >  > >  Tres Seaver wrote: > Try this (untested), assuming the name of the method you want to call > is stored somehow in the variable 'methodName':: > > <dtml-let methodToCall="_[ methodName ]"> > <dtml-in list_hrefs mapping> > <dtml-var "methodToCall( this(), href )"> > </dtml-in> > </dtml-let> > Alas! It doesn't work. It appears to fall down on the line: <dtml-let methodToCall="_[ methodName ]"> Traceback: File C:\PROGRA~1\Zope\lib\python\DocumentTemplate\DT_Let.py, line 144, in render (Object: methodToCall="_[select_method]") File C:\PROGRA~1\Zope\lib\python\DocumentTemplate\DT_Util.py, line 335, in eval (Object: _[select_method]) (Info: select_method) File <string>, line 0, in ? File C:\PROGRA~1\Zope\lib\python\DocumentTemplate\DT_Util.py, line 161, in careful_getitem File [snip]\Products\ExternalMethod\ExternalMethod.py, line 248, in __call__ (Object: pyFetchInvTopicsForHref) (Info: ((), {}, None)) TypeError: (see above) [above= Error Type: TypeError Error Value: not enough arguments; expected 2, got 0 ] It seems evaluating this doesn't just give the string contents of the dictionary item, it evaluates the actual external method call. I think I'll have to give up at this point and take the cludgy option (unless anyone can think up another way). Thanks for your help. Glenn

1 0

zope search problems w/squishdot with latin1, locale fi_FI
by Jyrki Kuoppala 27 Mar '00

27 Mar '00

I'm using Zope 2.1.4 on Linux with environment LC_ALL set to fi_FI and Zope started with -L fi_FI, and ZCatalog indexing & searching seems to work fine. (well, there's a problem with some documents having ä etc. spelled out, but if ä is used everything works OK.) BUT, ZDConfera & Squishdot don't work right. For some reason, at some phase of the indexing characters like ö (ö, octal 366) in words are converted to their octal codes. This can be seen by looking at the raw Data.fs file. The result is that when I have a word like kökköviesti in the ZDConfera or Squishdot article, it can be found by search as 366viesti It is converted in the db storing/indexing phase to something like k\366kk\366viesti, which is then broken into words k, 366kk, 366viesti which are stored into the textindex. Any ideas on how to fix this? I've looked at Splitter.c, TextIndex.py, ZDiscussions.py and Squishdot.py, but I can't find anything which would convert characters to their octal codes. //Jyrki

1 0

ZCatalog specificity
by Oleg Broytmann 27 Mar '00

27 Mar '00

---------- Forwarded message ---------- Date: Mon, 27 Mar 2000 12:12:06 +0400 From: bsv <bsv(a)ramax.spb.ru> To: Oleg Broytmann <phd(a)sun.med.ru> Subject: ZCatalog specificity By searching of the documents using ZCatalog, in query I list words with "OR", for example, "web or pop or mail" - all works well. But if in a query with "OR" there is a word that is not in the index, ZCatalog finds nothing, for example, "web or pop or mailvvv", but the documents with "web" and "pop" exist. On www.zope.org the same effect. Is it possible to correct this situation?

1 0

Sorting descending/ascending
by Ken Kinder 27 Mar '00

27 Mar '00

In a dtml-in, how do I explicitly sort ascendingly or descendingly?

4 3

Passing data to/from Python
by Mark Wexler 27 Mar '00

27 Mar '00

Dear Zopistas, I am in the process of learning about Zope, and trying to determine whether to make the switch from pure Python CGI scripts. If someone could answer the following questions (and I apologize, as it will probably seem naive), it would really help. Let's suppose I have a page where the user queries a data base, for instance a telephone directory. The telephone directory is in an Excel file. At present, I have a CGI script that opens an ODBC.Windows connection, executes an SQL query, and outputs the results (names, phone numbers) as an HTML table using "print". What is the best way to do this in Zope? Here are the points that I don't get: 1. It seems that the only way to establish a DB connection is through Gadfly. There are warnings everywhere that Gadfly is inefficient. Therefore, I want to use ODBC (ODBC.Windows), like I did before. How? 2. I might eventually need to do complex calculations on the retrieved data, so I would prefer using Python code in an external method, rather than a DTML method. In order to get Zope to accept the line "import ODBC.Windows", I had to copy the ODBC and DateTime directories from my regular Python installation (PYTHON/lib) to ZOPE/bin/lib. But then, when I try running the script, I get one of the following problems: a. If I go through authentification, the script runs, but I cannot establish the connection to the database. b. If I type in the URL of my search page directly, without authentification, I get a Zope error: "initialization of module mxODBC failed (exceptions.AttributeError:mxDateTimeAPI)". Is my ODBC.Windows installation wrong, or do I have a more fundamental problem? 3. What if my Python function in my external method wants to return information to my DTML method? For instance, how do I return a Python list, and what do I do in my DTML method to process it (e.g., output a line for every member of the list)? 4. The way I call my Python function now is through the <form method="POST" action="search"> tag, where "search" is the external method ID. The form has a field called "name", and my Python function has an argument also called "name", and so it recuperates the string that was entered in the "name" field. Can I pass the "name" as an argument to a DTML method? If these questions have answers, I would really appreciate them. Otherwise, if I'm completely off-base (for instance, if I'm thinking in Python terms and not in Zope terms), please point me in the right direction! Thanks in advance. Mark Wexler

7 7

Midgard vs Zope (Was RE: [Zope] PHP vs Zope)
by Alessandro Bottoni 27 Mar '00

27 Mar '00

Interesting topic! How does Zope compare with Midgard? Which one is more flexible (that is: which one can be used in a wider set of cases)? Which one is easier to learn for the webmaster? Which one is better fit to develop and maintain a complex web site (let's say: 10,000 documents and 50,000 pictures, updated weekly)? Which one is better for web sites that uses database and/or a "fragmented document" architecure (XML/SGML documents that are disassembled and re-assembled on demand)? Which one can better integrate other technologies (web-mail interfaces and so on)? Which one is easier to use for a remote customer, enabled to develop and maintain his pages on the server? Which one is better for "workgroup" intranet servers? Which one is safer (more anti-hacker)? Which features does Zope have that Midgard does not have? Thanks. PS What does "Midgard" mean (I'm italian...)? ---------------------------------------------------------------- Alessandro Bottoni (Alessandro.Bottoni(a)Think3.com) Web Programmer @ Think3 inc. (www.think3.com) I do not speak for think3 and they return the favour > -----Original Message----- > From: Eric W. Sink [SMTP:eric@sourcegear.com] > Sent: Wednesday, March 22, 2000 6:31 AM > To: Nikolai Vladychevski > Cc: zope(a)zope.org > Subject: Re: [Zope] PHP vs Zope > ---- omissis ----- > Compare PHP to Python > > Compare Zope to Midgard. > >

5 5

Problems with adding ZClass instances
by John Morton 27 Mar '00

27 Mar '00

I'm have problems with a ZClass I'm working on that uses Folder as it's base class and is supposed to install a set of default methods when it's created. The trouble is that it won't when the webDirectory is created, but it will when I invoke the same method from a tab afterwards. Here's the body of webDirectory_add: <dtml-with "webDirectory.createInObjectManager(REQUEST['id'], REQUEST)"> <dtml-if createDefaultMethods> <dtml-in "default_methods.objectIds()"> <dtml-try> <dtml-call "manage_delObjects(ids=[_['sequence-item']])"> <dtml-except> </dtml-try> <dtml-call "manage_clone( _.getattr(default_methods, _['sequence-item']), _['sequence-item'], REQUEST)"> </dtml-in> </dtml-if> </dtml-with> I'm assuming that the code within the <dtml-with ...> is effectively opperating in the context of the newly created object, but when the first manage_clone() is executed, it bombs out in _getCopy, tripping over on this line: ob=container._p_jar.importFile(f) ...because whatever container is, it's not a folderish object. So what do I need to do to switch to the new webDirectory object at creation time, so adding methods is automatic? Thanks, John.

1 0

Re: [Zope] Passing data to/from Python
by Curtis Maloney 27 Mar '00

27 Mar '00

On Mon, 27 Mar 2000, Terry Kerr wrote: > Curtis Maloney wrote: > > > 1. It seems that the only way to establish a DB connection is > > > through Gadfly. There are warnings everywhere that Gadfly is > > > inefficient. Therefore, I want to use ODBC (ODBC.Windows), like I > > > did before. How? > > > > > Nonono.... ZOPE uses the Gadfly db to store "zope objects". It can also > > connect to other DBs, and access them via SQL methods. There are a number of > > DB interfaces, including an ODBC interface. > > Umm...no...ZOPE used the ZODB to store objects...gadfly is another SQL written > python that is just bundled with zope since it is pretty cool. > Ok.. my mistake.. sorry.... thanks for correcting me. I was under the impression ZOPE stored data in a db, using gadfly as the default, and it was refered to as ZODB for convenience. Thanks for clearning that up for me. -- Have a better one, Curtis. <dtml-var standard_work_disclaimer>

1 0

Re: [Zope] Performance Testing / Expections
by J. Atwood 27 Mar '00

27 Mar '00

Here are my tests... Machine: 300 MHz 256 RAM IDE Hardrive Apache on same machine (different port) Document Length: 590 bytes Concurrency Level: 25 Time taken for tests: 0.165 seconds Complete requests: 100 Failed requests: 0 Total transferred: 92340 bytes HTML transferred: 63720 bytes Requests per second: 606.06 Transfer rate: 559.64 kb/s received Connnection Times (ms) min avg max Connect: 0 6 14 Processing: 22 25 34 Total: 22 31 48 Zope (ZServer Only, test document, just a few html bits and <dtml-var standard_html_footer>) Document Length: 682 bytes Concurrency Level: 25 Time taken for tests: 2.451 seconds Complete requests: 100 Failed requests: 0 Total transferred: 88500 bytes HTML transferred: 68200 bytes Requests per second: 40.80 Transfer rate: 36.11 kb/s received Connnection Times (ms) min avg max Connect: 0 66 333 Processing: 339 467 342 Total: 339 533 675 I upped it to 1000 and got... Concurrency Level: 25 Time taken for tests: 23.074 seconds Complete requests: 1000 Failed requests: 0 Total transferred: 885885 bytes HTML transferred: 682682 bytes Requests per second: 43.34 Transfer rate: 38.39 kb/s received I put it against a normal page with a lot dtml and a few highly cached sql calls (in DTML's) Document Length: 17000 bytes Concurrency Level: 25 Time taken for tests: 13.314 seconds Complete requests: 100 Failed requests: 0 Total transferred: 1727400 bytes HTML transferred: 1700000 bytes Requests per second: 7.51 Transfer rate: 129.74 kb/s received Connnection Times (ms) min avg max Connect: 0 82 379 Processing: 386 2898 4312 Total: 386 2980 4691 When I turned it on a highly dynamic page (lots of SQL, lots of DTML) It got worse.. Server Hostname: fundraising.gotschool.com Server Port: 80 Document Path: /search?category=11 Document Length: 22373 bytes Concurrency Level: 1 Time taken for tests: 25.820 seconds Complete requests: 100 Failed requests: 73 (Connect: 0, Length: 73, Exceptions: 0) Total transferred: 2263791 bytes HTML transferred: 2236391 bytes Requests per second: 3.87 Transfer rate: 87.68 kb/s received Connnection Times (ms) min avg max Connect: 0 0 2 Processing: 197 257 315 Total: 197 257 317 I upped that cache on the SQL and that id not really help... I then put the test against just one of the parts of the page (that is DTML, calls another DTML and has a SQL Select in it) Concurrency Level: 1 Time taken for tests: 19.768 seconds Complete requests: 100 Failed requests: 0 Total transferred: 987100 bytes HTML transferred: 966700 bytes Requests per second: 5.06 Transfer rate: 49.93 kb/s received Connnection Times (ms) min avg max Connect: 0 0 2 Processing: 151 197 211 Total: 151 197 213 Which was not bad... I upped to a 25 Concurrency... Document Length: 9667 bytes Concurrency Level: 25 Time taken for tests: 15.912 seconds Complete requests: 100 Failed requests: 81 (Connect: 0, Length: 81, Exceptions: 0) Total transferred: 691350 bytes HTML transferred: 670542 bytes Requests per second: 6.28 Transfer rate: 43.45 kb/s received Connnection Times (ms) min avg max Connect: 0 158 718 Processing: 727 3342 4341 Total: 727 3500 5059 Well, that is food for thought today. If anyone has some really good analysis of this (since this is my first time using AB) that would be great. JMA

1 0