[Zope-dev] Zope server speed

[Jens Vagelpohl]

what i have done before to get around the speed penalty is to create an 
external method. the external method calls the ZSQL method directly and 
iterates over the result objects to pull out all interesting attributes 
and stick them into simple python objects, in this case i returned a list 
of dictionaries from the external method. in DTML the iteration over it 
still looks the same but it was much faster because there is no security 
enforced on the simple python objects returned from the external method.

jens



On Thursday, December 6, 2001, at 09:33 , MANOTTI Alessandro wrote:

> Thank you for your suggestions (thanks to Matt).
>
> I will try to make smaller but more requests (1000 requests of 1000
> records).
>
> I will even try to "bypass" Zope and write an "external method" in Python 
> (I
> know Pytho very well).
> About this solution: can you tell me what I have to do to use ZOracle
> directly in Python (import ... ? ) ?
>
> Thanks to everybody!
>
>
>
>
> -----Original Message-----
> From: Matthew T. Kromer [mailto:matt@zope.com]
> Sent: Thursday, December 06, 2001 15:27
> To: MANOTTI Alessandro; zope-dev@zope.org
> Subject: Re: [Zope-dev] Zope server speed
>
>
> Hi Alessandro,
>
> Zope and Python combined aren't too bad if you write some wrappers that 
> help
> you manipulate data; the problem comes about when you try to do large 
> naive
> queries -- there is a LOT of overhead in taking Oracle data, encapsulating
> it in Python objects, and then wrapping that in the Zope machinery.
>
> I have a sample database in Oracle of US Zip codes -- it has about 45,000
> records in it.  I can pull this into DCOracle2 on the C adapter side
> (dco2.so) in less than a second.  However, wrapping each result cell in a
> python object blows that up into about 9 seconds.  If I were to iterate 
> over
> the whole thing in Zope (which I haven't timed) it would get a LOT longer,
> because Zope is enforcing policy on every object accessed.
>
> So, one of the ways to make that go faster is to use python modules
> 'outside' of Zope (ie external methods) to perform large calculations 
> which
> takes a ZOracleDA connection object to get a cursor object on the 
> database,
> does the query directly, and formats and returns the results.
>
> I realize this isn't ideal, but Zope is not optimized for the case of
> "extract thousands of records from the database and format them in a huge
> table."  Zope likes "extract hundreds of records from the database and
> format them in a big table" instead.
>
> With each release, Zope has more features that enable you to program 
> through
> the web, but when you're doing performance intensive calculations, it's
> usually time to write a python module and plug it in to zope (via an
> external method, or by writing a product).  If you have the time to learn,
>  I
> suggest the product route as being more satisfying, but it can be more
> frustrating as well.
>
>
> ----- Original Message -----
> From: "MANOTTI Alessandro" <alessandro.manotti@bfeurope.com>
> To: <zope-dev@zope.org>
> Sent: Thursday, December 06, 2001 8:52 AM
> Subject: [Zope-dev] Zope server speed
>
>
>>
>>> HI all,
>>>
>>> I have a problem:
>>>
>>> my company is looking for an application server. I know Zope (I
> presented
>>> it), and a friend of mine presented Macromedia ColdFusion.
>>>
>>> We made some tests about the speed of the systems, arghh!!! Zope is VERY
>>> slow compared to ColdFusion!
>>>
>>> Some examples:
>>>
>>> I used Zope with DCOracle2 drivers to connect and extract 257000 (yes!
> it
>>> is a test... two hundred fiftyseven thousand records!) and Zope took
> more
>>> than 10 minutes, since I stopped it after that time (call SQL and print
>>> data via <dtml-var ... > TAG).
>>>
>>> The same PC, the same server, ColdFusion took about 2 minutes to
> complete
>>> the operation!!!
>>>
>>> Coldfusion is more than 5 times faster than Zope!!!
>>>
>>> PHASE 2:
>>>
>>> I said: ok, maybe Zope is slower that Coldfusion in TAG formatting...
> and
>>> I tried to get the records and perform a calculation, without no
> output...
>>>
>>> Sigh! the results are the same!
>>>
>>> I am depressed, since I talk about Zope very well in my company, but
>>> now...
>>>
>>> Coldfusion and Zope are similar, so we shall select ColdFusion to
> develop
>>> web applications.
>>>
>>> SIGH !!!
>>>
>>> Can I try something to increase Zope power?!
>>>
>>>
>>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>>>
>>> Alessandro Manotti
>>> Presidente dell'Associazione "RIUSA"
>>>
>>> Sito web:     http://riusa.apritisesamo.net
>>> email:        riusa@email.it
>>> mailing-list: riusa-ml@yahoogroups.com
>>>
>>>
>>
>> _______________________________________________
>> Zope-Dev maillist  -  Zope-Dev@zope.org
>> http://lists.zope.org/mailman/listinfo/zope-dev
>> **  No cross posts or HTML encoding!  **
>> (Related lists -
>>  http://lists.zope.org/mailman/listinfo/zope-announce
>>  http://lists.zope.org/mailman/listinfo/zope )
>>
>
> _______________________________________________
> Zope-Dev maillist  -  Zope-Dev@zope.org
> http://lists.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists -
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope )