What can I do to maximize the peformance of a dtml-in statement? When iterating over a 2000 row database query, it takes about 20 seconds - time mostly spent doing security checks, and calling __getitem__ in DT_InSV.py. Is the performance of dtml-in just slow? -Brett
Yes, but iterating over a list of 2000 rows is a long process. Things like using mapping slow it down even further as you get each column. Less rows will make all the difference. ----- Original Message ----- From: "Brett Carter" <brett@kavi.com> To: <zope-dev@zope.org> Sent: Wednesday, April 04, 2001 5:54 PM Subject: [Zope-dev] dtml-in performance
What can I do to maximize the peformance of a dtml-in statement? When iterating over a 2000 row database query, it takes about 20 seconds - time mostly spent doing security checks, and calling __getitem__ in DT_InSV.py. Is the performance of dtml-in just slow? -Brett
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
call the daatabase query in an external method and convert the result to a simple object (like a dictionary), then pass that back and iterate over it jens on 4/4/01 20:54, Brett Carter at brett@kavi.com wrote:
What can I do to maximize the peformance of a dtml-in statement? When iterating over a 2000 row database query, it takes about 20 seconds - time mostly spent doing security checks, and calling __getitem__ in DT_InSV.py. Is the performance of dtml-in just slow? -Brett
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Brett Carter wrote:
What can I do to maximize the peformance of a dtml-in statement? When iterating over a 2000 row database query, it takes about 20 seconds - time mostly spent doing security checks, and calling __getitem__ in DT_InSV.py. Is the performance of dtml-in just slow? -Brett
Another performance consideration with looping in DTML is that whatever is in your DTML block is reinterpreted each time through. Although it is parsed (hopefully) only the first time through, this overhead is still considerable when multiplied over 2000 rows. You are essentially using an interpreter (Python) as an interpreter for DTML. I would seriously consider moving this entire operation (query, iteration and html generation) to native Python if performance is a big consideration. -- | Casey Duncan | Kaivo, Inc. | cduncan@kaivo.com `------------------>
What can I do to maximize the peformance of a dtml-in statement? When iterating over a 2000 row database query, it takes about 20 seconds - time mostly spent doing security checks, and calling __getitem__ in DT_InSV.py. Is the performance of dtml-in just slow? -Brett
Another performance consideration with looping in DTML is that whatever is in your DTML block is reinterpreted each time through. Although it is parsed (hopefully) only the first time through, this overhead is still considerable when multiplied over 2000 rows. You are essentially using an interpreter (Python) as an interpreter for DTML. I would seriously consider moving this entire operation (query, iteration and html generation) to native Python if performance is a big consideration.
We have had similar performance problems with the first version of our Content Management demo. It used both dtml-in loops and ZClasses (with other ZCLasses as base classes) heavily. The combination of the both can be very efficient in slowing down Zope. I remember the first versions of Martijn Faassen's XML-Widgets that had a similar performance problem. The reason is that in addition to the dtml-in overhead ZClasses take a lot of time for acquiring from their containers and inheriting from their "base classes" as all the inheritance stuff is only simulated - in fact almost everything is just stored in property sheets and has to be processed by the security engine if I got it right. In Python these problems go away. Maybe it is time for a good reimplementation of ZClasses, inlcuding the creation of REAL Python classes in the filesystem on the fly. BTW: Does anybody know if the Zope Page Templates (ZPT) parser is faster with looping than the DTML one? Cheers, Joachim.
What can I do to maximize the peformance of a dtml-in statement? When iterating over a 2000 row database query, it takes about 20 seconds - time mostly spent doing security checks, and calling __getitem__ in DT_InSV.py. Is the performance of dtml-in just slow? -Brett
Another performance consideration with looping in DTML is that whatever is in your DTML block is reinterpreted each time through. Although it is parsed (hopefully) only the first time through, this overhead is still considerable when multiplied over 2000 rows. You are essentially using an interpreter (Python) as an interpreter for DTML. I would seriously consider moving this entire operation (query, iteration and html generation) to native Python if performance is a big consideration.
We have had similar performance problems with the first version of our Content Management demo. It used both dtml-in loops and ZClasses (with other ZCLasses as base classes) heavily. The combination of the both can be very efficient in slowing down Zope. I remember the first versions of Martijn Faassen's XML-Widgets that had a similar performance problem. The reason is that in addition to the dtml-in overhead ZClasses take a lot of time for acquiring from their containers and inheriting from their "base classes" as all the inheritance stuff is only simulated - in fact almost everything is just stored in property sheets and has to be processed by the security engine if I got it right. In Python these problems go away. Maybe it is time for a good reimplementation of ZClasses, inlcuding the creation of REAL Python classes in the filesystem on the fly. BTW: Does anybody know if the Zope Page Templates (ZPT) parser is faster with looping than the DTML one? Cheers, Joachim.
participants (5)
-
Andy -
Brett Carter -
Casey Duncan -
Jens Vagelpohl -
Joachim Werner