Zope 2.12 20% slower than Zope 2.9?!
Hi All, Just finished upgrading a medium sized project from Zope 2.9 to 2.12. One part of that does a batch of static copies of a Zope-served site. I guess I was hoping for a bit of a speed up, so was slightly horrified to see the batch, which used to take 11hrs, now takes 13 hrs. This has been the same for 2 or 3 days since I did the upgrade so I don't think it's a freak occurrence. The site is classic Zope 2: zodb-based python scripts and page templates building the UI from Product-based content objects. Have others shared this experience? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
Am 11.10.2010, 14:16 Uhr, schrieb Chris Withers <chris@simplistix.co.uk>:
The site is classic Zope 2: zodb-based python scripts and page templates building the UI from Product-based content objects. Have others shared this experience?
Hiya Chris, I haven't run any performance tests but my Zope 2.12 feels quite a lot faster to me than pre-buildout versions. I thought there were some comparative figures upon release? Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 Mobile: +49-178-782-6226
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Chris Withers wrote:
Hi All,
Just finished upgrading a medium sized project from Zope 2.9 to 2.12. One part of that does a batch of static copies of a Zope-served site.
I guess I was hoping for a bit of a speed up, so was slightly horrified to see the batch, which used to take 11hrs, now takes 13 hrs. This has been the same for 2 or 3 days since I did the upgrade so I don't think it's a freak occurrence.
The site is classic Zope 2: zodb-based python scripts and page templates building the UI from Product-based content objects.
Try with a smaller "batch" (whatever you are doing) and profile it. - -aj -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQGUBAEBAgAGBQJMswH2AAoJEADcfz7u4AZjlKgLv1kY2RbJaiZjLcYsgf21E0qS AymUcPPKYyERWbq3akuXI5bL3LA5saWFRkF2NWeg3D1qmq6iSOrTu6fkGY355KsF T2tnPJSyultkLThAMBQQm1qSdp+Em7aojJA94hwzGhU39EJCaKX4IuUpKKMNKhFV LOfqkwlpAT5XxWERjBrIUWWHR5Yn22JxodkMo1A+gTCnrwaUafTDOM3fEOsZVKRB zD92cgZhvKThiRB14ZiqYWTlrgMKgHeD53dQK/kUTU+alIcblTYDfi8CLllxDasC 2ovg4kQRk/u3AYoyjFfaqWdm/PbOkYea+3w7V2Y+ed44jRM4b4r52PjuFnH6A5qV HsTnTrhJ7vQTzR7zBzgZF1s4wto7ZogpmfV9I3k+bPxwvcsPKCkJ+yrX+9LKxlcw A1dddG/+1B2za7ypxX4Jqy4sXQjEyQDlRYf/00ESe4l4NobdR4IAiKsT+DOF3COi ZwIwQ+Z/fAvauNIKAkHqNXLNmhqiwi4= =rfpF -----END PGP SIGNATURE-----
Try with a smaller "batch" (whatever you are doing) and profile it.
I 100% agree with what ajung says. post a profile of your batch run that takes ~30 minutes. I would suspect its probably something inside framework you are using which is constantly querying ZCA. -alan
On 11/10/2010 19:00, Alan Runyan wrote:
Try with a smaller "batch" (whatever you are doing) and profile it.
I 100% agree with what ajung says. post a profile of your batch run that takes ~30 minutes.
Related to this, what http load testing tools do people around here recommend? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Mon, Oct 11, 2010 at 2:16 PM, Chris Withers <chris@simplistix.co.uk> wrote:
Just finished upgrading a medium sized project from Zope 2.9 to 2.12. One part of that does a batch of static copies of a Zope-served site.
I guess I was hoping for a bit of a speed up, so was slightly horrified to see the batch, which used to take 11hrs, now takes 13 hrs. This has been the same for 2 or 3 days since I did the upgrade so I don't think it's a freak occurrence.
Without actual profiling data it's hard to tell what time has increased here. Most of the components in Zope have stayed the same. Some of them like the ZODB have improved performance. My only guess here is that you are calling a lot of page templates and those have gotten slower. When we switched to Unicode based templates in Zope 2.10, that move has of course cost some performance. Dealing with Unicode instead of bytes is always slower. If you want to improve rendering times and your analysis shows page templates to be the problem, try Chameleon. Hanno
On 11/10/2010 13:32, Hanno Schlichting wrote:
My only guess here is that you are calling a lot of page templates and those have gotten slower. When we switched to Unicode based templates in Zope 2.10, that move has of course cost some performance. Dealing with Unicode instead of bytes is always slower.
If you want to improve rendering times and your analysis shows page templates to be the problem, try Chameleon.
Yes, this is my guess too. Are there any docs on how to get Chameleon to work with classic, ZODB-based page templates? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On 10/11/10 5:39 AM, Chris Withers wrote:
On 11/10/2010 13:32, Hanno Schlichting wrote:
My only guess here is that you are calling a lot of page templates and those have gotten slower. When we switched to Unicode based templates in Zope 2.10, that move has of course cost some performance. Dealing with Unicode instead of bytes is always slower.
If you want to improve rendering times and your analysis shows page templates to be the problem, try Chameleon. Yes, this is my guess too.
Are there any docs on how to get Chameleon to work with classic, ZODB-based page templates? Just use cmf.pt: http://pypi.python.org/pypi/cmf.pt
---------- David Glick Web Developer davidglick@groundwire.org 206.286.1235x32 Groundwire: You Are Connected http://groundwire.org We're celebrating 15 years! Come to our big party. http://groundwire.org/events/groundwires-15th-anniversary-party
On 11/10/2010 16:53, David Glick wrote:
Are there any docs on how to get Chameleon to work with classic, ZODB-based page templates? Just use cmf.pt: http://pypi.python.org/pypi/cmf.pt
Ah, okay, well, I'm not using anything CMF, so I opted for just five.pt. Unfortunately, this hits a unicode problem. The unicode problem I solved with an IUnicodeEncodingConflictResolver using the normal page template engine: class Resolver: implements(IUnicodeEncodingConflictResolver) def resolve(self, context, text, expression): return unicode(text,'latin-1') chameleon doesn't appear to use this, so I get: File "five.pt-1.3.3-py2.6.egg/five/pt/patches.py", line 90, in call_template_and_wrap return template(self, *args, **kw) File "z3c.pt-1.2.1-py2.6.egg/z3c/pt/pagetemplate.py", line 145, in __call__ return bound_pt(*args, **kwargs) File "z3c.pt-1.2.1-py2.6.egg/z3c/pt/pagetemplate.py", line 262, in __call__ return self.im_func(**kw) File "z3c.pt-1.2.1-py2.6.egg/z3c/pt/pagetemplate.py", line 136, in render return self.render(**context) File "Chameleon-1.2.13-py2.6.egg/chameleon/zpt/template.py", line 17, in render return super(PageTemplate, self).render(*args, **kwargs) File "Chameleon-1.2.13-py2.6.egg/chameleon/core/template.py", line 221, in render return self.cook_and_render(kwargs, utils.emptydict, None, True) File "Chameleon-1.2.13-py2.6.egg/chameleon/core/template.py", line 205, in cook_and_render return func(econtext, rcontext) - __traceback_info__: (<BaseTemplate >,) File "<string>", line 79, in render UnicodeDecodeError: 'utf8' codec can't decode byte 0xa3 in position 0: unexpected code byte What's the chameleon or z3c.pt way of dealing with this*? Chris *'this' is stacks of ZODB-based page templates whose source may well be latin-1 encoded but, moreover, which are operating on a bunch of content that's *definitely* latin-1 encoded... -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Mon, Oct 11, 2010 at 10:54 PM, Chris Withers <chris@simplistix.co.uk> wrote:
What's the chameleon or z3c.pt way of dealing with this*?
*'this' is stacks of ZODB-based page templates whose source may well be latin-1 encoded but, moreover, which are operating on a bunch of content that's *definitely* latin-1 encoded...
Eeh, damn, haven't heard of latin-1 users in a decade. With latin-1 you probably won't be able to use Chameleon. It handles both Unicode and utf-8 encoded strings fine on its own. But that support is done deep down in the inner workings and not configurable. Your best bet now is to ignore Chameleon and put the "convert to proper Unicode" item up higher on the todo list. Hanno P.S. Of course it's all code, so you could subclass the right parts of the machinery and make this stuff configurable. For anyone who's not Malthe that will just be quite a bit of a pain and time consuming.
participants (6)
-
Alan Runyan -
Andreas Jung -
Charlie Clark -
Chris Withers -
David Glick -
Hanno Schlichting