Hi there, I thought it'd be interesting to note that PyPy 1.4.1 (with JIT) now works out of the box with buildout (at least if I use the '-d' option for distribute with bootstrap.py, but I suspect plain setuptools also works). I actually added a few compatibility fixes to buildout the other day that are now unnecessary: PyPy 1.4.1 adds the compatibility with released versions of buildout out of the box. This makes it possible to start testing some of the ZTK with PyPy. There are challenges of course: certain packages, such as zope.interface, use C extensions and would need to be installed in "plain-python" mode (if available). The ZODB would definitely be an interesting challenge too. One snag is that PyPy is only compatible up to Python 2.5. This might make some of the code fail. Then again, if it's the 'with' statement the fix is an "from __future__ import with_statement" away. A release of PyPy with Python 2.7 support is still a while away, but being worked on. The benefit of using PyPy? It has a JIT on board, so that should speed up some code significantly. Whether it'll speed up some of our tests is hard to say though - the JIT might be more overhead than gain in that case. Regards, Martijn
On Wed, Dec 22, 2010 at 9:24 AM, Martijn Faassen <faassen@startifact.com> wrote:
Hi there,
I thought it'd be interesting to note that PyPy 1.4.1 (with JIT) now works out of the box with buildout (at least if I use the '-d' option for distribute with bootstrap.py, but I suspect plain setuptools also works). I actually added a few compatibility fixes to buildout the other day that are now unnecessary: PyPy 1.4.1 adds the compatibility with released versions of buildout out of the box.
Cool.
This makes it possible to start testing some of the ZTK with PyPy. There are challenges of course: certain packages, such as zope.interface, use C extensions and would need to be installed in "plain-python" mode (if available).
It is available. The setup script could, presumably, detect if it is running under PyPy and not even try building extensions.
The ZODB would definitely be an interesting challenge too.
This is something I want. The hardest part of this is coming up with a Python BTree implementation.
One snag is that PyPy is only compatible up to Python 2.5. This might make some of the code fail. Then again, if it's the 'with' statement the fix is an "from __future__ import with_statement" away.
A release of PyPy with Python 2.7 support is still a while away, but being worked on.
The benefit of using PyPy? It has a JIT on board, so that should speed up some code significantly.
It would be cool to see some ZTK-related benchmarks.
Whether it'll speed up some of our tests is hard to say though - the JIT might be more overhead than gain in that case.
I haven't found tests to be good measures of performance improvements. Thanks for pointing this out. Jim -- Jim Fulton
On Wed, 2010-12-22 at 09:43 -0500, Jim Fulton wrote:
This makes it possible to start testing some of the ZTK with PyPy. There are challenges of course: certain packages, such as zope.interface, use C extensions and would need to be installed in "plain-python" mode (if available).
It is available. The setup script could, presumably, detect if it is running under PyPy and not even try building extensions.
It already does. zope.interface falls back to using its Python implementation if it's installed on a platform that doesn't support C compilation. As does zope.i18nmessageid.
The ZODB would definitely be an interesting challenge too.
This is something I want. The hardest part of this is coming up with a Python BTree implementation.
I'd be willing to work on this at a sprint. FWIW, for our web stuff I've found PyPy to be about 3X as slow as CPython (even allowing for the JIT to warm up). But the elephant dances! - C
On Wed, Dec 22, 2010 at 03:56:11PM -0500, Chris McDonough wrote:
On Wed, 2010-12-22 at 09:43 -0500, Jim Fulton wrote:
This makes it possible to start testing some of the ZTK with PyPy. There are challenges of course: certain packages, such as zope.interface, use C extensions and would need to be installed in "plain-python" mode (if available).
It is available. The setup script could, presumably, detect if it is running under PyPy and not even try building extensions.
It already does. zope.interface falls back to using its Python implementation if it's installed on a platform that doesn't support C compilation. As does zope.i18nmessageid.
Incidentally, PyPy has some source-level support for C extensions: http://morepypy.blogspot.com/2010/04/using-cpython-extension-modules-with.ht... Marius Gedminas -- http://pov.lt/ -- Zope 3/BlueBream consulting and development
Hi there, On 12/22/2010 09:56 PM, Chris McDonough wrote:
FWIW, for our web stuff I've found PyPy to be about 3X as slow as CPython (even allowing for the JIT to warm up). But the elephant dances!
Any clue what's taking up the time? Do you think it's simply that it doesn't speed up zope.interface to the level of the C version? I vaguely recall something about PyPy's JIT not being good with interpreter-style patterns in code - so that might slow down things if templating is involved. Regards, Martijn
I vaguely recall something about PyPy's JIT not being good with interpreter-style patterns in code - so that might slow down things if templating is involved.
That would suggest the precompiling chameleon templates to .pt.py would give a better performance for templates ;-)
Regards,
Martijn
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
participants (6)
-
Chris McDonough -
Jim Fulton -
Lennart Regebro -
Marius Gedminas -
Martijn Faassen -
Tim Hoffman