-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm reconsidering the way we RPM package byte-code compiled python, and although I suspect optimised python is a bit of an anacronysm, I thought I'd check with the list ;) If one was to start Zope with python -O, then it would look for (and generate) .pyo files instead of .pyc's in all the Products right? Would there be any performance gain by this change? How much? Is it worth me doing byte-compiles with -O and starting Zope's interpreter in this mode? Alan -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFDpQWqCfroLk4EZpkRAlPzAJ90GRzBC7MxpvnkFBkNvcEuwn71LgCfUDB5 PgOhpCiGveqHdNQx/cABRrw= =8P0/ -----END PGP SIGNATURE-----
On 18 Dec 2005, at 06:46, Alan Milligan wrote:
I'm reconsidering the way we RPM package byte-code compiled python, and although I suspect optimised python is a bit of an anacronysm, I thought I'd check with the list ;)
If one was to start Zope with python -O, then it would look for (and generate) .pyo files instead of .pyc's in all the Products right?
Would there be any performance gain by this change? How much?
Is it worth me doing byte-compiles with -O and starting Zope's interpreter in this mode?
There is no anachronism at all. Building Python by hand will also build .pyo files alonside .pyc files. There is a speed increase running Zope with .pyos, but if I remember correctly you have to hack one of the startup files to correctly pass all python flags given upon startup. I don't know anyone who has done specific testing to quantify the speed increase, though. There is a drawback: Debugging becomes a lot harder and sometimes impossible because you get no or faulty line number information in tracebacks and when using pdb. jens
Jens Vagelpohl wrote:
On 18 Dec 2005, at 06:46, Alan Milligan wrote:
I'm reconsidering the way we RPM package byte-code compiled python, and although I suspect optimised python is a bit of an anacronysm, I thought I'd check with the list ;)
If one was to start Zope with python -O, then it would look for (and generate) .pyo files instead of .pyc's in all the Products right?
Would there be any performance gain by this change? How much?
Is it worth me doing byte-compiles with -O and starting Zope's interpreter in this mode?
There is no anachronism at all. Building Python by hand will also build .pyo files alonside .pyc files. There is a speed increase running Zope with .pyos, but if I remember correctly you have to hack one of the startup files to correctly pass all python flags given upon startup. I don't know anyone who has done specific testing to quantify the speed increase, though.
There is a drawback: Debugging becomes a lot harder and sometimes impossible because you get no or faulty line number information in tracebacks and when using pdb.
That isn't true. Line numbers are included and valid. All -O does is: - cause __debug__ to be false - cause assert statements to be noop. ZODB, and especially ZEO, have lots of logging calls conditioned on __debug__, and running with -O can have a measurable (a few percent, If I remember correctly) impact on performance especially when lots of database access is going on. I think this effect is more pronounced for ZEO storage servers. Production Zope and ZEO servers should run with -O IMO. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
On Mon, Dec 19, 2005 at 06:50:19AM -0500, Jim Fulton wrote: | >There is a drawback: Debugging becomes a lot harder and sometimes | >impossible because you get no or faulty line number information in | >tracebacks and when using pdb. | | That isn't true. Line numbers are included and valid. | | All -O does is: | | - cause __debug__ to be false | | - cause assert statements to be noop. I think Jens was referring to the fact that since some version (which I don't remember correctly), when run with -O python will show the line number of the function definition on tracebacks instead of the real line number where the exception occurred. -- Sidnei da Silva Enfold Systems, LLC. http://enfoldsystems.com
Sidnei da Silva wrote:
On Mon, Dec 19, 2005 at 06:50:19AM -0500, Jim Fulton wrote: | >There is a drawback: Debugging becomes a lot harder and sometimes | >impossible because you get no or faulty line number information in | >tracebacks and when using pdb. | | That isn't true. Line numbers are included and valid. | | All -O does is: | | - cause __debug__ to be false | | - cause assert statements to be noop.
I think Jens was referring to the fact that since some version (which I don't remember correctly), when run with -O python will show the line number of the function definition on tracebacks instead of the real line number where the exception occurred.
I don't think this is the case for recent versions of Python. Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org
On Mon, Dec 19, 2005 at 07:12:44AM -0500, Jim Fulton wrote: | Sidnei da Silva wrote: | >On Mon, Dec 19, 2005 at 06:50:19AM -0500, Jim Fulton wrote: | >| >There is a drawback: Debugging becomes a lot harder and sometimes | >| >impossible because you get no or faulty line number information in | >| >tracebacks and when using pdb. | >| | >| That isn't true. Line numbers are included and valid. | >| | >| All -O does is: | >| | >| - cause __debug__ to be false | >| | >| - cause assert statements to be noop. | > | >I think Jens was referring to the fact that since some version (which I | >don't remember correctly), when run with -O python will show the line | >number of the function definition on tracebacks instead of the real | >line number where the exception occurred. | | I don't think this is the case for recent versions of Python. Indeed, was speaking from memory, and understood the SET_LINENO changes backwards. Mark help me out understand the issue: """ actually, when -O was first introduced, -O did magic with the SET_LINENO opcode. but in recent (even 2.3 I believe), even non-optimised Python manages to use the same perf tricks, but keep a correct lineno. This is reflected if you ever look in the debugger - a frame's f_lineno structure element is indeed only ever the function decl. Magically f_lineno as exposed to Python does get the right thing though... """ -- Sidnei da Silva Enfold Systems, LLC. http://enfoldsystems.com
On 19 Dec 2005, at 12:00, Sidnei da Silva wrote:
On Mon, Dec 19, 2005 at 06:50:19AM -0500, Jim Fulton wrote: | >There is a drawback: Debugging becomes a lot harder and sometimes | >impossible because you get no or faulty line number information in | >tracebacks and when using pdb. | | That isn't true. Line numbers are included and valid. | | All -O does is: | | - cause __debug__ to be false | | - cause assert statements to be noop.
I think Jens was referring to the fact that since some version (which I don't remember correctly), when run with -O python will show the line number of the function definition on tracebacks instead of the real line number where the exception occurred.
Thanks Sidnei, that's what I meant. Just didn't remember the details, something with line numbers ;) jens
Am Sonntag, den 18.12.2005, 17:46 +1100 schrieb Alan Milligan:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I'm reconsidering the way we RPM package byte-code compiled python, and although I suspect optimised python is a bit of an anacronysm, I thought I'd check with the list ;)
If one was to start Zope with python -O, then it would look for (and generate) .pyo files instead of .pyc's in all the Products right?
Would there be any performance gain by this change? How much?
Is it worth me doing byte-compiles with -O and starting Zope's interpreter in this mode?
Debian python installer makes two "compile" steps after installation, one to create the .pyc and one to create all the .pyo files. I'd say apart from disk space it does not hurt :) And you are free to start your python apps in the way you like.
participants (5)
-
Alan Milligan -
Jens Vagelpohl -
Jim Fulton -
Sidnei da Silva -
Tino Wildenhain