[ZODB-Dev] profiling zLOG and inspect

[Guido van Rossum]

> My 2.1/2.2 profile runs are showing me that the import zLOG step is
> taking a bit less than 1 second on some of the slower boxes here
> (400~500mhz). In the end, it piles down to the fact that import inspect
> takes an amazingly long time:
> 
>     kiko at banzai:~/ZODB3/ZODB$ python2.1 
>     Python 2.1.3 (#1, Sep  7 2002, 15:29:56) 
>     [GCC 2.95.4 20011002 (Debian prerelease)] on linux2
>     Type "copyright", "credits" or "license" for more information.
>     You have pprint, _halt, class Foo Bar:Foo available"
>     To call _halt, use _halt(locals())
>     >>> import time; t = time.time() ; import inspect ; time.time() - t
>     0.76367998123168945
> 
>     kiko at banzai:~/ZODB3/ZODB$ python2.2 
>     Python 2.2.1 (#1, Sep  7 2002, 14:34:30) 
>     [GCC 2.95.4 20011002 (Debian prerelease)] on linux2
>     Type "help", "copyright", "credits" or "license" for more information.
>     You have pprint, _halt, class Foo Bar:Foo available"
>     To call _halt, use _halt(locals())
>     >>> import time; t = time.time() ; import inspect ; time.time() - t
>     0.49863600730895996

inspect imports tons of other modules; it begins with this line:

  import sys, os, types, string, re, dis, imp, tokenize, linecache

Several of these import more things in turn.

> I know that zLOG will only actually output something if EVENT_LOG_* are
> set. What's the opinion of avoiding the zLOG import, and nopping out the
> calls to zLOG.register_subsystem and zLOG.LOG when these environment
> variables are set?
> 
> I realize that in 2.3 things are much better, but I can't run 2.3 on the
> client environment; I suppose others are in a similar spot, and 1s of
> import time in worst-case scenarios isn't something to sneeze at.

Inspect in 2.3 has the same line of imports.  It appears faster
because more of those modules are already loaded during Python's
startup, so the later import doesn't execute any code.

> (As a parting note, the whole ZODB import takes an amazingly long time:
>     >>> import time; t = time.time() ; import ZODB ; time.time() - t
>     1.7814439535140991
> so I suppose getting that number down would be highly welcome)

Yeah, well, good luck. :-(  ZODB just imports lots of *stuff*.

Anyway, I'm unclear why one-time startup costs are important for you.
Do you start a lot of processes?

--Guido van Rossum (home page: http://www.python.org/~guido/)