[Zope-dev] apelib performance

[Chris Kratz]

Hello All,

Our application contains about 38k objects running via apelib into the 
filesystem using the filesystem mapper.  We have been extremely happy with the 
ability to use version control on everything in zope which apelib affords us 
as well as allowing our developers to use editors/ide of choice.

The problem we keep running into is performance.  On even fast machines, right 
after zope starts up, it is extremely slow.  Once the cache get's populated, 
the site runs as normal (we have the cache set to 10k objects).  

On a server, we set the scan-interval to 5 minutes or higher.  But on our 
development boxes, we have the scan-interval set to 
1s.  This allows changes as they are made on the file system to show up in 
zope for testing immediately.  This works quite well, but is very, very slow 
for developing under unless you have a pretty heavy duty processor.  In 
profiling this, it doesn't appear to be I/O bound (as I would have guessed), 
but instead processor bound.  Profiling points to Apelib as being the 
bottleneck (getmtime, getannotations, compile, etc).  

To verify this,  I copied our application back into a standard zope storage 
and the performance after startup was night and day different.  Very snappy.  

Has anyone found a way to make apelib faster, or have hints on performance 
tuning?

Is there a better way then apelib to get zope objects (dtml, python scripts, 
zsql statements) into text files?

Thanks for your time,

-Chris