[Zope-CMF] Making TypesTool faster

[Alexander Limi]

Hi all,

I would like some input on making CMF (and by extension Plone) more  
efficient.

This issue goes back to the original discovery that Zope had a bug that  
made its Product lookup mechanism be very slow in Zope 2.10.0/1, and  
affected TypesTool lookups in particular.

When we experimented with this during the Performance Sprint at Google, we  
lowered the call time from 6.26s to 0.29s by caching it in a thread local  
cache.

(All calls and timings are 10 calls to the standard Plone front page on a  
2GHz MacBook)

The contentmenu.pt template in Plone (which pretty much just does the  
TypesTool lookup for what types are addable), currently takes up roughly  
30% (!) of the total cost of rendering a standard page in Plone.

After the bug fix was applied to Zope 2.10.3, the call went down to 0.80s  
with the same hardware/#calls — there's still a significant potential for  
improvement here: 0.80s -> 0.29s.

Here's what Alec Mitchell wrote about it at the time:
"""
The fundamental issue is not that product lookup is slow in Zope, but that
CMF uses the Product lookup mechanism as a registry without any sort
of caching.  This is quite easy to fix either in Plone or in the CMF
depending on the timeframe we need it merged in for.  I initially used
a thread local for safety because I wasn't entirely sure what sorts of
things would end up in this cache.  However, by now it's pretty clear
to me that an instance level cache should be perfectly safe, which
should be a little faster yet.

"""

So what I'm wondering is whether it's possible to get this instance-level  
caching of the lookup in the types tool before CMF 2.1 final? I'd  
obviously prefer it if Plone didn't have to subclass and override parts of  
TypesTool to fix this. :)

(Oh, and for reference — the original thread local cache that we used for  
testing is here: http://paste.plone.org/13211 )

-- 
Alexander Limi · http://limi.net