[Zope-CMF] Re: Making TypesTool faster

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Alexander Limi wrote:
> Hi all,
> 
> I would like some input on making CMF (and by extension Plone) more  
> efficient.
> 
> This issue goes back to the original discovery that Zope had a bug that  
> made its Product lookup mechanism be very slow in Zope 2.10.0/1, and  
> affected TypesTool lookups in particular.
> 
> When we experimented with this during the Performance Sprint at Google, we  
> lowered the call time from 6.26s to 0.29s by caching it in a thread local  
> cache.
> 
> (All calls and timings are 10 calls to the standard Plone front page on a  
> 2GHz MacBook)
> 
> The contentmenu.pt template in Plone (which pretty much just does the  
> TypesTool lookup for what types are addable), currently takes up roughly  
> 30% (!) of the total cost of rendering a standard page in Plone.
> 
> After the bug fix was applied to Zope 2.10.3, the call went down to 0.80s  
> with the same hardware/#calls — there's still a significant potential for  
> improvement here: 0.80s -> 0.29s.
> 
> Here's what Alec Mitchell wrote about it at the time:
> """
> The fundamental issue is not that product lookup is slow in Zope, but that
> CMF uses the Product lookup mechanism as a registry without any sort
> of caching.  This is quite easy to fix either in Plone or in the CMF
> depending on the timeframe we need it merged in for.  I initially used
> a thread local for safety because I wasn't entirely sure what sorts of
> things would end up in this cache.  However, by now it's pretty clear
> to me that an instance level cache should be perfectly safe, which
> should be a little faster yet.
> 
> """
> 
> So what I'm wondering is whether it's possible to get this instance-level  
> caching of the lookup in the types tool before CMF 2.1 final? I'd  
> obviously prefer it if Plone didn't have to subclass and override parts of  
> TypesTool to fix this. :)
> 
> (Oh, and for reference — the original thread local cache that we used for  
> testing is here: http://paste.plone.org/13211 )

First, thanks for investigating.  I know Plone developers have suspected
that the "add list" (which is rendered on every page view in a typical
Plone site) was a major source of sluggishness;  I'm glad to get it
confirmed with specifics.

Before we look at optimizing the use of the product dispatch mechanism,
I'd like to propose deprecating it in favor of the factory-utility
mechanism:  leaving the 'product' field blank in an FTI, and having the
'factory' field be the name of a utility registered for
'zope.component.interfaces.IFactory'.

Typically, the code to enable this looks like the following (the CMF
already does this)::

  # Products/MyProduct/MyContent.py
  from zope.component.factory import Factory
  ...
  MyContentFactory = Factory(MyContent)  # right after InitializeClass


  # Products/MyProduct/configure.zcml
  ...
  <utility
      component=".MyContent.MyContentFactory"
      name="myproduct.mycontent"
      />
  ...

  # Products/MyProduct/profiles/myproduct/types/MyContent.xml
  ...
  <property name="product"></property>
  <property name="factory">myproduct.mycontent</property>
  ...


If y'all still have your benchmarks available, could you publish the
setup code you used?  I'd really like to see the results on a site set
up using the new-style factories.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD4DBQFGOL1N+gerLs4ltQ4RAmkhAJdpZCdNGSH8K/LKl9R3eRQEES58AJ0XH/wF
FUwezLhNvQdLkmjeXtdo9A==
=PXSC
-----END PGP SIGNATURE-----