[Zope-dev] ZCatalog caching with memcached

[Jens Vagelpohl]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Oct 27, 2008, at 13:08 , Roché Compaan wrote:

> On Sun, 2008-10-26 at 14:07 -0400, Tres Seaver wrote:
>> - Plone uses too many indexes, and in particular, uses multiple text
>>   indexes.  Having extra indexes around "just in case" is a sure lose
>>   a write time, and may even be expensive at query time (depending on
>>   the query).
>>
>> - Particular indexes have performance characteristics based on their
>>   designed purpose:  for instance, the stock FieldIndex  
>> implementation
>>   assumes that the number of documents indexed will be >> the  
>> number of
>>   discrete indexable values.  Using such an index in an application
>>   domain with a very large set of indexable values probably loses,  
>> and
>>   in ways which don't show up in early / small-scale testing.
>>
>> - I'm pretty sure that we haven't yet found the best data structure  
>> for
>>   "hierarchy indexes" (e.g., the Plone EPI index, or the stock Zope2
>>   PathIndex, etc.).  Something like a 'trie' might be optimal for
>>   pure prefix searching of hierarchies.
>>
>> - I am confident that the TopicIndex is underutiliized:  it does  
>> *all*
>>   the work for a given query at write time, and can thus be  
>> blindingly
>>   fast at query time.
>>
>> - Other special-purpose indexes (e.g., a "recent items" index) would
>>   be worth a look, especially for applications with large volumes of
>>   content.
>
> I agree that one should look at improving performance without  
> caching as
> well. But this is a lot harder and takes significantly more  
> development
> and debugging time than introducing some form caching. So I'm not
> convinced that it needs to happen in a certain order. If caching gives
> you lots of performance with little effort now, then why shouldn't you
> use it?

It's the typical trade-off. One course is expedient and fast for your  
use case now. The other requires more resources, but benefits  
everyone. Including those who don't want to depend on yet another  
package, like memcached, for performance.

When it comes to integrating anything in Zope itself I'd choose the  
latter.

jens


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (Darwin)

iEYEARECAAYFAkkFssEACgkQRAx5nvEhZLITiQCgskifGYaixaj6lVLk85l6rz6E
aQwAoI9PRcJHL8oZPatlHWADA0h6orCe
=YLhP
-----END PGP SIGNATURE-----