[Zope] Re: updating catalog takes forever (never really completes)

[Andreas Kostyrka]

Am Mon, 2002-09-23 um 15.09 schrieb Luca Olivetti:
> Chris Withers wrote:
> > Luca Olivetti wrote:
> > 
> >> In indexes, yes, that's not really redundant, is an index after all.
> >> But metadata? It seems it is just a workaround to a limitation of the 
> >> zodb.
> > 
> > 
> > Urm, no, it is a form of caching.
> > 
> > Have you heard of apps like Squid, Mozilla, Internet Explorer ? ;-)
> 
> Yes, caching is a workaround for dealing with slow backends, with the 
> risk of getting stale data.
Well, nothing magic here. If you got CatalogAware object, your Catalog
gets reindexed automatically the same way an index in database system
> If the backend is fast enough there should be no need for caching (and 
> for duplicating the same data again and again).
Well, every backend hits somewhere a ceiling. Zope is (depending upon
your methods, in my test with some basic navigation stuff) not really
that much slower than an static Apache fileserver. (I get about 60-70%
of requests compared to static file on apache).

Now every website that offers "Search the website" functionality has
some kind of search engine. And all these have index files that can
become stale.

Zope's solution offer only benefit over the competition:
-) it's more flexible (because Zope is dynamic). Nobody considers using
   a search engine on it's website to provide navigation.
-) with the right kind of objects (and plain OFS objects are not really 
   useable for content) Catalog are never stale, because they are 
   autoupdated.
-) it's builtin and it's fast. 
-) it can do quite expressive searches, because it's more than just a 
   full text database.

Basically: Yes it's reduant data. And yes we all learned that reduant
data is bad at University. Well, and some of us learned in the
real-world that sometimes reduant data is not bad at all :)
Especially in this case, as it is just an cache, that can be
automatically validated.

Well, OTOH, perhaps your are that pure in your thinking, ...
But then you should turn of the CPU caches off too, that is reduant data
too. (And with write back caches it's not even reduant data that is NOT
IN SYNC!)

Seems to be you've missed some courses on system design ;)

Andreas