[Zope-dev] ZCatalog hackery

[Chris McDonough]

Has the physical path of the object changed?  If not, the newer (2.3.0 + )
catalog stuff should be smart enough to figure out whether anything inside
the object has changed during catalog_object.  If nothing has changed, none
of the indexes or metadata columns should be updated.  We're scrambling
right now to write tests for this kind of thing.  :-(

Note that the algoritm is simple - for each index, compare the what exists
in the index to what is to be put in.  If they're the same, do nothing.  If
they're different, reindex.  I wasn't able to understand completely from
your description whether the object method your're attempting to index via a
TextIndex actually returns different data or not when you recatalog it.
Does it?

There's a huge possibility that the merge stuff isn't working completely
because until now there have been no deterministic tests.

----- Original Message -----
From: "R. David Murray " <bitz@bitdance.com>
To: <zope-dev@zope.org>
Sent: Saturday, March 03, 2001 2:29 PM
Subject: Re: [Zope-dev] ZCatalog hackery


> I've got an analogous but different problem with ZCatalog udpates.  I'd
> like to ask for ideas about the best way to handle this.
>
> I've got a ZPatterns based ap that is pulling data from a postgresql
> database.  But associated with the object created out of the database
> are signficant chunks of html (Author bios, book descriptions).  I've
> stored these html chunks in the database as well, though I don't think
> that matters to the problem or possible solution strategies.
>
> The issue is that parts of the database get updated periodically from an
> external source.  That is, the author and book tables get replaced
> wholesale.  But the Book and Author objects are cataloged using
> ZCatalog, doing a full text index on a combination of fields from
> the external database and the html chunks.  As far as I can see
> at the moment, this means that every object has to get uncataloged
> and recataloged, meaning every time the database update happens
> the zodb inflates considerably, even though very little data has
> actually changed.
>
> So far I can think of two simple solutions to this problem: (1) pack
> often (2) put the Catalog into a mounted storage backed by a
> non-undoable storage.   I'd rather not do either one of these
> (the first for obvious reasons, the second simply because I don't
> want to take the time to learn how to set up a non-undoable storage).
> Am I missing some other obvious options?  It seems like there *ought*
> to be a way to avoid the overhead of updating the catalog for objects
> that haven't really changed.
>
> --RDM
>
>
> _______________________________________________
> Zope-Dev maillist  -  Zope-Dev@zope.org
> http://lists.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists -
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope )
>