[Zope-dev] ZCatalog hackery

[R. David Murray]

I've got an analogous but different problem with ZCatalog udpates.  I'd
like to ask for ideas about the best way to handle this.

I've got a ZPatterns based ap that is pulling data from a postgresql
database.  But associated with the object created out of the database
are signficant chunks of html (Author bios, book descriptions).  I've
stored these html chunks in the database as well, though I don't think
that matters to the problem or possible solution strategies.

The issue is that parts of the database get updated periodically from an
external source.  That is, the author and book tables get replaced
wholesale.  But the Book and Author objects are cataloged using
ZCatalog, doing a full text index on a combination of fields from
the external database and the html chunks.  As far as I can see
at the moment, this means that every object has to get uncataloged
and recataloged, meaning every time the database update happens
the zodb inflates considerably, even though very little data has
actually changed.

So far I can think of two simple solutions to this problem: (1) pack
often (2) put the Catalog into a mounted storage backed by a
non-undoable storage.   I'd rather not do either one of these
(the first for obvious reasons, the second simply because I don't
want to take the time to learn how to set up a non-undoable storage).
Am I missing some other obvious options?  It seems like there *ought*
to be a way to avoid the overhead of updating the catalog for objects
that haven't really changed.

--RDM