[Zope] ZODB and ZCatalog related questions

[Dieter Maurer]

Thierry Florac writes:
 > ...
 > 1. I have a new Zope database (= 2.5 Mo). After adding a single PDF file
 > of 3.5 Mo, the database is nearly 10 Mo huge !! Is required space into
 > ZODB "reserved" by big blocks, or may the ZODB's size increase twice as
 > the size of files which are stored into it ??
Objects are stored as pickles. There are larger (usually slightly larger)
than the original object.

I am astonished that you see your database grow by twice the amount
one would expect. Try whether packing reduces the size. Maybe,
the object is accidentally modified.

It may well be that the object is rewritten, when you modify
properties in a separate request. I hope not, but there might
be a chance.

 > 2. .... Does it means that full text indexing is
 > using as much space as the document itself (but I'm starting with an
 > empty ZCatalog) ??
Together with each object, the index maintains the set of all its words.
This is done in order to be able to unindex the object
without the need to touch every indexed term (but only those contained
in the object).
As a consequence, each object takes almost as much space in
the index as it takes itself (at least for textindexes with many
different words).

 > 3. I'd like my products instances to be indexed automatically, as soon
 > as they are created or modified. But as indexing seems to be quite long
 > with big files, is it possible to index these documents in the
 > background, so that the user does not have to wait until indexation is
 > finished...
Shane et. al. has some idea in this direction:

    Perform indexing in a background task.

    Indexing requests are put onto a work queue.
    When Zope is lazy, a background task fetches requests
    from the work queue and executes them.

    You need be a bit careful. Read the ZODB description
    (--> Zope.org) to understand the interaction between
    ZODB connections, transactions and threads.


Dieter