On Fri, 2002-06-28 at 22:43, Dieter Maurer wrote:
Thierry Florac writes:
... 1. I have a new Zope database (= 2.5 Mo). After adding a single PDF file of 3.5 Mo, the database is nearly 10 Mo huge !! Is required space into ZODB "reserved" by big blocks, or may the ZODB's size increase twice as the size of files which are stored into it ?? Objects are stored as pickles. There are larger (usually slightly larger) than the original object.
What are "pickles" ?? What do you mean when you say "slightly larger" ???
I am astonished that you see your database grow by twice the amount one would expect. Try whether packing reduces the size. Maybe, the object is accidentally modified.
I've done a really simple test, adding the PDF file as a "File" object from the ZMI. No property was modified... But you you mean also that when a single little property like "title" is modified, it's the whole object which is replicated into the ZODB ???
2. .... Does it means that full text indexing is using as much space as the document itself (but I'm starting with an empty ZCatalog) ?? Together with each object, the index maintains the set of all its words. This is done in order to be able to unindex the object without the need to touch every indexed term (but only those contained in the object). As a consequence, each object takes almost as much space in the index as it takes itself (at least for textindexes with many different words).
3. I'd like my products instances to be indexed automatically, as soon as they are created or modified. But as indexing seems to be quite long with big files, is it possible to index these documents in the background, so that the user does not have to wait until indexation is finished... Shane et. al. has some idea in this direction:
Perform indexing in a background task.
Indexing requests are put onto a work queue. When Zope is lazy, a background task fetches requests from the work queue and executes them.
You need be a bit careful. Read the ZODB description (--> Zope.org) to understand the interaction between ZODB connections, transactions and threads.
Thierry