[ZODB-Dev] Non versioned objects

[Jim Fulton]

Cyril Elkaim wrote:
> The reason I need this and I asked for is that I maintain Persistent 
> indexes in the ZODB.

Technically, I'd call this an optimization, not a need.

> Each time I add or modify an entry in an index it get duplicated by the 
> transaction.
> And I don't need this behavior> Of course one solution is to have Indexes NOT Persistent and reindex at 
> each
> application startup. It should not be more lengthy than a packing of the 
> database.
> Another is to have subtransaction with many objects between like 
> ZCatalog but it's not
> an option in my case.
> 
> Anyway if some helpful mind can take a look at this or give me a pointer 
> to where I
> must look in the ZODB code... :-)

If, in a transaction, you didn't store previous versions for some objects,
then you couldn't undo that transaction, because, if you did, you'd
undo to an inconsistent state. So, if, for a transaction, you didn't store
previous versions for some objects, you might as well not store previous
versions for *all* objects.  This would be a lot simpler.
We have a similar desire for something like this. We are taking a two-
pronged approach:

1. We've implemented a queued catalog. (The initial version is in
    Products/QueueCatalog in CVS if you want to play with it.  This thing
    queues catalog updates persistently. You need to write a separate process
    or thread (or cron job) that periodically processes the queue. This
    has a number of advantages:

    - You can combine many updates making cataloging more efficient in terms
      of processing time and database growth.

    - You make content updates much faster, since updates don't have to
      wait for cataloging.

    - Duplicate updates are combined.

    - Undo is supported even if later transactions added events to the queue.
      This will make undo useful even when the catalog is involves.

    This is designed to work with ZCatalog, but you should be able to
    adapt the ideas to other indexing schemes.

2. We put the catalog data in a separate non-undoing storage, like the
    Berkeley packless storage.


Jim


-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (888) 344-4332            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org