Thanks for the fast reply Casey. Casey Duncan wrote:
On Thursday 27 September 2001 12:48 pm, Oliver Bleutgen allegedly wrote:
Hi, I'm resending this to zope-dev because on zope nobody answered, it would be very nice if someone could step up with a small hint.
Can somenone briefly explain what exactly gets locked in zope 2.3.3's catalog when it tries to index an object which is hold in a version? The whole catalog?
Any Btree buckets (in the indexes) which get changed get locked, which can effectively lock out other changes to the catalog. This is a limitation of the way the indexes are implemented, for which, sadly, there is no easy fix.
I found some answers which indicate that it has to do with the catalog when we see version lock errors where there shouldn't be any (from a naive POV). I would like to know how far reaching these problems are, because I'm currently rewriting an application, and I might be able to work around that.
I would suggest that if any objects are reindexed in a version, that no cataloged objects should be reindexed in any other version until that version is saved.
I hope I understand you correctly, but I'd say that if there's already a locked bucket, I've lost. There's no guarantee how long this particular version will stay uncommitted. I would also have to check anyhow whether there is a lock somewhere in the catalog's index.
You could also get around that by deferring the indexing until the version is commited, but this will take some coding on your part.
This seems easier - if I'm a little bit lax about when the indexing occurs. Like making the object only index/reindex/unindex itself if it's not in a version, and combining that with a nightly cronjob which reindexes all (non-versioned) documents. With "some coding on your part" you mean making Version.py more intelligent? Like instead of just doing commitVersion(s,'') doing the following: 1) search for objects which have been deleted in the version 2) search for objects which want to catalog themselves and are locked in version s 3) unindex the objects found in 1) 4) commit version 5) index/reindex objects found in 2) Where it's not clear to me how to prevent that 3) and 5) will not itself get versioned.
Ultimately I think that ZCatalog should do this for you, or at least somehow let indexes have concurrent versioning (any volunteers?)
I should say that I really don't grasp this ZODB voodoo, but I suspect that this will also be not too easy. AFAIK, the decision to write "in" a version (and in which) is taken deep down in zope's innards.
I am thinking about writing a fishbowl proposal for ZCatalog upgrades sometime next month, and this is one potential problem areas to address there, especially as things like the CMF make it more ubiquitous
If I don't misunderstand how versions work (not unlikely), it might be necessary in the end rather to improve versions than the zcatalog in order to remedy this specific problem. thanks again, oliver