[Zope] Intersection/Union of ZCatalog result sets

[Johan Carlsson]

Jonathan Hobbs wrote:

>From: "Johan Carlsson" <johanc at easypublisher.com>
>  
>
>>Why would it be smaller?
>>You still need to load the indexes when you do a search, right?
>>Or do you intend to index different objects in different catalogs?
>>In that case couldn't you use the idxs attribute
>>of ZCatalog::catalog_object(self, obj, uid=None, idxs=None,
>>update_metadata=1)?
>>    
>>
>
>Moving only the ZCTextIndex (and its Lexicon) into a separate ZCatalog
>should result in a smaller ZCatalog, as the space required by the other 4
>indexes (3 Field Indexes and another ZCTextIndex) will be located in a
>different folder - I am going to load the ZCatalog containing the main
>ZCTextIndex into a Temporary Folder (to hold it in memory).
>  
>
You could also always create an external (to ZCatalog)  Id Generator 
Service, that can be accessed from both zcatalogs/catalogs
to get a unique RID that can be used in both catalogs. Skiping the 
problem with longs and potentially
the problem of supporting a modified version of BTrees.

There's some code for making transition-aware counter that you might 
want have a look at.
I guess it needs some improvements though?

#This is browed from Zope 2.4.3 ZODB.tests.ConflictResolution
from Persistence import Persistent
#This PCounter doesn't provide a unique ID.
#It does increment ones per call (even if several threads collide)
#but the value returned will be +2 for both threads.
class PCounter(Persistent):
    _value = 0
    def __init__(self, val=None):
        if val is not None:
            if type(val)==IntType:
                self._value=val
            elif hasattr(val, '_count'):
                self._value=getattr(val, '_count',0)
            else:
                self._value=0
    def __repr__(self):
        return self._value
    def getUniqueId(self):
        self._value = self._value + 1
        return self._value
    def _p_resolveConflict(self, oldState, savedState, newState):
        savedDiff = savedState['_value'] - oldState['_value']
        newDiff = newState['_value'] - oldState['_value']
        oldState['_value'] = oldState['_value'] + savedDiff + newDiff
        return oldState

class PCounter2(PCounter):
    def _p_resolveConflict(self, oldState, savedState, newState):
        raise ConflictError

>Thanks for the 'heads-up'.  I had hoped to use OIDs instead of RIDs, but
>hadn't considered the 64/32 bit problem. I'll have to see if I can find a
>64bit BTrees package, and failing that, try to modify the existing package
>to use long ints - this just keeps getting better and better :)
>  
>
Cool!
I love to hear how this turns out, so please keep me posted :-)

>Thanks for the help!
>  
>
My pleasure :-)