[Zope-dev] ZCatalog + ZCatalog

[Casey Duncan]

I am making it my mission to enhance the beast that is ZCatalog. This is the
first of many improvements I hope to make to it as time goes on.

<DISCLAIMER>
This patch has only been moderately tested, and all side affects are not
known. I am submitting it here for review before adding it to the collector.
You take full risk in using this patch on your installation.
</DISCLAIMER>

OK, what this patch does is allow you to concatenate ZCatalog result
sequences (Lazy sequences) without loading the whole enchilada into memory.
It also dispenses with the funky workaround/hack notation that currently
exists (although it still works AFAIK). Once this patch is in place, you
will be able to write code as follows:

<dtml-in "Catalog(<args>)+Catalog(<other args>)+...">
   ...
</dtml-in>

or even:

<dtml-in "Catalog1(<args>)+Catalog2(<args>)">
   ...
</dtml-in>

The latter might be useful for dealing with ZCatalog scaling issues, or
combining completely objects from entirely different locations.

The patch changes Lazy.py and Catalog.py (very slightly). These files can be
found in your {Zopedir}/lib/python/Products/ZCatalog directory.

Add the following code into the Lazy class definition in Lazy.py. I added it
at the end following line 118 (slice=__getslice__):

    def __add__(self,other):
        # Concatenate result sequences together while remaining lazy

        # Make sure the object we are concating is also Lazy
        try:
                if other.__class__.__bases__[0].__name__ != 'Lazy':
                        raise TypeError, """Cannot concatenate objects. Both
must be lazy result sequences (ie ZCatalog results)"""
        except:
                raise TypeError, """Cannot concatenate objects. Both must be
lazy result sequences (ie ZCatalog results)"""

        if self.__class__.__name__ == 'LazyCat':
                # Don't nest LazyCat objects unnecessarily
                if hasattr(self,'_seq'):
                        seq = self._seq
                else:
                        seq = [self._data]
        else:
                seq = [self]

        if other.__class__.__name__ == 'LazyCat':
                # Don't nest LazyCat objects unnecessarily
                if hasattr(other,'_seq'):
                        seq = seq + other._seq
                else:
                        seq = seq + [other._data]
        else:
                seq = seq + [other]

        return LazyCat(seq)

The change to Catalog.py is a single line as follows:

Change line 603 (if not r: return r) to:
	if not r: return LazyCat(r)

The change to Catalog.py prevents empty catalog search results from throwing
an exception when concatenating it with a non-empty search result. This is
the only part that I see as a risk to break existing code. If you, for
whatever reason, relied on a ZCatalog returning an empty list ([]) when the
search came up empty, your code will no longer work. Here is an example:

<dtml-let result="Catalog(<args>)">
<dtml-if "result == []">		<--- This no longer works
   ...
<dtml-else>
   ...
</dtml-if>
</dtml-let>

However this still does:

<dtml-let result="Catalog(<args>)">
<dtml-if result>
   ...
</dtml-if>
</dtml-let>

And this does what you expect of it when both searches come up empty:

<dtml-in "Catalog(<args>)+Catalog(<other args>)">
   ...
<dtml-else>
   ...
</dtml-in>

IMHO this is a very minor side affect, and well worth the gain.

Please scrutinize this patch and pick it apart and let me know what works
and what doesn't or if I've missed something.

Enjoy,
Casey Duncan