ZCatalog + ZCatalog
I am making it my mission to enhance the beast that is ZCatalog. This is the first of many improvements I hope to make to it as time goes on. <DISCLAIMER> This patch has only been moderately tested, and all side affects are not known. I am submitting it here for review before adding it to the collector. You take full risk in using this patch on your installation. </DISCLAIMER> OK, what this patch does is allow you to concatenate ZCatalog result sequences (Lazy sequences) without loading the whole enchilada into memory. It also dispenses with the funky workaround/hack notation that currently exists (although it still works AFAIK). Once this patch is in place, you will be able to write code as follows: <dtml-in "Catalog(<args>)+Catalog(<other args>)+..."> ... </dtml-in> or even: <dtml-in "Catalog1(<args>)+Catalog2(<args>)"> ... </dtml-in> The latter might be useful for dealing with ZCatalog scaling issues, or combining completely objects from entirely different locations. The patch changes Lazy.py and Catalog.py (very slightly). These files can be found in your {Zopedir}/lib/python/Products/ZCatalog directory. Add the following code into the Lazy class definition in Lazy.py. I added it at the end following line 118 (slice=__getslice__): def __add__(self,other): # Concatenate result sequences together while remaining lazy # Make sure the object we are concating is also Lazy try: if other.__class__.__bases__[0].__name__ != 'Lazy': raise TypeError, """Cannot concatenate objects. Both must be lazy result sequences (ie ZCatalog results)""" except: raise TypeError, """Cannot concatenate objects. Both must be lazy result sequences (ie ZCatalog results)""" if self.__class__.__name__ == 'LazyCat': # Don't nest LazyCat objects unnecessarily if hasattr(self,'_seq'): seq = self._seq else: seq = [self._data] else: seq = [self] if other.__class__.__name__ == 'LazyCat': # Don't nest LazyCat objects unnecessarily if hasattr(other,'_seq'): seq = seq + other._seq else: seq = seq + [other._data] else: seq = seq + [other] return LazyCat(seq) The change to Catalog.py is a single line as follows: Change line 603 (if not r: return r) to: if not r: return LazyCat(r) The change to Catalog.py prevents empty catalog search results from throwing an exception when concatenating it with a non-empty search result. This is the only part that I see as a risk to break existing code. If you, for whatever reason, relied on a ZCatalog returning an empty list ([]) when the search came up empty, your code will no longer work. Here is an example: <dtml-let result="Catalog(<args>)"> <dtml-if "result == []"> <--- This no longer works ... <dtml-else> ... </dtml-if> </dtml-let> However this still does: <dtml-let result="Catalog(<args>)"> <dtml-if result> ... </dtml-if> </dtml-let> And this does what you expect of it when both searches come up empty: <dtml-in "Catalog(<args>)+Catalog(<other args>)"> ... <dtml-else> ... </dtml-in> IMHO this is a very minor side affect, and well worth the gain. Please scrutinize this patch and pick it apart and let me know what works and what doesn't or if I've missed something. Enjoy, Casey Duncan
Casey Duncan wrote:
OK, what this patch does is allow you to concatenate ZCatalog result sequences (Lazy sequences) without loading the whole enchilada into memory. It also dispenses with the funky workaround/hack notation that currently exists (although it still works AFAIK). Once this patch is in place, you will be able to write code as follows:
<snip> This looks great. Have you put it in the collector yet? (http://classic.zope.org:8080/Collector/ as a 'Feature w/patch') You should chat to Chris Petrilli as he too seems to be on a Catalog mission, ATM :-) cheers, Chris
Yes, a revised version of this has been submitted to the collector. If anyone is interested, the revised Catalog files can be found on my Zope home page at http://www.zope.org/Members/Caseman I actually have talked to Chris Petrilli about some other ideas for enhancing the Catalog. I want to add a query method that allows you to pass a query string something like an SQL where clause. Chris suggested we start a fishbowl discussion about it. I'm working on a product for creating JPython applets within Zope at the moment, although that should be done fairly soon I think. Once that is done and things in the Catalog for Zope 2.3 get a bit more finalized, I will push that project forward. Other ideas on enhancing the Catalog are also welcome. -Casey D. -----Original Message----- From: Chris Withers [mailto:chrisw@nipltd.com] Sent: Monday, November 13, 2000 3:32 AM To: casey.duncan@state.co.us Cc: zope-dev@zope.org; Christopher Petrilli Subject: Re: [Zope-dev] ZCatalog + ZCatalog Casey Duncan wrote:
OK, what this patch does is allow you to concatenate ZCatalog result sequences (Lazy sequences) without loading the whole enchilada into
memory.
It also dispenses with the funky workaround/hack notation that currently exists (although it still works AFAIK). Once this patch is in place, you will be able to write code as follows:
<snip> This looks great. Have you put it in the collector yet? (http://classic.zope.org:8080/Collector/ as a 'Feature w/patch') You should chat to Chris Petrilli as he too seems to be on a Catalog mission, ATM :-) cheers, Chris
participants (2)
-
Casey Duncan -
Chris Withers