"Phillip J. Eby" wrote:
I've been spelunking through the ZCatalog source this morning, and there are a couple of things I don't understand, that look to me like memory leaks and other problems.
First, in Catalog.py, it seems like virtually every method of the Catalog class calls self.useBrains(). useBrains() creates a new class each time it's called, and then deliberately creates a circular reference in that class, such that it can't be deleted. Finally, the logic of where useBrains() is called, is such that it will be called once for every item retrieved from every search done on the catalog. Thus, every record should end up as an instance of a different class, specifically created for that record. This has me baffled, because I can't see why that would be something useful. Am I missing something really deep here, or is it just a bug? It would seem to me that there should be something like "_v_brains_inuse =_v_brains" at the end of the useBrains() method, so that at the beginning you can check for whether "brains is _v_brains_inuse" and skip creating a new class, etc. once the brains are initialized.
The circular reference thing is a hold over from ZTables. I don't know why it was there, and I preserved it. I do think you are right about the calling logic on useBrains. I will look into that early this week. At the same time, i will experiment with taking out the circular link and seing if anything breaks.
The next thing I've noticed, is that Catalog.py imports from Query, but then never actually uses anything from it. Oh, sure, there's an orify() function that does stuff, and a query_map local variable set up in searchResults() that references orify, but query_map never gets used. (And ZCatalog.py makes up another copy of the same query_map object and passes it in, but that never gets used either.) I even dug down to Index.py and TextIndex.py to make sure query_map wasn't used down at that level, and then I realized that query_map wasn't part of the namespace being passed down there anyway. This all looks like some sort of vestigial leftovers, perhaps during porting from ZTables...?
Probably. I would have to look at the model, but it might have something to do with the Z Search Interface.
The third thing I saw, is that there are lots of constructs of the form:
if something in somethingelse.keys():
where the object 'somethingelse' appears to have a perfectly good has_key() method available. Again, is there some sort of deep Zen here, like trying to force index pages to be loaded, or...? I mean, it seems like this would be much less efficient than just hitting the has_key() methods.
My Python amateurism showing through there. I'll make those changes.
Last, and definitely least, I notice that useBrains() uses a loop to create scopy, when "scopy=self.schema.copy()" would suffice. That wouldn't be anything worth 'fixing', except considering how often that code would appear to be being called.
This is also code straight out of ZTables. I will examine your suggestion.
I'm really tempted to run something through the debugger or profiler just to see if I'm totally off base on how often useBrains() is getting called. But I thought I'd run this by the experts, since if y'all did this stuff on purpose, you'd be able to tell me. :)
Well I had some problems with keeping the brains set early on in development, I may have been overzealous on my calling logic instead of thinking. Jim say's thinking is bad. ;)
Oh, one other thing. Not a bug, but a feature... It seems to me that if you are searching a ZCatalog on index 'foo', it looks possible to do something like:
Catalog(foo=['bar','baz'],foo_usage='range:min:max')
And receive records with values of foo ranging from 'bar' to 'baz'. Is this a supported, intended feature, or...?
It is an intended and supported feature yes, but not very well documented. This is partuclarly handy for doing, as you deduced, range searches. For example, the ZPT has a pull down box on it's advanced search page that lets you pick date ranges in simple english like "new since yeserday" and "new since last week" etc. Note that range searching only works on FieldIndexes. -Michel
Just wondering.
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://www.zope.org/mailman/listinfo/zope-dev
(To receive general Zope announcements, see: http://www.zope.org/mailman/listinfo/zope-announce
For non-developer, user-level issues, zope@zope.org, http://www.zope.org/mailman/listinfo/zope )