-----Original Message----- From: Jon Udell [mailto:udell@monad.net] Sent: Saturday, January 08, 2000 9:08 PM To: zope-dev@zope.org Subject: [Zope-dev] searching/viewing Zope source?
On the "eat your own dogfood" principle, I decided today that since -- let's face it -- the Zope Python, DTML, and C source files *are* the definitive documentation -- I ought to index them, and why not with ZCatalog?
Sounds reasonable.
First, I imported /zope/lib/python into Zope. Which took longer than I expected, mainly because although I remembered having used fsimport.py in Zope 1.x, I could not find it in 2.x, nor in any searchable Zope place I could think of (zope.org, egroups.com, altavista, deja.com, etc.).
I could not find it either, it's not distributed with Zope. I eventually found it by hunting it down on the old site through the managment 'find'.
Fortunately, I found a backup of Zope 1 that had fsimport.py, so I set it up as an external method, made a Zopedir called src, and ran this:
http://host/src/fsimport?fsdir=/usr/zope2/lib/python
Cool. Now the source tree is in Zope. I added a ZCatalog to the src folder, and told it to add everything. I'd have liked to have added just *.py, *.dtml, and *.c, but none of the strings I could think of typing into the
with ids: containing: expr:
boxes of the "Find Items to ZCatalog"
seemed to find anything. I'm guessing that regexps can be used here somewhere, but none of the usual suspects (zope.org, egroups.com, altavista, deja.com, etc.) yielded any clues. (This, of course, is *why* I was trying to index the source in the first place!)
Not regexprs, but python exprs. 'with ids' does an exact string comparison to the object's id, 'containing' does a grep style search through the object's 'PrincipiaSearchSource' value (slow!), and 'expr' does a python expression evaluation against the object. It's the identical interface to the regular Zope find, which I believe is documented in one of the guides. Note that Zope's 'find mechanism' is pre-catalog, it does not use the catalog in any way, it's alot like UNIX 'find' and 'grep' rolled into one tool.
Whatever, I just took the whole lot, 800 and some files. Then tried searching, and realized -- oops, no text index, only a meta_data index. After searching the usual suspects again (zope.org, egroups.....) I came up with the idea of indexing the 'data' attribute of the File objects.
That sounds right.
But no joy. It's looking like either:
You created a text index called 'data'? That should do it, I'm trying to do it myself but fsimport doesn't apear to work with the new help system in the CVS version of Zope, argh. Did you create a new search interface to query the text index also?
a) wrap those files, in some way, so that they have a catalog-able attribute (which I guess means changing fsimport.py?)
Yes, you would need fsimport to create instances of this new object, instead of File object, but File objects should work on their 'data' attribute, I can't explain why it doesn't.
or
b) use a conventional search engine
I guess b) sounds easier :-)
Not after I fix whatever bug is causing this... It's a short hill up, but it's a long ride down. ;) Obviously, one of the big problems here is documentation, as you mention. There is little to no ZCatalog docs, other than the managment interface, and such a complex beast should have more documentation. Bad me! -Michel