[Zope-dev] searching/viewing Zope source?

[Michel Pelletier]

> -----Original Message-----
> From: Jon Udell [mailto:udell@monad.net]
> Sent: Saturday, January 08, 2000 9:08 PM
> To: zope-dev@zope.org
> Subject: [Zope-dev] searching/viewing Zope source?
> 
> 
> On the "eat your own dogfood" principle, I decided today that since --
> let's face it -- the Zope Python, DTML, and C source files *are* the
> definitive documentation -- I ought to index them, and why not with
> ZCatalog?

Sounds reasonable.
 
> First, I imported /zope/lib/python into Zope. Which took longer than I
> expected, mainly because although I remembered having used fsimport.py
> in Zope 1.x, I could not find it in 2.x, nor in any searchable Zope
> place I could think of (zope.org, egroups.com, altavista, deja.com,
> etc.).

I could not find it either, it's not distributed with Zope.  I
eventually found it by hunting it down on the old site through the
managment 'find'.
 
> Fortunately, I found a backup of Zope 1 that had fsimport.py, so I set
> it up as an external method, made a Zopedir called src, and ran this:
> 
> http://host/src/fsimport?fsdir=/usr/zope2/lib/python
> 
> Cool. Now the source tree is in Zope. I added a ZCatalog to the src
> folder, and told it to add everything. I'd have liked to have 
> added just
> *.py, *.dtml, and *.c, but none of the strings I could think of typing
> into the 
> 
> with ids:
> containing:
> expr:
> 
> boxes of the "Find Items to ZCatalog"
> 
> seemed to find anything. I'm guessing that regexps can be used here
> somewhere, but none of the usual suspects (zope.org, egroups.com,
> altavista, deja.com, etc.) yielded any clues. (This, of 
> course, is *why*
> I was trying to index the source in the first place!)

Not regexprs, but python exprs.  'with ids' does an exact string
comparison to the object's id, 'containing' does a grep style search
through the object's 'PrincipiaSearchSource' value (slow!), and 'expr'
does a python expression evaluation against the object.

It's the identical interface to the regular Zope find, which I believe
is documented in one of the guides.  Note that Zope's 'find mechanism'
is pre-catalog, it does not use the catalog in any way, it's alot like
UNIX 'find' and 'grep' rolled into one tool.
 
> Whatever, I just took the whole lot, 800 and some files. Then tried
> searching, and realized -- oops, no text index, only a 
> meta_data index.
> After searching the usual suspects again (zope.org, 
> egroups.....) I came
> up with the idea of indexing the 'data' attribute of the File objects.

That sounds right.
 
> But no joy. It's looking like either:

You created a text index called 'data'?  That should do it, I'm trying
to do it myself but fsimport doesn't apear to work with the new help
system in the CVS version of Zope, argh.  Did you create a new search
interface to query the text index also?

> a) wrap those files, in some way, so that they have a catalog-able
> attribute (which I guess means changing fsimport.py?)

Yes, you would need fsimport to create instances of this new object,
instead of File object, but File objects should work on their 'data'
attribute, I can't explain why it doesn't.

> or
> 
> b) use a conventional search engine
> 
> I guess b) sounds easier :-)

Not after I fix whatever bug is causing this...  It's a short hill up,
but it's a long ride down. ;)
 
Obviously, one of the big problems here is documentation, as you
mention.  There is little to no ZCatalog docs, other than the managment
interface, and such a complex beast should have more documentation.  Bad
me!

-Michel