[Zope] Using ZCatalog for Outside Content

[sean.upton@uniontrib.com]

Haven't done this, but I've got a good idea how you could...  Remember that
Catalog indexes objects, which means methods or properties, which means, if
you can grab remote content via a method, you can index it; just keep in
mind you would need a one-to-one relationship between a local (proxy) object
and a remote page.

You need to come up with an API and methods framework for spidering your
content via http, using a python http client.  For each 'page' you will need
to create a CatalogAware proxy object in the object database that stores
metadata such as the URL (put these proxy objects in a container capable of
having lots of objects, like a BTreeFolder).  Then you use Catalog to index
methods (which really get remote text content) using a text index.
Consider, if you have the ability, to cache as much useful metadata in these
proxy objects about those pages.  I am doing something similar with
relational data being indexed with ZCatalog; in the case of data in an
relational datastore, I have to have a proxy-object-per-record; in your
case, you would need a proxy-object-per-page.

Sean

-----Original Message-----
From: abg@comco-inc.com [mailto:abg@comco-inc.com]
Sent: Tuesday, November 06, 2001 7:01 AM
To: zope@zope.org
Subject: [Zope] Using ZCatalog for Outside Content


I'd like to use ZCatalog to catalog websites stored outside of Zope. I have
seen a few passing references to this being a feature of ZCatalog, but no
documentation on the subject.

Has anyone used ZCatalog in this manner? Any tips on how to accomplish this?

Thanks in advance,

Aaron Gillette

_______________________________________________
Zope maillist  -  Zope@zope.org
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )