[Zope3-dev] Zope 3 Alpha 1 in 2002

[Jim Fulton]

Chris Withers wrote:
> Jim Fulton wrote:
> 
>> I've learned that premature optimization is extremely silly.
> 
> 
> fair enough :-)
> 
>> The ZCatalog object cache stores quite a bit of data in each database
>> record.
> 
> 
> I thought metadata was just one tuple?

Tuples aren't persistent.  The persistent object is a BTree bucket
that contains a bunch of tuples. I'd be willing to bet $.05 that
these are among the larger objects in the database.


>>> I like the way Lucene does this. When you specify an attribute name, 
>>> you specify if it's indexed, if it's store and if it's fed through a 
>>> lexicon/splitter.
>>
>>
>> I think that this is a little different because Lucene doesn't store the
>> original data.
> 
> 
> Yes it does, if you set 'store' for an attribute name it stores the 
> original data it indexed in a very similar way to ZCatalog's metadata.

What I meant was that aside from this feature, Lucene doesn't store the original
data. That is, for Lucene, the storage is not redundant. For Zope, the data stored in
the catalog is redundant.


>>> Now, here I'd say we want to specify for each name:
>>>
>>> 1. What type of index to use
>>>
>>> 2. Whether or not to store/cache the value
>>>
>>> 3. Maybe a chain of processors to pre-process the attribute value 
>>> before indexing. I'm thinking lexicons, splitters, filters and the 
>>> like here.
>>>
>>> That way, the whole kaboodle is all part of the Indexing Service.
>>
>>
>> You may create different index on the same name or use multiple names for
>> a single index, so this shouldn't be name centric.
> 
> 
> I agree with your point, and maybe I'm talking more from a UI point of 
> view. I have an attribute name, I want to know what indexes will apply 
> to that name, and whether or not its value will be cached. The chains of 
> pre-processors depend on the index.
> 
> That said, I really can't think of a _good_ use case for an index that 
> processes multiple names. YAGNI?

Not at all. Imagine a text index that indexes title and descriotion together.
Or imagine an index that indexes hierarchical relationships extressed as
classification data (e.g. classifications for newspaper classified ads).  The
product from which ZCatalog was derived had this feature and it's omission in
ZCatalog has been rather annoying.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (888) 344-4332            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org