[Zope] searching and serving large textfiles ~120 Mb

[Dieter Maurer]

Small Business Services wrote at 2003-12-8 08:19 -0500:
> ...
>> Something seems to be strange with your setup.
>>
>>   In general, it is no problem to load a few 10k objects from
>>   the ZODB (this is different when you load hundreds or thousands).
>>
>>   Putting the text in your MetaData causes huge MetaData blocks.
>>   With your avarage 10k content, you get 300k to 600k
>>   MetaData blocks (each block contains metadata records
>>   for up to 60 objects).
>
>When we used getitem to access the fulltext field of the target document our
>average response time (using Call Profiler) was in the range of 5-7 seconds
>(we actually had to call getitem on 10 different documents for each search
>result set).  When we moved the fulltext data to  metadata the average
>response time dropped to 1-1.5 seconds.
>
>There were no other differences between the tests, so we concluded that
>doing a getitem call to the actual document was very expensive.

I recently measured ZEO interaction (on a quite fast computer
with ZEO client and server on the same machine).
Usually, a ZEO load took about 1 to 3 ms for small
to medium size objects. When you access an object from a
catalog search, you probably load more than on object (all
objects on the path must be loaded, unless they are cached).
Nevertheless, I have difficulties to understand the 5-7 seconds.

-- 
Dieter