[Zope] searching and serving large textfiles ~120 Mb

[Dieter Maurer]

Small Business Services wrote at 2003-12-5 08:31 -0500:
> ...
>We also do a similar thing in that we locate the users search term within
>the record and display the relevant sections of the search results records.
>We found that we had to include the full-text of the record within the
>metadata table (even though the recommended practice is to have a maximum of
>200 bytes in the metadata table) because the time required to access the
>original document was much too long.

Something seems to be strange with your setup.

  In general, it is no problem to load a few 10k objects from
  the ZODB (this is different when you load hundreds or thousands).

  Putting the text in your MetaData causes huge MetaData blocks.
  With your avarage 10k content, you get 300k to 600k
  MetaData blocks (each block contains metadata records
  for up to 60 objects).

-- 
Dieter