[Zope-CMF] cmf zpt_generic/search - sequence_length is very slow for large collections, len () is much faster.

[Dieter Maurer]

Henryk Paluch wrote at 2005-1-3 08:56 +0100:
> ...
>Florent Guillaume wrote:
>
>>>items = ctool.searchResults(kw)
>>>batch_obj = Batch(items, 25, b_start, orphan=1)
>>>length = batch_obj.sequence_length
>>>
>>>### my guess: len(items)==batch_obj.sequence_length
>>>###    - but len(items) is much faster.
>>>    
>>>
>>
>>Well then that's strange, because in the end batch_obj.sequence_length
>>does just call lazily len(items). From reading the code I think there
>>shouldn't be any speed difference. Are you sure about your measurements?
> ...
>  For me, it matters very much for large collections ( > 5000 objects) - 
>the results are reproducible for both BDB and standard FileStorage.

I have seen similar behaviour:

  When I analysed, I detected that although I never have
  specified to use authorization filtering, the complete
  sequence was fetched and checked with an empty authorization filter.
  As in my case fetching an element was very expensive,
  batching became ineffective.

  This was a long time ago and our Zope version immediately lost
  this behaviour.

  I do not know whether modern Zope version still behave like
  this and whether this is your problem as well.
  A profile will tell you (with e.g. ZopeProfiler

    <http://www.dieter.handshake.de/pyprojects/zope>
  )


-- 
Dieter