[ZPT] re: Please make "total" lazy!

[Steve Alexander]

Chris Withers wrote:
> Steve Alexander wrote:
> 
>># IMPORTANT NOTE
>>#
>># Getting the length of a BTree, TreeSet, or output of keys,
>># values, or items of same is expensive. If you need to get the
>># length, you need to maintain this separately.
> 
> 
> But why would the length of a ZCatalog results set need to be calculated in this way?

Why do you think that ZTUtils batching is only used with ZCatalog?


> PS: If it does need to calculate it in this way, I think that's a bug in the BTrees
> implementation. The users shouldn't have to care about how the length is stored or
> computed, the BTrees internally should keep a seperate counter ifthat's what they need to
> do, then it can be kept in synch in a much tigher loop..

That's not a bug in the BTrees implementation. It is one of the central 
trade-offs when using that particular abstract data type. It is a design 
decision.

Getting the length is expensive for certain data-types. It is still 
possible, but it is expensive. As I have pointed out, this is clearly 
documented in the BTrees package,

Your latest cvs commit causes the length to be calculated for every 
batch, whether it is needed or not. This is a bug. Please fix it so that 
the length is only calculated when required, or back out your changes.

--
Steve Alexander