[Grok-dev] Re: Performance of OrderedContainer

[Sebastian Ware]

I was thinking something like a Btree holding PersistentLists which  
would act as buckets of (say) 1000 keys each and an optional startAt=  
and limit= argument at the appropriate places. Maybe even such that  
the first 1000 keys are stored the way it is done now, and the Btree  
is added when the container grows above the first bucket limit.

Mvh Sebastian

17 jun 2008 kl. 14.27 skrev Martijn Faassen:

> Sebastian Ware wrote:
>> I was looking at the trunk of grok because I need to use  
>> OrderedContainer. But, if I add say 1 million objects to an  
>> OrderedContainer, won't it become very slow due to the  
>> implementation with PersistentList? Say that I want to show the  
>> last 100 of the objects, I would have to return all 1 million keys.
>
> Hm, thanks for that review, I think you are correct. Once the  
> million keys are all in memory it'll be okay, but loading all those  
> million keys into memory from the ZODB isn't very efficient.
>
> I think it should use a BTree of some kind for the order information  
> (_order) to prevent this problem. BTrees are however automatically  
> sorted, so the design isn't obvious. The original OrderedContainer  
> in Zope 3 seems to also use PersistentList, so that code is no help  
> here.
>
> Ideas, patches welcome. :)
>
> Regards,
>
> Martijn
>
> _______________________________________________
> Grok-dev mailing list
> Grok-dev at zope.org
> http://mail.zope.org/mailman/listinfo/grok-dev