zc.dict.OrderedDict efficiency
The documentation of OrderedDict from zc.dict 1.2.1 states that the current implementation is inefficient for large collections because it uses a PersistentList to store the order. It also says that a BList which would be preferrable is not used as it is not yet released. - What's the state of those BLists? Are they just around the corner or would it be worthwhile to consider some interim solution to the efficiency issue? - If the latter, what about storing the order by * introducing internal integer keys incremented for each newly added item and * mapping from integers to keys and values in one BTree and from keys to integers and values in a second (or using more BTrees to avoid storing those pairs)? This introduces somewhat more overhead in general but prevents the bad behaviour for large collections. Would that be an acceptable trade-off? -- Thomas
On Thu, Sep 18, 2008 at 3:52 PM, Thomas Lotze <thomas@thomas-lotze.de> wrote:
The documentation of OrderedDict from zc.dict 1.2.1 states that the current implementation is inefficient for large collections because it uses a PersistentList to store the order. It also says that a BList which would be preferrable is not used as it is not yet released.
- What's the state of those BLists? Are they just around the corner or would it be worthwhile to consider some interim solution to the efficiency issue?
I'm not aware of zc.blist's current state, but I think it's more-or-less ready. If not and anyone knows what needs to be done to ready it for a release, you could take that up (the code is in svn.zope.org). -- Benji York Senior Software Engineer Zope Corporation
On Sep 18, 2008, at 4:02 PM, Benji York wrote:
On Thu, Sep 18, 2008 at 3:52 PM, Thomas Lotze <thomas@thomas- lotze.de> wrote:
The documentation of OrderedDict from zc.dict 1.2.1 states that the current implementation is inefficient for large collections because it uses a PersistentList to store the order. It also says that a BList which would be preferrable is not used as it is not yet released.
- What's the state of those BLists? Are they just around the corner or would it be worthwhile to consider some interim solution to the efficiency issue?
I'm not aware of zc.blist's current state, but I think it's more-or- less ready. If not and anyone knows what needs to be done to ready it for a release, you could take that up (the code is in svn.zope.org).
Yup, it's ready. Stress tests run for several hours without bugs, last I worked on it. I had an idea of automatically converting failing stress tests to repeatable tests, but by the time I had the idea, the stress test wasn't failing any more. That was really the only thing holding me back from releasing. To release it responsibly now, someone needs to claim maintainership. Gary
Gary Poster wrote: [zc.blist]
To release it responsibly now, someone needs to claim maintainership.
As I was the one asking in the first place, I'm willing to do this unless there's a policy for zc.* packages to be maintained by ZC people or something similar. -- Thomas
On Sep 23, 2008, at 2:54 AM, Thomas Lotze wrote:
Gary Poster wrote:
[zc.blist]
To release it responsibly now, someone needs to claim maintainership.
As I was the one asking in the first place, I'm willing to do this
Great! Thank you!
unless there's a policy for zc.* packages to be maintained by ZC people or something similar.
I'm pretty sure there is not such a policy, but perhaps a ZC person will disagree. Assuming this works out, I'm happy to help as much as I can: shoot me a note if you need anything. Gary
On Tue, Sep 23, 2008 at 2:54 AM, Thomas Lotze <thomas@thomas-lotze.de> wrote:
Gary Poster wrote:
[zc.blist]
To release it responsibly now, someone needs to claim maintainership.
As I was the one asking in the first place, I'm willing to do this unless there's a policy for zc.* packages to be maintained by ZC people or something similar.
Nope, no policy like that. In fact I've always hoped this day would come. You're making history. ;) -- Benji York Senior Software Engineer Zope Corporation
participants (3)
-
Benji York -
Gary Poster -
Thomas Lotze