[ZODB-Dev] Re: What makes the ZODB slow?

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Chris Withers wrote:
> Florent Guillaume wrote:
>>
>> BTrees perform best when keys' prefixes are randomly distributed.
>> So if your application generates keys like 'foo001', 'foo002',...
>> you'll get lots of conflicts. Same for consecutive integers in IOBTree.
> 
> Tempted to call bullshit on this, since there's code in the catalog to
> specifically assign series of keys...

Note that that code starts from a per-thread random base, and resets the
base.  The following code, from Products.ZCatalog.Catalog's
'updateMetadata' method, explains:

                # New style, get random id

                index=getattr(self, '_v_nextid', 0)
                if index % 4000 == 0:
                    index = randint(-2000000000, 2000000000)
                while not data.insert(index, newDataRecord):
                    index = randint(-2000000000, 2000000000)

                # We want ids to be somewhat random, but there are
                # advantages for having some ids generated
                # sequentially when many catalog updates are done at
                # once, such as when reindexing or bulk indexing.
                # We allocate ids sequentially using a volatile base,
                # so different threads get different bases. This
                # further reduces conflict and reduces churn in
                # here and it result sets when bulk indexing.
                self._v_nextid=index+1


Tres.
- --
===================================================================
Tres Seaver          +1 202-558-7113          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFEoB9R+gerLs4ltQ4RAjIfAJwOudWWE9NWPWRs5mtEg9s9AR+B+gCgwDDi
/whz9qepq4uTtx24jddlaG8=
=d5Ec
-----END PGP SIGNATURE-----