[ZODB-Dev] Iterating through a B-Tree

[Gfeller Martin]

Dear all,

I thought I let you know a timing surprise I had with iterating through all keys of a B-Tree:

for k in list(conn.root.keys()):
	d = conn.root[k]

is 8 times faster than the same loop without the conversion of the keys to a list, with
conn.root being an OOBTree of length 47000. The timings on a 1 GHz Windows 2000 machine
are 5.8 vs. 46.7 seconds.

The actual difference seems to be in ZODB.FileStorage._load[615], which performs the physical read. My (naive) conjecture is that the list() forces all nodes to be read sequence, whereas without the list, a lot of file seeks are performed.

Of course, iterating through a whole tree is not very common, but when it happens it is good to know that it can be speed up.

Best regards,
Martin Gfeller
_______________________ 

COMIT Gruppe 
Risk Advice Services 
Pflanzschulstrasse 7 
CH-8004 Zürich 

Telefon	+41 (0) 1 298 92 00

http://www.comit.ch 
http://www.quantax.com - Quantax Professional - Downsized Trading and Risk System