[ZODB-Dev] ZODB open/close profilings
Jeremy Hylton
jeremy@zope.com (Jeremy Hylton)
Tue, 10 Dec 2002 14:17:15 -0500
Thanks for doing the leg work, Greg!
>>>>> "GW" == Greg Ward <gward@mems-exchange.org> writes:
GW> Executive summary:
GW> * the presence or absence of an index file makes very little
GW> difference: 49 sec to open without index, 48 sec to open
GW> with
GW> * it might be worth rewriting the OID packing/unpacking
GW> functions in C; opening the database calls U64() 537,404
GW> times, accounting for ~12 sec of the ~48 sec total open time
In a ZODB4 branch, I'm experimenting with generous use of the "Q"
struct format code, which does with U64() does.
GW> * but the real culprit is FileStorage._sane(), which takes 28
GW> sec of the ~48 sec total opening time
I've never thought about the cost of _sane(), so this is a great
clue. I think part of the problem is that you don't use undo() enough
<wink>. The code scans the entire data.fs backwards, looking for a
transaction marked with 'u' for undo. Since you never do an undo, it
reads the whole file backwards after reading it forwards. Yikes.
Jeremy