[ZODB-Dev] High Write Applications

[Casey Duncan]

I had a thought a while ago about a seemingly straightforward way to improve 
write performance for FileStorage. It was inspired by bsddb which allows 
something similar.

You would divide the filestorage into two files, a read-only file for past 
transactions and a write file for new transactions. At first, you would have 
only the write file. When you pack, the packed transactions would go into the 
read file and a new write file would be started. The index could keep track 
of the file boundary so that, given an oid, the storage could figure out 
which file the current data resides in. These files would be put on different 
drive spindles to improve performance.

The thing that might be complicated is undo, but assuming the read file length 
doesn't need to change, except at pack time, it should be relatively easy to 
figure out whether the data resides in one file or the other.

-Casey