On Sat, 2003-05-31 at 13:42, Dieter Maurer wrote:
I do not yet fully understand why, but the complete host seems to be affected by the high disc bandwidth required during packing.
Note that with Berkeley storages, one of the biggest boosts to performance you can get is by separating the BerkeleyDB database and log files to different disks. This is a low-level BerkeleyDB configuration option that's exposed in the storage configuration parameters. FileStorage doesn't have this "luxury".
In the second packing phase (the so called copying phase), the packing thread helds the storage lock most of the time. This can make accessing the ZODB impossible by normal operations, especially because the "lock_release", "lock_acquire" probably does not succeed in giving the lock to a waiting thread. I have a patch that restricts this locking to only a minor fraction of time.
Berkeley storages release the storage lock during the various phases of packing. It's not terribly granular, but it would give opportunities for other work to happen during packing. That, and regular autopacking might make things acceptably responsive. -Barry