[ZODB-Dev] RelStorage branch 1.4.0-fastimport

[David Blewett]

On Thu, Jan 6, 2011 at 11:51 PM, Shane Hathaway <shane at hathawaymix.org> wrote:
>> I am no zodb or relstorage expert, but the changes for the
>> --single-transaction code look good to me.
>
> I looked at that branch before, but I felt like the changes were
> complicated enough to require a comparison with simpler solutions first.
>  In particular, Postgres has an option to disable fsync.  Set it in
> postgresql.conf.  Disabling fsync is not normally recommended, but for a
> large import it's obviously a good idea.  Would you compare the speed of
> --single-transaction with disabled fsync on vanilla RelStorage?

Sorry, it's been awhile since I started work on that branch. Disabling
fsync is an option for the load, and it should help. However, using a
single transaction is much faster in PostgreSQL. Off the top of my
head, I can't recall if the branch also uses COPY mode. I recall
encountering some problems trying to get COPY to work correctly.
Single-transaction mode + COPY, however, is the fastest way to get
data into PostgreSQL.

David Blewett