[Zope] Why can't I add lots of objects at once?

[Chris Withers]

"Kyler B. Laird" wrote:
> 
> A month or two ago I asked here for advice on putting
> roster data (~233K entries/semester) in our database.
> One of the suggestions was to use XML-RPC in order to
> avoid the timeouts I was getting.
> 
> I've been working on an XML-RPC solution and it is
> quite adequate for small data sets, but if I try to
> add a few hundred student/class entires, it hangs.
> 

<snip>

> restart it.  Then I can run the script again and it
> will get a little bit further (because it doesn't
> have to create the existing objects, I assume).
> 
> Of course, I'd prefer not to have to break this up
> into one thousand chunks with server restarts in
> between each one.

<stab type="my, it's dark in here ;-)">
This might be write conflicts in the ZODB. Is your XML-RPC process threaded? If
so, make it not.
If not, then it really is too dark in here for me to stab
</stab>

In any case, I'd recommend using BTree folders instead of nromal ones,
especially with that number of objects.
See if you can persuade Shane to make a BTree folder that uses the new 2.3.1
BTrees, which are also supposed to make this osrt of thing better.

Apart from that, I'm outta ideas :-S

cheers,

Chris