[Zope] best way to populate ZODB with ~11,000 folders?

[Andy McKay]

> This means I'll have paths like
> /courses/2001.1/AT/249/03/02
> and that would have
> /courses/2001.1/AT/249/03/02/instructor
> even though the instructor might be acquired
> from a parent folder.

Been there done it as well... ah this brings back memories.

> Today I learned about XML-RPC.  We're hard at
> work trying to get the HTTPS capability
> welded on to it.  (We only use HTTPS for
> Zope.)  It seems that it might be better to
> use this to populate the ZODB.

Yes it is. Either this or do it as an External Method, I wrote one that
populated 100 folders, committed the transaction. And so on. Really when are
doing this sort of this make sure using ZODB is the correct thing. In the
end I decided it wasn't.

> Is this a reasonable way to handle this?  Am
> I going to kill the ZODB?  Will I swamp it
> with transaction logs?  I recall seeing that
> the ZODB isn't so great for writing.  I'll
> be doing a lot of writing.

If you do it all in one go, yes. Do a it a bit at time (by script) and
commit it. At the extreme you may want to include restarting Zope as well
each time. Zope is optimised for reading, but writing a lot once is fine.
Make a seperate transaction log for yourself to monitor your script. Its
continual writing that is a problem.

> We've used Oracle for everything, but I want
> so much to move completely to ZODB so that I
> can use ZEO.  I'm willing to put in some
> extra effort to move down this path.

You can't use a relational DB with ZEO? Thats a problem.

--
  Andy McKay.