[Zope-dev] limit to number of operations in a transaction?

[Jens Vagelpohl]

in your script you must commit a transaction once in a while with 
"get_transaction().commit()".

when you use xml-rpc then every call to zope will end up being its own 
transaction and you will not have this problem.

jens

On Sunday, Aug 25, 2002, at 10:12 US/Eastern, Christopher N. Deckard 
wrote:

> Hello,
> So I've run into a problem.  I had the need to migrate our a
> database of people from an old storage format to a new one.  The
> old format uses Folders and Properties to store information about
> a person.  Name, phone, email, address, etc.  The new format uses a
> Product that I wrote which stores said data in a ParsedXML document.
>
> I have roughly 1,500 people in the database.  I wrote a script to
> migrate the people.  Basically it does getProperty for each property
> on each person folder, then creates the new person object which uses
> XML.  Zope apparently cannot handle this number of operations in one
> transaction.  Zope gets slower and slower and eventually becomes on
> responsive.  It looks like the script has completed, but nothing is
> ever committed to the ZODB, and since Zope is unresponsive it must
> be restarted.  This, as expected, kills that entire transaction
> which was never committed.
>
> Is it known that large numbers of operations, such as above, in a
> single transaction can cause problems?  A transaction, of course,
> being a request, and an operation being something like
> manage_addProduct.
>
> I've solved the "problem" by using xmlrpc and for person in people
> calling my migrate_person script for only one person at a time.
> This is SO MUCH FASTER.  I previously ran the script that migrates
> all of the people, and after 8 hours it still had not completed.
>
> Thoughts?
>
> -Chris
>