Problems adding many objects
Hi, I've created a script that imports XML into the ZODB, using regular TTW methods - and all is fine, except for when I import large files, with many objects. After about 288 objects, the import fails when the method that adds an object in the ZODB returns None, instead of the object it added. From what I can tell, the added object is not None before it is returned, but the name object, set via the expression object = self.manage_add_issue(...) is None. So it looks like somewhere between 'return issue_' in manage_add_issue and the expression above, the object is lost. Any ideas what could be happening here? I've managed to work around it for now, but it is interesting nonetheless. Regards, Morten
Hi Morten, You'd probably need to show us the implementation of whatever "manage_add_issue" is. - C On Sep 25, 2005, at 10:09 AM, Morten W. Petersen wrote:
Hi,
I've created a script that imports XML into the ZODB, using regular TTW methods - and all is fine, except for when I import large files, with many objects.
After about 288 objects, the import fails when the method that adds an object in the ZODB returns None, instead of the object it added. From what I can tell, the added object is not None before it is returned, but the name object, set via the expression
object = self.manage_add_issue(...)
is None. So it looks like somewhere between 'return issue_' in manage_add_issue and the expression above, the object is lost.
Any ideas what could be happening here? I've managed to work around it for now, but it is interesting nonetheless.
Regards,
Morten
<morten.vcf> _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Chris, the implementation looks like this def manage_add_issue(self, id=None, title='', contents='', state='', type='', referrer='', format=None, REQUEST=None): """Add an Issue.""" if id is None: id = self.get_unique_id() issue_ = issue(id, title, contents, state, type, creator=self.get_user().get_id(), owner=self.get_user().get_id(), format=format or self.get_user_preferences().issue_format) self._setObject(id, issue_) self.order.append(issue_.id) self.order = self.order issue_ = self[id] issue_.version = self.get_issue_dealer().version try: self.index_object() except AttributeError: pass if referrer and REQUEST is not None: REQUEST.RESPONSE.redirect(referrer) print 'redirect' else: if REQUEST is not None: if self.inCMF(): print 'admin url' return self.get_admin_url() else: print 'manage_main' return self.manage_main(self, REQUEST) else: print 'returning issue', issue_ return issue_ and before returning none, it says 'returning issue' on stdout. -Morten Chris McDonough wrote:
Hi Morten,
You'd probably need to show us the implementation of whatever "manage_add_issue" is.
- C
On Sep 25, 2005, at 10:09 AM, Morten W. Petersen wrote:
Hi,
I've created a script that imports XML into the ZODB, using regular TTW methods - and all is fine, except for when I import large files, with many objects.
After about 288 objects, the import fails when the method that adds an object in the ZODB returns None, instead of the object it added. From what I can tell, the added object is not None before it is returned, but the name object, set via the expression
object = self.manage_add_issue(...)
is None. So it looks like somewhere between 'return issue_' in manage_add_issue and the expression above, the object is lost.
Any ideas what could be happening here? I've managed to work around it for now, but it is interesting nonetheless.
Regards,
Morten
<morten.vcf> _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
-- Regards, Morten Tlf: +47 45 44 00 69 Blog: http://www.blogologue.com
On Sep 26, 2005, at 7:21 AM, Morten W. Petersen wrote:
Chris,
the implementation looks like this
def manage_add_issue(self, id=None, title='', contents='', state='', type='', referrer='', format=None, REQUEST=None): """Add an Issue.""" if id is None: id = self.get_unique_id() issue_ = issue(id, title, contents, state, type, creator=self.get_user().get_id(), owner=self.get_user().get_id(), format=format or self.get_user_preferences().issue_format) self._setObject(id, issue_) self.order.append(issue_.id) self.order = self.order issue_ = self[id] issue_.version = self.get_issue_dealer().version try: self.index_object() except AttributeError: pass if referrer and REQUEST is not None: REQUEST.RESPONSE.redirect(referrer) print 'redirect' else: if REQUEST is not None: if self.inCMF(): print 'admin url' return self.get_admin_url() else: print 'manage_main' return self.manage_main(self, REQUEST) else: print 'returning issue', issue_ return issue_
and before returning none, it says 'returning issue' on stdout.
It doesn't say "returning issue, None"? My guess is that the if request is not none branch is called and it goes that way. But there's no way to know for sure without running it. This is a good place to use the Python debugger... as the first line of this method, insert "import pdb; pdb.set_trace()", then start up Zope in the foreground (runzope or zopectl fg), then cause the method to be invoked through a browser. On the console you will see a pdb prompt. You can step through the code by using "n". To print the value of a variable type "p <variablename>", to step in to a function call press "s". When you're finished, press "c" to continue or "q" to quit. This will be the fastest way to see what's happening in this method. - C
-Morten
Chris McDonough wrote:
Hi Morten,
You'd probably need to show us the implementation of whatever "manage_add_issue" is.
- C
On Sep 25, 2005, at 10:09 AM, Morten W. Petersen wrote:
Hi,
I've created a script that imports XML into the ZODB, using regular TTW methods - and all is fine, except for when I import large files, with many objects.
After about 288 objects, the import fails when the method that adds an object in the ZODB returns None, instead of the object it added. From what I can tell, the added object is not None before it is returned, but the name object, set via the expression
object = self.manage_add_issue(...)
is None. So it looks like somewhere between 'return issue_' in manage_add_issue and the expression above, the object is lost.
Any ideas what could be happening here? I've managed to work around it for now, but it is interesting nonetheless.
Regards,
Morten
<morten.vcf> _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
-- Regards,
Morten
Tlf: +47 45 44 00 69 Blog: http://www.blogologue.com
<morten.vcf>
participants (2)
-
Chris McDonough -
Morten W. Petersen