"can't pickle instancemethod objects" in testcase
My code works fine in normal running zope but not when run as a testcase (I'm not using bin/zopectl test if that matters) I get the following error. I feel guilty since I've got this line in my code: transaction.get().commit() (Expense.py#4458) I remember having come across this error before but can't remember what the solution was. ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/peterbe/zope/zope285/lib/python/Testing/ZopeTestCase/profiler.py", line 98, in __call__ testMethod() File "testExpenses2.py", line 81, in test_uploadPhotos eids = context.uploadExpenseFiles(files) File "/home/peterbe/zope/zope285/Products/MExpenses/Expense.py", line 4458, in uploadExpenseFiles transaction.get().commit() File "lib/python/transaction/_transaction.py", line 381, in commit File "lib/python/transaction/_transaction.py", line 379, in commit File "lib/python/transaction/_transaction.py", line 424, in _commitResources File "/home/peterbe/zope/zope285/lib/python/ZODB/Connection.py", line 462, in commit self._commit(transaction) File "/home/peterbe/zope/zope285/lib/python/ZODB/Connection.py", line 503, in _commit self._store_objects(ObjectWriter(obj), transaction) File "/home/peterbe/zope/zope285/lib/python/ZODB/Connection.py", line 525, in _store_objects p = writer.serialize(obj) # This calls __getstate__ of obj File "/home/peterbe/zope/zope285/lib/python/ZODB/serialize.py", line 330, in serialize return self._dump(meta, obj.__getstate__()) File "/home/peterbe/zope/zope285/lib/python/ZODB/serialize.py", line 339, in _dump self._p.dump(state) File "copy_reg.py", line 69, in _reduce_ex raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle instancemethod objects ---------------------------------------------------------------------- -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com
--On 7. Januar 2008 18:29:50 +0000 Peter Bengtsson <peter@fry-it.com> wrote:
My code works fine in normal running zope but not when run as a testcase (I'm not using bin/zopectl test if that matters) I get the following error. I feel guilty since I've got this line in my code: transaction.get().commit() (Expense.py#4458)
I remember having come across this error before but can't remember what the solution was.
---------------------------------------------------------------------- Traceback (most recent call last): File "/home/peterbe/zope/zope285/lib/python/Testing/ZopeTestCase/profiler.py", line 98, in __call__ testMethod() File "testExpenses2.py", line 81, in test_uploadPhotos eids = context.uploadExpenseFiles(files) File "/home/peterbe/zope/zope285/Products/MExpenses/Expense.py", line 4458, in uploadExpenseFiles transaction.get().commit() File "lib/python/transaction/_transaction.py", line 381, in commit File "lib/python/transaction/_transaction.py", line 379, in commit File "lib/python/transaction/_transaction.py", line 424, in _commitResources File "/home/peterbe/zope/zope285/lib/python/ZODB/Connection.py", line 462, in commit self._commit(transaction) File "/home/peterbe/zope/zope285/lib/python/ZODB/Connection.py", line 503, in _commit self._store_objects(ObjectWriter(obj), transaction) File "/home/peterbe/zope/zope285/lib/python/ZODB/Connection.py", line 525, in _store_objects p = writer.serialize(obj) # This calls __getstate__ of obj File "/home/peterbe/zope/zope285/lib/python/ZODB/serialize.py", line 330, in serialize return self._dump(meta, obj.__getstate__()) File "/home/peterbe/zope/zope285/lib/python/ZODB/serialize.py", line 339, in _dump self._p.dump(state) File "copy_reg.py", line 69, in _reduce_ex raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle instancemethod objects
Well, this error is self-speaking :-) The standard solution for figuring out the related property/attr keeping the instancemethod is: pdb. Look at the object or its __dict__ and check what is inside or think about your code where you might be assigning a method somewhere. -aj
Andreas Jung wrote:
--On 7. Januar 2008 18:29:50 +0000 Peter Bengtsson <peter@fry-it.com> wrote:
My code works fine in normal running zope but not when run as a testcase (I'm not using bin/zopectl test if that matters) I get the following error. I feel guilty since I've got this line in my code: transaction.get().commit() (Expense.py#4458)
I remember having come across this error before but can't remember what the solution was.
---------------------------------------------------------------------- Traceback (most recent call last): File "/home/peterbe/zope/zope285/lib/python/Testing/ZopeTestCase/profiler.py", line 98, in __call__ testMethod() File "testExpenses2.py", line 81, in test_uploadPhotos eids = context.uploadExpenseFiles(files) File "/home/peterbe/zope/zope285/Products/MExpenses/Expense.py", line 4458, in uploadExpenseFiles transaction.get().commit() File "lib/python/transaction/_transaction.py", line 381, in commit File "lib/python/transaction/_transaction.py", line 379, in commit File "lib/python/transaction/_transaction.py", line 424, in _commitResources File "/home/peterbe/zope/zope285/lib/python/ZODB/Connection.py", line 462, in commit self._commit(transaction) File "/home/peterbe/zope/zope285/lib/python/ZODB/Connection.py", line 503, in _commit self._store_objects(ObjectWriter(obj), transaction) File "/home/peterbe/zope/zope285/lib/python/ZODB/Connection.py", line 525, in _store_objects p = writer.serialize(obj) # This calls __getstate__ of obj File "/home/peterbe/zope/zope285/lib/python/ZODB/serialize.py", line 330, in serialize return self._dump(meta, obj.__getstate__()) File "/home/peterbe/zope/zope285/lib/python/ZODB/serialize.py", line 339, in _dump self._p.dump(state) File "copy_reg.py", line 69, in _reduce_ex raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle instancemethod objects
Well, this error is self-speaking :-)
The standard solution for figuring out the related property/attr keeping the instancemethod is: pdb. Look at the object or its __dict__ and check what is inside or think about your code where you might be assigning a method somewhere.
Damn it! Not the answer I was hoping for. I was hoping for something simpler and more obvious :) What object do you suggest I inspect in the debugger then? I don't know how the ZODB figures out what it needs to save. -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com
--On 7. Januar 2008 19:17:06 +0000 Peter Bengtsson <peter@fry-it.com> wrote:
Damn it! Not the answer I was hoping for. I was hoping for something simpler and more obvious :)
What object do you suggest I inspect in the debugger then? I don't know how the ZODB figures out what it needs to save.
That's trivial start your debugger in ZODB.serialize._dump() and check the 'state' parameter (should be a dict). Using pprint.pprint(state) will make the output less painful...just check the output for something with instancemethod or something similar smelly. -aj
Peter Bengtsson wrote at 2008-1-7 18:29 +0000:
My code works fine in normal running zope but not when run as a testcase (I'm not using bin/zopectl test if that matters) I get the following error. I feel guilty since I've got this line in my code: transaction.get().commit() (Expense.py#4458)
I remember having come across this error before but can't remember what the solution was. ... File "/home/peterbe/zope/zope285/lib/python/ZODB/serialize.py", line 339, in _dump self._p.dump(state) File "copy_reg.py", line 69, in _reduce_ex raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle instancemethod objects
----------------------------------------------------------------------
Apparently, you have an "instancemethod" in your test setup where you do not have one in the normal Zope. An alternative explanation could be that in your normal Zope additional pickle registrations are performed. If you would use, e.g., my "CompiledExecutable" product, then the import of this product would register "code" and "instancemethod" instances as picklable. A test setup may not import "CompiledExecutables" and its pickle registrations would be missing. A side note: Usually, it is a very bad idea to perform a full "commit" in a test. Such "commit"s are incompatible with techniques that share (for efficiency reasons) part of the test setup among several tests. "ZopeTestCase", e.g., makes use of such techniques. -- Dieter
Thanks Andreas and Dieter, The explanation was quite simple, I set an attribute on an object that was a method like this:: class TestBase(ZopeTestCase.ZopeTestCase): def dummy_redirect(self, *a, **kw): self.has_redirected = a[0] if kw: print "*** Redirecting to %r + (%s)" % (a[0], kw) else: print "*** Redirecting to %r" % a[0] def afterSetUp(self): self._setupDBConnection() dispatcher = self.folder.manage_addProduct['MExpenses'] dispatcher.manage_addHomepage('mexpenses') self.mexpenses = self.folder['mexpenses'] self.mexpenses.http_redirect = self.dummy_redirect # NOTICE!! request = self.app.REQUEST sdm = self.app.session_data_manager request.set('SESSION', sdm.getSessionData()) self.has_redirected = False The reason for the dummy redirector is boring. The reason for the transaction.get().commit() is to not cancel everything when a ConflicrError happens during a very long-lasting file upload. Again, thanks. PS. Andreas, I did use your set_trace() tip on the ZODB code. Isn't Python great! On 08/01/2008, Dieter Maurer <dieter@handshake.de> wrote:
Peter Bengtsson wrote at 2008-1-7 18:29 +0000:
My code works fine in normal running zope but not when run as a testcase (I'm not using bin/zopectl test if that matters) I get the following error. I feel guilty since I've got this line in my code: transaction.get().commit() (Expense.py#4458)
I remember having come across this error before but can't remember what the solution was. ... File "/home/peterbe/zope/zope285/lib/python/ZODB/serialize.py", line 339, in _dump self._p.dump(state) File "copy_reg.py", line 69, in _reduce_ex raise TypeError, "can't pickle %s objects" % base.__name__ TypeError: can't pickle instancemethod objects
----------------------------------------------------------------------
Apparently, you have an "instancemethod" in your test setup where you do not have one in the normal Zope.
An alternative explanation could be that in your normal Zope additional pickle registrations are performed. If you would use, e.g., my "CompiledExecutable" product, then the import of this product would register "code" and "instancemethod" instances as picklable. A test setup may not import "CompiledExecutables" and its pickle registrations would be missing.
A side note: Usually, it is a very bad idea to perform a full "commit" in a test. Such "commit"s are incompatible with techniques that share (for efficiency reasons) part of the test setup among several tests. "ZopeTestCase", e.g., makes use of such techniques.
-- Dieter
-- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com
participants (3)
-
Andreas Jung -
Dieter Maurer -
Peter Bengtsson