Dieter Maurer <dieter@handshake.de> on 5/11/04 wrote:

george donnelly wrote at 2004-5-10 23:14 -0500:

...

i am trying to pack a zeo-served filestorage, i brought it up as read-only, both the zeo server and a client and packed. after about 20 mins it failed with this error below. can anyone make sense of this error for me? and perhaps suggest what i might do to get past it and pack my zeo-served zodb? ... File "/usr/local/zope/270/lib/python/ZODB/referencesf.py", line 38, in referencesf raise ValueError, 'Error unpickling, %s' % p ValueError: Error unpickling, ((U^NBTrees.OIBTreeq^AU^GOIBTreeq^Btq^CNt.(((U^H^@^@^@^@^@#\x86\xe1q^D ...

I fear this is a difficult one.

Apparently, your "BTrees.OIBTree.OIBTree" contains something which is not unpicklable.

To find out what is the real problem, you should log the traceback in "ZODB/referencesf.py" near line 33 (at the start of the "except:").

Python 2.3 also comes with a pickle utility. It is useful to analyse pickles (and find out what might be wrong with them).

thanks dieter. appreciate very much your suggestions. i think i found the object(s) causing the problem, tried deleting them but of course they are still there and i can't pack them out. any consultants out there interested/available in assisting us with this one? <--> george donnelly ~ http://www.zettai.net/ ~ "Quality Zope Hosting" Shared and Dedicated Zope Hosting ~ Zope Servers ~ Zope Websites Yahoo, AIM: zettainet ~ ICQ: 51907738 ~ Sales (USA): 1-866-967-3669

On Wed, May 12, 2004 at 08:41:14AM +0000, Felix Ulrich-Oltean wrote:

george donnelly <list <at> zettai.net> writes:

...

Dieter Maurer <dieter <at> handshake.de> on 5/11/04 wrote:

...

george donnelly wrote at 2004-5-10 23:14 -0500:

...

i am trying to pack a zeo-served filestorage, i brought it up as read-only, both the zeo server and a client and packed. after about 20 mins it failed with this error below. can anyone make sense of this error for me? and perhaps suggest what i might do to get past it and pack my zeo-served zodb? ... I fear this is a difficult one.

i think i found the object(s) causing the problem, tried deleting them but of course they are still there and i can't pack them out.

I've just experienced the same problem. I haven't yet identified which object(s) are causing trouble, but was wondering about truncating the Data.fs - I've heard talk in the past that you can just chop it and it should still pick itself up. Can anyone confirm this, or give further advice? As George said, if the bad objects are in the past, you can't delete them as usual, can you?

Have you guys seen this? Not sure if it helps... http://www.zopelabs.com/cookbook/1054240694 -- Paul Winkler http://www.slinkp.com

Paul Winkler <pw_lists@slinkp.com> writes: Paul Winkler <pw_lists <at> slinkp.com> writes:

On Wed, May 12, 2004 at 08:41:14AM +0000, Felix Ulrich-Oltean wrote:

...

I've just experienced the same problem. I haven't yet identified which object(s) are causing trouble, ....... Have you guys seen this? Not sure if it helps... http://www.zopelabs.com/cookbook/1054240694

I've had a look at that recipe and I've tried to use fsrefs.py, as well as checkbtrees.py (from the ZODB tools), but both scripts fail. When trying checkbtrees.py, I get: $ python checkbtrees.py Data.fs.bak-20040513 <root> {'Application': <Application instance at 403ac890>, 'ZGlobals': <BTree object at 0x403b5e20>} ['Application'] <Application instance at 403ac890> Traceback (most recent call last): File "checkbtrees.py", line 121, in ? main(fname) File "checkbtrees.py", line 109, in main add_if_new_persistent(todo, v, newpath) File "checkbtrees.py", line 27, in add_if_new_persistent getattr(obj, '_', None) # unghostify File "/web/zope-2.7.0/lib/python/ZODB/Connection.py", line 564, in setstate self._set_ghost_state(obj, p) File "/web/zope-2.7.0/lib/python/ZODB/Connection.py", line 603, in _set_ghost_state state = unpickler.load() File "/web/zope-2.7.0/lib/python/ZODB/Connection.py", line 200, in _persistent_load return self[oid] File "/web/zope-2.7.0/lib/python/ZODB/Connection.py", line 162, in __getitem__ klass=self._db._classFactory(self, module, name) File "/web/zope-2.7.0/lib/python/ZODB/DB.py", line 127, in _classFactory return getattr(__import__(location, _globals, _globals, _silly), ImportError: No module named CMFPlone.Portal I have tried to export a PYTHONPATH that includes the relevant Products directory, but it still doesn't work. When I try fsrefs.py, I get: python fsrefs.py Data.fs.bak-20040513 Traceback (most recent call last): File "fsrefs.py", line 107, in ? main(path) File "fsrefs.py", line 81, in main refs = get_refs(data) File "fsrefs.py", line 42, in get_refs u.noload() EOFError I can't seem to get past the first hurdle, i.e. finding the dodgy object. Any help greatly appreciated. I've appended my original packing traceback at the end of the post. Thanks, Felix. $ ./packstorage.py Traceback (most recent call last): File "./packstorage.py", line 9, in ? cs.pack(wait=1, days=2.0) File "/web/zope-2.7.0/lib/python/ZEO/ClientStorage.py", line 815, in pack return self._server.pack(t, wait) File "/web/zope-2.7.0/lib/python/ZEO/ServerStub.py", line 79, in pack self.rpc.call('pack', t, wait) File "/web/zope-2.7.0/lib/python/ZEO/zrpc/connection.py", line 372, in call raise inst # error raised by server ValueError: Error unpickling, (( GOBBLEDEEHOOK SPAT OUT HERE ))

Dieter Maurer <dieter@handshake.de> writes:

Felix Ulrich-Oltean wrote at 2004-5-13 17:23 +0100:

...

$ ./packstorage.py Traceback (most recent call last): File "./packstorage.py", line 9, in ? cs.pack(wait=1, days=2.0) File "/web/zope-2.7.0/lib/python/ZEO/ClientStorage.py", line 815, in pack return self._server.pack(t, wait) File "/web/zope-2.7.0/lib/python/ZEO/ServerStub.py", line 79, in pack self.rpc.call('pack', t, wait) File "/web/zope-2.7.0/lib/python/ZEO/zrpc/connection.py", line 372, in call raise inst # error raised by server ValueError: Error unpickling, (( GOBBLEDEEHOOK SPAT OUT HERE ))

Thus, this exception occured in the ZEO server.

Copy your "Data.fs" and pack the copy directly (open it as a "FileStorage" and call "DB.pack(...)") in an interactive interpreter. Use "pdb.pm()" ("postmortem") to analyse the state that laid to this error. You will learn at least the class and the oid of the broken object.

Probably, you will also see the parent. This is your handle to delete the broken object.

Dieter Thanks very much for your pointers. I have copied the Data.fs, and packed it in an interactive python session, with: fs = FileStorage('Data.fs.bak-20040512') db = ZODB.DB(fs) db.pack(days = 14.0) The stack trace I get has unreadable stuff towards the end:

...

...

db.pack(days = 14.0) Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/local/zope/lib/python/ZODB/DB.py", line 528, in pack try: self._storage.pack(t,referencesf) File "/usr/local/zope/lib/python/ZODB/FileStorage.py", line 1557, in pack opos = p.pack() File "/usr/local/zope/lib/python/ZODB/fspack.py", line 698, in pack self.gc.findReachable() File "/usr/local/zope/lib/python/ZODB/fspack.py", line 454, in findReachable self.findReachableAtPacktime([z64]) File "/usr/local/zope/lib/python/ZODB/fspack.py", line 529, in findReachableAtPacktime todo.extend(self.findrefs(pos)) File "/usr/local/zope/lib/python/ZODB/fspack.py", line 602, in findrefs return referencesf(self._file.read(dh.plen)) File "/usr/local/zope/lib/python/ZODB/referencesf.py", line 38, in referencesf raise ValueError, 'Error unpickling, %s' % p ValueError: Error unpickling, ((UBT%G⎼␊␊⎽%@.IOBT%G⎼␊␊%@

─IOB┤デ  ダ┐デ  ダ├─├─N├.((J ý ┴ ýUOý ýýZýý ýý?ýýZýýý ýýý\ý"ýVý%G⎼%@ýý@ýýý#ý%G⎼%@Zý\ýVýýýAýýýýýBýýCýýý ▒  ─J ý ┴ ýUOý!ýýZýý!ýý?ýýZýýý!ýýý\ý"ýVý%G⎼%@ýý@ýýý#ý%G⎼%@Zý\ýVýýýAýýýBýýCýýý ▒  ─J ý ┴ ýUOý$ýýZýý$ýý?ýýZýýý$ýýý\ý"ýVý%G⎼%@ýý@ýýý#ý%G⎼%@Zý\ýVýýýAýýýýýBýýCýýý ▒  ─J ý ┴ ýU*ýýýýý·ý I've then run pdb.pm() as you suggested, but I'm not sure how to proceed from here. I've got the stack trace again, this time with the w(here) command in pdb, and I get different nonsense at the end: (Pdb) w <stdin>(1)?() /usr/local/zope/lib/python/ZODB/DB.py(531)pack() -> raise /usr/local/zope/lib/python/ZODB/FileStorage.py(1587)pack() -> self._lock_release() /usr/local/zope/lib/python/ZODB/fspack.py(698)pack() -> self.gc.findReachable() /usr/local/zope/lib/python/ZODB/fspack.py(454)findReachable() -> self.findReachableAtPacktime([z64]) /usr/local/zope/lib/python/ZODB/fspack.py(529)findReachableAtPacktime() -> todo.extend(self.findrefs(pos)) /usr/local/zope/lib/python/ZODB/fspack.py(602)findrefs() -> return referencesf(self._file.read(dh.plen))

/usr/local/zope/lib/python/ZODB/referencesf.py(38)referencesf() -> raise ValueError, 'Error unpickling, %s' % p

I tried printing the value of p and got: (Pdb) p p '((U\x0eBTrees.IOBTreeq\x01U\x08IOBucketq\x02tq\x03Nt.((J\xb7v\x15\xecUO\xc1 \xaa\x8eZ\x85\xc1 \xaa\x95?\x85\x8eZ\x85\xaf\xc1 \x84\x85\x8a\\\x95"\x93V\x81r\x89\x95@\x8a\xb7\x81#\x81r\xc4\x8cZ\x8a\\\x93V\x89\xa7\x95A\x82\x8f\x84\x85\x82\xc4\x95B\x82\x95C\x8a\x8b\x83a\x95D\x8a\x93\x94\x8a~\x95E\xd3\x8a\x8d+\xcc\x84\x85q\x04J\xb9v\x15\xecUO\xc1!\xaa\x8eZ\x85\xc1!\xaa\x95?\x85\x8eZ\x85\xaf\xc1!\x84\x85\x8a\\\x95"\x93V\x81r\x89\x95@\x8a\xb7\x81#\x81r\xc4\x8cZ\x8a\\\x93V\x89\xa7\x95A\x82\x8f\x84\x85\x82\xc4\x95B\x82\x95C\x8a\x8b\x83a\x95D\x8a\x93\x94\x8a~\x95E\xd3\x8a\x8d+\xcc\x84\x85q\x05J\xbbv\x15\xecUO\xc1$\xaa\x8eZ\x85\xc1$\xaa\x95?\x85\x8eZ\x85\xaf\xc1$\x84\x85\x8a\\\x95"\x93V\x81r\x89\x95@\x8a\xb7\x81#\x81r\xc4\x8cZ\x8a\\\x93V\x89\xa7\x95A\x82\x8f\x84\x85\x82\xc4\x95B\x82\x95C\x8a\x8b\x83a\x95D\x8a\x93\x94\x8a~\x95E\xd3\x8a\x8d+\xcc\x84\x85q\x06J\xbcv\x15\xecU*\x86\x13\x84\xff\x86\x8a~\x83' What is this stuff? How can I get the offending object and parent from this? Thanks, Felix.