POSKeyErrors was Re: [Zope] Zope leaking memory?

[Richard Jones]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, 16 Sep 2004 04:31 pm, Tim Peters wrote:
> [Richard Jones]
>
> > Deleting the index had no impact.
>
> Then, in your example:
>
>     oid 0x0265be BTrees.IOBTree.IOBucket
>     last updated: 2004-09-16 02:32:47.973507, tid=0x357DBF8CCAFDCCCL
>     refers to invalid object:
>            oid 0x02b6c2 missing: 'BTrees.IOBTree.IOBucket'
>
> oid 0x02b6c2 is not in your .fs.index, and so an attempt to load oid
> 0x02b6c2 should cause a POSKeyError.  When you said:
>
>     but when I dig in there, the IOBucket appears to just have strings as
> the values. And they're all present
>
> it wasn't clear what "when I dig in there" meant.  What specifically
> did you do to inspect oid 0x02b6c2?  Or you were looking at oid
> 0x0265be?  ("the IOBucket" was ambiguous, since two distinct IOBuckets
> are mentioned in the output).

Sorry, by "dig in there" I meant that I loaded up the object with oid 0x0265be 
using:

>>> from Zope.Startup.run import configure;configure('zope-19100/zope.conf')
>>> from Zope import app; root = app()
>>> from ZODB.utils import p64
>>> o = root._p_jar[p64(0x0265be)]

and then I had a poke at that:

>>> for k,v in o.items():
...  print k, type(o[k]), o[k]
...
1531753053 <type 'str'> /CGPublisher/publishers/12/messages/17
1610364516 <type 'str'> /CGPublisher/works/171/messages/1
1610364517 <type 'str'> /CGPublisher/publishers/11/messages/31
1610364518 <type 'str'> /CGPublisher/publishers/11/messages/32
1610364519 <type 'str'> /CGPublisher/works/173/messages
1610364520 <type 'str'> /CGPublisher/publishers/11/messages/33
1610364521 <type 'str'> /CGPublisher/publishers/11/messages/34
1637779823 <type 'str'> /CGPublisher/publishers/11/messages/30
1655774688 <type 'str'> /CGPublisher/works/163/messages/4
1660892580 <type 'str'> /CGPublisher/publishers/11/messages/75
1660892581 <type 'str'> /CGPublisher/publishers/11/messages/76
1660892582 <type 'str'> /CGPublisher/publishers/11/messages/77
[snip many similar lines]
1701534533 <type 'str'> /CGPublisher/publishers/13/messages/63
1701534534 <type 'str'> /CGPublisher/publishers/13/messages/64
1701534535 <type 'str'> /CGPublisher/publishers/13/messages/65
1701534536 <type 'str'> /CGPublisher/publishers/13/messages/66
1701534537 <type 'str'> /CGPublisher/publishers/13/messages/67
1701534538 <type 'str'> /CGPublisher/publishers/13/messages/68
1701534539 <type 'str'> /CGPublisher/publishers/13/messages/69
1708905051 <type 'str'> /CGPublisher/works/170/messages
1716432762 <type 'str'> /CGPublisher/publishers/13/messages/68/2
1716432763 <type 'str'> /CGPublisher/works/183/messages
>>>

and just to confirm I'm not going mad:

>>> root._p_jar[p64(0x02b6c2)]
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/opt/zope/cgpublisher-prod/Zope/lib/python/ZODB/Connection.py", line 
170, in __getitem__
  File "/opt/zope/cgpublisher-prod/Zope/lib/python/ZEO/ClientStorage.py", line 
749, in load
  File "/opt/zope/cgpublisher-prod/Zope/lib/python/ZEO/ServerStub.py", line 
82, in zeoLoad
  File "/opt/zope/cgpublisher-prod/Zope/lib/python/ZEO/zrpc/connection.py", 
line 372, in call
ZODB.POSException.POSKeyError: 0x02b6c2


I guess one issue here is that I'm poking fsrefs.py directly at the Data.fs, 
whereas the above session is done through a ZEO connection. Not sure how ZEO 
could "hide" the erroroneous data from me, but then I don't know the inner 
workings of ZEO and its caches...

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFBSTTHrGisBEHG6TARAhpRAJ9fjHBmh+onuKJVfUGeCUBuR5ziSQCeNyv0
pF53zqqHHOdbYDHShzGKW0I=
=gSl/
-----END PGP SIGNATURE-----