[Zope-dev] ERROR(200) ZODB Couldn't load state for... Memory problem?

[Tim Peters]

[Chris McDonough]
> ...
> I think the problem is related to ZEO client storage "cache flips".

Me too.  The 3.2 ZEO cache alternates between two cache files, in the
two-element list self._f.  Both elements are initialized to None, and
the index of the current file in use (0 or 1) is in self._current. 
There's an implicit assumption throughout the code that
self._f[self._current] is always an actual file object, but the "flip
logic" is excruciating and that global invariant certainly isn't
self-evident.

> I found an error in the log near the time of the "None has no attribute
> seek" symptom indicating that the Zope process tried to "flip" a ZEO
> cache file (by creating a new file) but UNIX file system permissions
> apparently prevented it.

This was a traceback ending somewhere in ClientCache.checkSize()? 
That's where a cache flip happens.  It changes its idea of
self._current (from 0 to 1 or from 1 to 0) *before* making sure
there's an actual file object in "the other" self._f slot.  So, e.g.,
if self._f started life as [good_file_object, None] and self._current
started at 0, and it came time for a cache flip, and a new file object
couldn't be created in self._f[1], self._current would end up as 1
anyway, pointing to None.  But this code gives me a headache, and I'm
not sure that can actually happen (despite that I hear you guys saying
it is <wink>).

> But then I turned off persistent ZEO client cachefile storage (but omitting
> the "zeo-client-name" parameter from zope.conf), believing this would be a
> workaround, but it hasn't been.  I gave up at that point and that's where I am now.

Did you continue to get errors in the log near cache-flip times?  I
don't see a way for checkSize() to screw up unless an unexpected
exception is raised.

> ...
> My theory is that it will happen as often as a Zope client's ZEO client storage
> needs to flip its cache file.  The cache file is only flipped when it exceeds a
> certain size and it only exceeds a certain size after a certain pattern of usage
> causes it to do so (lots of loads from the database of new items, typically).

It appears that once self._f[self._current] is None, all future
attempts by ZEO to store into its client cache will fail the same way.
 So I'd be even more surprised if you saw just one of these occur.

> It would be nice if you could confirm this.  Reading the Zope event log
> file of the client that generated the error would be a good start.

The log is everything here.  The ZEO client cache logs most relevant
messages at info level, producing msgs starting with "ZEC".