[Zope-dev] Re: Converting from old-style BTreess

[Chris McDonough]

On Sun, 2003-12-14 at 22:08, Jim Fulton wrote:
> Chris McDonough wrote:
> > I think this problem is actually easier to solve in the common case than
> > requring that people run a conversion script.  The ZGlobals data
> > structure is a cache, and can be blown away and recreated
> > indiscriminately.  It's recreated whenever a Product is added or
> > removed, and Zope already does this as a matter of course if anything
> > about the data structure is goofy.
> 
> Hm, OK.  Then the question is how to decide that it's goofy.

The current way it's done is in OFS.Application.Application via:

    def checkGlobalRegistry(self):
        """Check the global (zclass) registry for problems, which can
        be caused by things like disk-based products being deleted.
        Return true if a problem is found"""
        try:
            keys=list(self._p_jar.root()['ZGlobals'].keys())
        except:
            LOG('Zope', ERROR,
                'A problem was found when checking the global product '\
                'registry.  This is probably due to a Product being '\
                'uninstalled or renamed.  The traceback follows.',
                error=sys.exc_info())
            return 1
        return 0

If the check fails, the registry is blown away and rebuilt in the same
class' fixupZClassDependencies method which does a bunch of sniffing
around in Products for zclass ids.  I admit I have no idea whether this
check is valid and haven't had to care in a long time.  It's been there
since before 2.3, however.

> I'll note that I certainly did not intend it to be a cache when
> I added it a long time ago.  I'm a bit uncomfortable assuming that
> it's always used that way, but you know more than I about the current usage.

Probably not, but I do know that there haven't been many complaints
about the rebuilding code.

> > If you comment out the "if" clause and everything underneath of it, make
> > a dummy BTree.py module with the following and put in in the software
> > home:
> > 
> >  from Globals import Persistent
> > 
> >  class BTree(Persistent):
> >      pass
> > 
> > ... and restart Zope with an "old" filestorage file, all is well:
> 
> There needs to be a cleaner solution than this.  A Dummy BTree module is
> not acceptable.

Here's an idea: create a simple script that manufactured a module which
set up the Zope configuration, "filled in" for BTree in sys.modules, and
got a hold of Zope.app(), causing ZGlobals to be replaced by a new-style
BTree due to the above magic.  People upgrading to 2.8 with databases
created by Zope versions older than 2.3 would still need to run
convertBTrees inside a different Zope version, but people upgrading
databases created in versions equalling or later than 2.3 wouldn't
because we stopped using the old BTree module for anything except
ZGlobals after 2.3 AFAICT.
 
> > I'm not sure what the redefinition is supposed to achieve?
> 
> persistent.Persistent is not an extension class. It's a pure
> new-style persistent base class.

Out of curiosity, could you explain why it matters in this context
whether it's an extensionclass or not?  Is it because there might be a
set of people using PersistentMapping objects for whom its important
that they be able to use extenionclass semantics against them?  I'd just
never thought of using __of__ or inheritedAttribute or any of the other
EC-specific stuff on a PersistentMapping.

> Persistent.Persistent is an extension class if ExtensionClass is
> installed.  If ExtensionClass is not installed, then a warning is issued and
> it is set tp persistent.Persistent.  This is to ease the transition for
> people who use ZODB outside of Zope.

I see.. thanks.

Well, even in the case that a (dummy or real) BTree module doesn't
exist, if Zope is told to use the persistent.mapping.PersistentMapping
class instead of the one that inherits from Persistence.Persistent, it
does not fail in the way Sidnei reported.  Additionally, if you switch
the redefined PersistentMapping's base class order, it also begins to
work.  The fact it can't import BTree.BTree causes it to complain, but
the ZGlobals rebuilder does begin work under both circumstances and Zope
starts.

------
2003-12-14T22:45:49 ERROR(200) ZODB Couldn't load state for
000000000000000a
Traceback (most recent call last):
  File "/home/chrism/software/Trunk/lib/python/ZODB/Connection.py", line
428, in setstate
    self._reader.setGhostState(obj, p)
  File "/home/chrism/software/Trunk/lib/python/ZODB/serialize.py", line
203, in setGhostState
    obj.__setstate__(state)
TypeError: function takes exactly 2 arguments (0 given)
------
2003-12-14T22:33:35 ERROR(200) Zope A problem was found when checking
the global product registry.  This is probably due to a Product being
uninstalled or renamed.  The traceback follows.
Traceback (most recent call last):
  File "/home/chrism/software/Trunk/lib/python/OFS/Application.py", line
233, in checkGlobalRegistry
    keys=list(self._p_jar.root()['ZGlobals'].keys())
  File "/home/chrism/software/Trunk/lib/python/ZODB/Connection.py", line
428, in setstate
    self._reader.setGhostState(obj, p)
  File "/home/chrism/software/Trunk/lib/python/ZODB/serialize.py", line
203, in setGhostState
    obj.__setstate__(state)
TypeError: function takes exactly 2 arguments (0 given)
------
2003-12-14T22:33:35 INFO(0) Zope Beginning attempt to rebuild the global
ZClass registry.
------
2003-12-14T22:33:36 INFO(0) Zope The global ZClass registry has
successfully been rebuilt.
------

Given this, would it make sense that the base class order of the
redefined PersistentMapping class just needs to be turned around?

- C