On Wed, Dec 10, 2003 at 11:32:27AM -0500, Chris McDonough wrote: | The default Zope setup has two databases: the main database and the | session database. So it's not completely nonsensical for any | default-configured Zope to raise the error (although I have no idea how | this error is occurring). It might be worthwhile to specify a main | database explicitly in zope.conf. This will prevent the default | database setup from being invoked and then you will be sure that you | only have one database connection. If I don't specify the temp_folder mount database, I get this error: 2003-12-10T14:37:46 ERROR(200) ZODB Failed to mount database. DBTab.Exceptions.DBTabConfigurationError (No database configured for mount point at /temp_folder) Traceback (most recent call last): File "/var/src/zope/head/lib/python/Products/ZODBMountPoint/Mount.py", line 86, in _getOrOpenObject conn = self._getMountedConnection(anyjar) File "/var/src/zope/head/lib/python/Products/ZODBMountPoint/Mount.py", line 63, in _getMountedConnection db_name = self._getDBName() File "/var/src/zope/head/lib/python/Products/ZODBMountPoint/MountedObject.py", line 154, in _getDBName return getConfiguration().getDatabaseFactory(self._path).getName() File "/var/src/zope/head/lib/python/DBTab/DBTab.py", line 107, in getDatabaseFactory self._mountPathError(mount_path) File "/var/src/zope/head/lib/python/DBTab/DBTab.py", line 71, in _mountPathError raise DBTabConfigurationError( DBTabConfigurationError: No database configured for mount point at /temp_folder Traceback (most recent call last): File "/var/src/zope/head/lib/python/Zope/Startup/run.py", line 27, in ? run() File "/var/src/zope/head/lib/python/Zope/Startup/run.py", line 24, in run start_zope(opts.configroot) File "/var/src/zope/head/lib/python/Zope/Startup/__init__.py", line 50, in start_zope starter.startZope() File "/var/src/zope/head/lib/python/Zope/Startup/__init__.py", line 218, in startZope Zope.startup() File "/var/src/zope/head/lib/python/Zope/__init__.py", line 46, in startup _startup() File "/var/src/zope/head/lib/python/Zope/App/startup.py", line 95, in startup OFS.Application.initialize(application) File "/var/src/zope/head/lib/python/OFS/Application.py", line 348, in initialize tf._setObject('session_data', toc) AttributeError: _setObject -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher "Nuclear war can ruin your whole compile." -- Karl Lehenbauer

On Wed, Dec 10, 2003 at 12:19:47PM -0500, Chris McDonough wrote: | Right. Ugh. I need to fix this. To work around if for the meantime, | you could comment out Application.py's line 348 and other related lines | around it. Ok, commented out 292-360, and now I'm back to the previous traceback. So, it seems to be happening with only one database open. -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher Linux - Where do you want to fly today? -- Unknown source

On Wed, Dec 10, 2003 at 12:36:53PM -0500, Chris McDonough wrote: | And you see only the "main" database in Control_Panel->Databases? No, I can't get this far. The error happens on startup. -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher %DCL-MEM-BAD, bad memory VMS-F-PDGERS, pudding between the ears

On Wed, Dec 10, 2003 at 12:50:29PM -0500, Chris McDonough wrote: | Right. ;-) At this point I should just try it myself but if you remove | the Formulator product, does Zope start? If I remove Plone and Formulator it starts yes. -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher Equal bytes for women.

| Probably a good next step is to set a breakpoint in | ZODB/coptimizations.c where this exception is raised and find out what | object is causing the problem, which jar it's already in, and why it's | getting added to a different jar. Or you could add some print | statements. I figured out how to set a breakpoint in gdb, but I don't know how to get at the object and inspect it. Can I get a quick hint? I know there has to be some pointer cast probably, just need to know which one. :) -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher fortune: cannot execute. Out of cookies.

| I think you're looking for the helpers defined in Misc/gdbinit in the | Python distribution. If you add them to your .gdbinit, you'll be | golden. | | I've included that file's contents below so you don't have to go | hunting. Neat! There we have 'object', 'self' and 'self->jar', as of line 183 of coptimizations.c: object : <class 'Products.Formulator.StandardFields.LabelField'> type : ExtensionClass.ExtensionClass refcount: 7 address : 0x8503bec object : <persistent_id object at 0x415b87b8> type : persistent_id refcount: 1 address : 0x415b87b8 object : <Connection at 414b80ac> type : Connection refcount: 238 address : 0x414b80ac 'jar' gives: object : <attribute '_p_jar' of 'persistent.Persistent' objects> type : getset_descriptor refcount: 2 address : 0x40621e6c Does that help? -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher [We] use bad software and bad machines for the wrong things. -- R.W. Hamming

On Wed, Dec 10, 2003 at 02:17:23PM -0500, Jeremy Hylton wrote: | On Wed, 2003-12-10 at 14:18, Sidnei da Silva wrote: | > 'jar' gives: | > object : <attribute '_p_jar' of 'persistent.Persistent' objects> | > type : getset_descriptor | > refcount: 2 | > address : 0x40621e6c | > | > Does that help? | | Yes. The code is getting an unexpected raw descriptor. Someone should | be calling __get__() on this descriptor to get the value. I'll have to | look into it to see how it's going wrong. If I can do anything to help, just ask. -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher The program isn't debugged until the last user is dead.

On Wed, Dec 10, 2003 at 03:26:03PM -0500, Jeremy Hylton wrote: | Actually, there are two equally valuable clues from the debugger | session: | | object : <class 'Products.Formulator.StandardFields.LabelField'> | type : ExtensionClass.ExtensionClass | refcount: 7 | address : 0x8503bec | | If I understand correctly: LabelField is a class that inherits from | Persistence and some other object has a reference to it that isn't via | __class__. Makes sense to me. How can I find the referencing object? | >>> persistent.Persistent._p_jar | <attribute '_p_jar' of 'persistent.Persistent' objects> | | If you ask a class for an attribute, it seems you can get a descriptor | back. I vaguely recall Guido saying that this was the intended behavior | last time I asked about it. Yes, I've learnt this the hard way, when Jim tried to explain me and I kept banging my head against the wall :) | 'jar' gives: | object : <attribute '_p_jar' of 'persistent.Persistent' objects> | type : getset_descriptor | refcount: 2 | address : 0x40621e6c | | Thus, asking for the class's _p_jar gives the descriptor back. In ZODB4 | I hacked around this by requiring that the value of _p_jar be None or a | string. Would there be any objection to having the same requirement in | ZODB3? I have no clue about the implications of this, but it seems like a reasonable constraint to me. -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher MS-DOS, you can't live with it, you can live without it. -- from Lars Wirzenius' .sig

I think there's a fix for this in CVS. Can you try it again? Jeremy

On Thu, 2003-12-11 at 10:24, Sidnei da Silva wrote:

On Thu, Dec 11, 2003 at 10:07:51AM -0500, Jeremy Hylton wrote: | I think there's a fix for this in CVS. Can you try it again?

Just tried. Seems to have fixed it. Now, for the next step *wink*.

If I try starting Zope with an old existing database I get the traceback below. Correct me if I'm wrong, but I think the error with 'UserDict' has nothing to do with the 'Could not import class 'BTree'' or am I wrong?

I think the attribute error on data is just a symptom of the BTree failure. The object has no data attribute, because ZODB failed to unpickle the object that would be assigned to data. I don't know what happened with backwards compatibility for the old BTree classes. When Jim and I discussed it a while back, I thought we concluded that we would provide a script that would convert all the old BTree instances to new BTrees instance. But I don't know if that decision stuck or if the script was ever written. Jeremy

| Jim asked about it on zope-dev, and got no response; see: | | "Can we forsake database backward compatability on the head for | a while?" | | http://mail.zope.org/pipermail/zope-dev/2003-November/021055.html | | ... | | > This would mean that old databases would not be useable with the | > CVS head in the near term. Would this cause anyone any problems? | | I'm happy no one responded to this. Silence is assent, :) :P Is there any special trick to such script? Provided some pointers of what the script should do, I wouldn't mind writing and testing it. -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher baz bat bamus batis bant. -- James Troup

On Thu, 2003-12-11 at 15:28, Tim Peters wrote:

Anyone? Since Sidnei's efforts to use the head are exposing problems, it would be great if we could keep him obsessed <wink> with it a bit longer.

Just point Sidnei torwards (sp?) some pretty API docs about old and new BTrees and he'll be happy as an otter. He'll come up with a conversion script in no time :-) []'s -- Ideas don't stay in some minds very long because they don't like solitary confinement.

[Jeremy Hylton]

I don't know if there is any documentation for BTree,

I don't know of any dcos for (the old) BTree either.

so the code will need to be your guide. I checked the code and each of the BTree types does support items(). So walk over the database, find each top-level BTree object, extract its items, put them in a new BTrees-style BTree, then save that in place of the old object.

Then it may not be necessary to extract the items -- a constructor for a BTrees-style BTree object: /** ** Accepts a sequence of 2-tuples, or any object with an items() ** method that returns an iterable object producing 2-tuples. */ IOW, the constructor should invoke .items() for you -- just pass it the old-style BTree. Of course this has to be done under a version of ZODB that still *has* old-style BTree support! I don't know whether the old-style BTree comes in different type-dependent flavors, either. If not, an old BTree should probably become a new OOBTree, and an old IntSet a new IISet.

On Thu, 2003-12-11 at 15:15, Sidnei da Silva wrote:

Do'h, that's a bit difficult since Jim removed them for good :(

Hmmm... I suppose you could write the script in Zope 2.7 and tell people they have to use that version of Zope to convert the database before upgrading.

Now, back to the automagically-converting idea, would it be possible to provide an alias for the old BTree package that would feed the old pickle into a new BTrees constructor on unpickling *wink*?

I just remembered why the problem is harder. Any object that references as BTree has the name of the BTree class embedded in its pickle. The conversion script needs to frob each of those database records to refer to the new class. I'm not sure how to do that. Unpickle and repickle each one? A "simple" binary rewrite of the pickle to past in the new name. Just not sure. Jeremy

| >Is there any special trick to such script? Provided some pointers of | >what the script should do, I wouldn't mind writing and testing it. | | There are lots of special tricks to such a script. Otherwise, I would | have written it already. :) Obviously. /me slaps his forehead | OK, here's how I think it needs to happen, with some, hopefuly helpful | notes. | | 1. The script needs to be run under Zope 2.6 or 2.7. There should | be a product that people can install that installs some method to | be run. People should be able to run this method in a running zope. | They should not need to shut down their site to do the conversion. | | 2. The method should walk the object tree, converting: | | - old-style BTrees to modern BTrees | - old-style buckets to modern buckets | - intSets to IITreeSets. Just to check: +----------+-----------+ | Old | New | +----------+-----------+ | BTree | OOBTree | +----------+-----------+ | IIBTree | IIBtree | +----------+-----------+ | OIBTree | OIBTree | +----------+-----------+ | IOBTree | IOBTree | +----------+-----------+ | intSet | IITreeSet | +----------+-----------+ | bucket? | bucket? | +----------+-----------+ | The individual conversions should be pretty simple. At worst involving | a for loop, but, in many cases, the constructors for the new types will | be able to accept the old types. | | 3. The hard part is walking the object tree. You will need a function that, | given an object, will return the oids of the objects it references. | Perhaps Jeremy can help you with that. I hope so *wink*. | 4. You should do each conversion in a separate transaction. That is, when | you | encounter an object that needs to be converted, you should convert it | and commit. | | 5. You will want to keep a dictionary of the oids visited, just in case | there are | any cyles. Also, if you encounter an oid that was converted, you will | need to | subtitute the new value. This might be really hard. :( I think this | probably | won't happen, but perhaps you should at least check for it. | | 6. After the conversion has been run, the database should be usable with | the head. Kewl. I'm eager to get my hands on it. -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher Somebody's terminal is dropping bits. I found a pile of them over in the corner.

| Think of it in terms of modules: | | objects from BTree -> objects from BTrees.OOBTree | objects from IOBTree -> objects from BTrees.IOBTree | objects from OIBTree -> objects from BTrees.OIBTree | objects from IIBTree -> objects from BTrees.IIBTree | objects from intSet -> objects from BTrees.IIBTree Ok. +--------------------+------------------------+ |Old |New | +--------------------+------------------------+ |BTree.BTree |BTrees.OOBTree.OOBtree | +--------------------+------------------------+ |BTree.Bucket |BTrees.OOBTree.OOBucket | +--------------------+------------------------+ |IOBTree.BTree |BTrees.IOBTree.IOBTree | +--------------------+------------------------+ |IOBTree.Bucket |BTrees.IOBTree.IOBucket | +--------------------+------------------------+ |OIBTree.BTree |BTrees.OIBTree.OIBTree | +--------------------+------------------------+ |OIBTree.Bucket |BTrees.OIBTree.OIBucket | +--------------------+------------------------+ |IIBTree.IIBTree |BTrees.IIBTree.IIBTree | +--------------------+------------------------+ |IIBTree.Bucket |BTrees.IIBTree.IIBUcket | +--------------------+------------------------+ |intSet |BTrees.IIBTree.IITreeSet| +--------------------+------------------------+ | >| 3. The hard part is walking the object tree. You will need a function | >that, | >| given an object, will return the oids of the objects it references. | >| Perhaps Jeremy can help you with that. | > | >I hope so *wink*. | | Me too. He's pretty busy. If he can't, I'll try to help. I started with a slight variation of Shane's change_modules.py [1]. Had to make some changes because stuff seem to have changed in the Connection class. For example, invalidate() now expects a dict instead of a oid. My plan is to change around line 130, where it creates a new pickle with new_class_spec to wake the old object, create a new one and pickle the new object. Sounds reasonable? [1] http://cvs.sourceforge.net/viewcvs.py/zodbex/zodbex/ChangeModules/change_mod... -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher A programming language is low level when its programs require attention to the irrelevant.

On Fri, 2003-12-12 at 10:09, Sidnei da Silva wrote:

/me grumbles

Is it right that in general the only old-style BTree in a ZODB is 'ZGlobals' at the root? It seems ZCatalog have a manage_convertBTrees method since 2.3.1, and at least on my instance, the indexes BTrees are already new.

I think that is true. On the other hand, it was a bit of a surprise to find a BTree in ZGlobals. Maybe they're hiding somewhere else, too.

I'm going to change the code to use BTrees.convert, now that I've found it *wink*.

Didn't know that was there either. Jeremy

I think this problem is actually easier to solve in the common case than requring that people run a conversion script. The ZGlobals data structure is a cache, and can be blown away and recreated indiscriminately. It's recreated whenever a Product is added or removed, and Zope already does this as a matter of course if anything about the data structure is goofy. I think the problem Sidnei has run into isn't so much that the BTree module can't be imported, it's instead that the Persistence.PersistentMapping module redefines PersistentMapping in order to do something I don't quite understand, and the redefinition is what appears to be causing the AttributeError, 'data' to appear. If it's not used, everything works fine. Here's the module: import Persistence import persistent from persistent.mapping import PersistentMapping if Persistence.Persistent is not persistent.Persistent: class PersistentMapping(Persistence.Persistent, PersistentMapping): """Legacy persistent mapping class This class mixes in ExtensionClass Base if it is present. Unless you actually want ExtensionClass semantics, use persistent.mapping.PersistentMapping instead. """ If you comment out the "if" clause and everything underneath of it, make a dummy BTree.py module with the following and put in in the software home: from Globals import Persistent class BTree(Persistent): pass ... and restart Zope with an "old" filestorage file, all is well: [chrism@beach zopemafia.org]$ bin/zopectl-trunk debug Starting debugger (the name "app" is bound to the top-level Zope object)

...

...

app.objectIds() ['acl_users', 'Control_Panel', 'temp_folder', 'browser_id_manager', 'session_data_manager', 'error_log', 'standard_error_message', 'standard_html_footer', 'standard_html_header', 'standard_template.pt', 'vhm', 'tacoma.bridge.jpg', 'tacoma.bridge1.jpg', 'index_html_orig', 'index_html', 'misc', 'innomatch', 'newplone', 'New Folder', 'scratch'] root = app._p_jar.root() root {'Application': <OFS.Application.Application object at 0x41419cac>, 'ZGlobals': <BTrees._OOBTree.OOBTree object at 0x4185826c>} type(root) <class 'persistent.mapping.PersistentMapping'>

I'm not sure what the redefinition is supposed to achieve? FWIW, I've sort of got plope.com (which implies Plone) working against the trunk, although some other unrelated problems still appear to exist. On Fri, 2003-12-12 at 10:41, Sidnei da Silva wrote:

| I think that is true. On the other hand, it was a bit of a surprise to | find a BTree in ZGlobals. Maybe they're hiding somewhere else, too.

At least, the script didn't found another one. It did found 7800 objects total, of which one was a BTree (ZGlobals).

Now, in 2.7 I get this output (starting zope via 'zopectl debug'):

...

...

...

app <Application instance at 414c8c80> app._p_jar <Connection at 414ac74c> app._p_jar.root() {'Application': <Application instance at 414c8c80>, 'ZGlobals': <OOBTree object at 0x414d1c60>, 'RefreshData': {'auto': {'CCHS': 1}, 'products': {'Archetypes': {'dependent_products': ('CCHS',)}}}}

Which looks sane to me. But if I copy the same Data.fs to another instance with Zope HEAD, the "AttributeError: data" is there again. It *seems* to me that the root PersistentMapping is not being unpickled correctly.

...

/var/src/zope/head/lib/python/ZODB/ZApplication.py(32)__init__() -> conn=db.open() (Pdb) n /var/src/zope/head/lib/python/ZODB/ZApplication.py(33)__init__() -> root=conn.root() (Pdb) n /var/src/zope/head/lib/python/ZODB/ZApplication.py(34)__init__() -> if not root.has_key(name): (Pdb) print root *** AttributeError: data (Pdb) type(root) <class 'Persistence.PersistentMapping.PersistentMapping'> (Pdb) print conn.root() *** AttributeError: data

On Sun, 2003-12-14 at 22:08, Jim Fulton wrote:

Chris McDonough wrote:

...

I think this problem is actually easier to solve in the common case than requring that people run a conversion script. The ZGlobals data structure is a cache, and can be blown away and recreated indiscriminately. It's recreated whenever a Product is added or removed, and Zope already does this as a matter of course if anything about the data structure is goofy.

Hm, OK. Then the question is how to decide that it's goofy.

The current way it's done is in OFS.Application.Application via: def checkGlobalRegistry(self): """Check the global (zclass) registry for problems, which can be caused by things like disk-based products being deleted. Return true if a problem is found""" try: keys=list(self._p_jar.root()['ZGlobals'].keys()) except: LOG('Zope', ERROR, 'A problem was found when checking the global product '\ 'registry. This is probably due to a Product being '\ 'uninstalled or renamed. The traceback follows.', error=sys.exc_info()) return 1 return 0 If the check fails, the registry is blown away and rebuilt in the same class' fixupZClassDependencies method which does a bunch of sniffing around in Products for zclass ids. I admit I have no idea whether this check is valid and haven't had to care in a long time. It's been there since before 2.3, however.

I'll note that I certainly did not intend it to be a cache when I added it a long time ago. I'm a bit uncomfortable assuming that it's always used that way, but you know more than I about the current usage.

Probably not, but I do know that there haven't been many complaints about the rebuilding code.

...

If you comment out the "if" clause and everything underneath of it, make a dummy BTree.py module with the following and put in in the software home:

from Globals import Persistent

class BTree(Persistent): pass

... and restart Zope with an "old" filestorage file, all is well:

There needs to be a cleaner solution than this. A Dummy BTree module is not acceptable.

Here's an idea: create a simple script that manufactured a module which set up the Zope configuration, "filled in" for BTree in sys.modules, and got a hold of Zope.app(), causing ZGlobals to be replaced by a new-style BTree due to the above magic. People upgrading to 2.8 with databases created by Zope versions older than 2.3 would still need to run convertBTrees inside a different Zope version, but people upgrading databases created in versions equalling or later than 2.3 wouldn't because we stopped using the old BTree module for anything except ZGlobals after 2.3 AFAICT.

...

I'm not sure what the redefinition is supposed to achieve?

persistent.Persistent is not an extension class. It's a pure new-style persistent base class.

Out of curiosity, could you explain why it matters in this context whether it's an extensionclass or not? Is it because there might be a set of people using PersistentMapping objects for whom its important that they be able to use extenionclass semantics against them? I'd just never thought of using __of__ or inheritedAttribute or any of the other EC-specific stuff on a PersistentMapping.

Persistent.Persistent is an extension class if ExtensionClass is installed. If ExtensionClass is not installed, then a warning is issued and it is set tp persistent.Persistent. This is to ease the transition for people who use ZODB outside of Zope.

I see.. thanks. Well, even in the case that a (dummy or real) BTree module doesn't exist, if Zope is told to use the persistent.mapping.PersistentMapping class instead of the one that inherits from Persistence.Persistent, it does not fail in the way Sidnei reported. Additionally, if you switch the redefined PersistentMapping's base class order, it also begins to work. The fact it can't import BTree.BTree causes it to complain, but the ZGlobals rebuilder does begin work under both circumstances and Zope starts. ------ 2003-12-14T22:45:49 ERROR(200) ZODB Couldn't load state for 000000000000000a Traceback (most recent call last): File "/home/chrism/software/Trunk/lib/python/ZODB/Connection.py", line 428, in setstate self._reader.setGhostState(obj, p) File "/home/chrism/software/Trunk/lib/python/ZODB/serialize.py", line 203, in setGhostState obj.__setstate__(state) TypeError: function takes exactly 2 arguments (0 given) ------ 2003-12-14T22:33:35 ERROR(200) Zope A problem was found when checking the global product registry. This is probably due to a Product being uninstalled or renamed. The traceback follows. Traceback (most recent call last): File "/home/chrism/software/Trunk/lib/python/OFS/Application.py", line 233, in checkGlobalRegistry keys=list(self._p_jar.root()['ZGlobals'].keys()) File "/home/chrism/software/Trunk/lib/python/ZODB/Connection.py", line 428, in setstate self._reader.setGhostState(obj, p) File "/home/chrism/software/Trunk/lib/python/ZODB/serialize.py", line 203, in setGhostState obj.__setstate__(state) TypeError: function takes exactly 2 arguments (0 given) ------ 2003-12-14T22:33:35 INFO(0) Zope Beginning attempt to rebuild the global ZClass registry. ------ 2003-12-14T22:33:36 INFO(0) Zope The global ZClass registry has successfully been rebuilt. ------ Given this, would it make sense that the base class order of the redefined PersistentMapping class just needs to be turned around? - C

Chris McDonough wrote:

On Sun, 2003-12-14 at 23:02, Chris McDonough wrote:

...

Here's an idea: create a simple script that manufactured a module which set up the Zope configuration, "filled in" for BTree in sys.modules, and got a hold of Zope.app(), causing ZGlobals to be replaced by a new-style BTree due to the above magic. People upgrading to 2.8 with databases created by Zope versions older than 2.3 would still need to run convertBTrees inside a different Zope version, but people upgrading databases created in versions equalling or later than 2.3 wouldn't because we stopped using the old BTree module for anything except ZGlobals after 2.3 AFAICT.

Actually, given my findings later in this email, this isn't even necessary.

Right. Whew. I checked in the necessary fix for PersistentMapping. There was a change in the data structure and I had to add some logic to convert some old state. I also had to check in a %$#@ travesty because Data.fs.in still has references to BoboPOS. With these changes, I can create a database with 2.6 and open it with the head. Thanks Chris for helping us figure out that this was easier than we thought. :) Jim -- Jim Fulton mailto:jim@zope.com Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org

On Mon, 2003-12-15 at 02:39, Jim Fulton wrote:

...

I checked in the necessary fix for PersistentMapping. There was a change in the data structure and I had to add some logic to convert some old state.

Oh, geez, I remember this setstate/getstate business now. FWIW, I think Jeremy added it at my insistence after he changed PersistentMapping to inherit from UserDict in order to maintain backwards compatibility for versions that didn't. The next panic will start when UserDict becomes a new-style class. ;-) - C

On Mon, 2003-12-15 at 02:47, Jim Fulton wrote:

Chris McDonough wrote: ...

...

Out of curiosity, could you explain why it matters in this context whether it's an extensionclass or not? Is it because there might be a set of people using PersistentMapping objects for whom its important that they be able to use extenionclass semantics against them? I'd just never thought of using __of__ or inheritedAttribute or any of the other EC-specific stuff on a PersistentMapping.

<shrug> Who knows. I'd rather be safe. It's not that hard. In any case, we would have needed the fix to handle old pickles correctly.

Gotcha. Can you answer a question about the features we're allowed now? If we don't use extensionclass, is it possible to persist instances of new-style classes in ZODB now (ie. is persistent.Persistent really a new-style type/class)? - C

Woot! ;-) Descriptors, yum... On Mon, 2003-12-15 at 10:02, Jeremy Hylton wrote:

On Mon, 2003-12-15 at 08:15, Chris McDonough wrote:

...

Can you answer a question about the features we're allowed now? If we don't use extensionclass, is it possible to persist instances of new-style classes in ZODB now (ie. is persistent.Persistent really a new-style type/class)?

Yes. (i.e. yes.)

Jeremy

On Mon, 2003-12-15 at 05:21, Jim Fulton wrote:

Chris McDonough wrote:

...

Woot! ;-) Descriptors, yum...

But you can use descriptors in extension classes now too. Extension classes are now just new-style classes with some extra features and a different mro algorithm.

Whoa. Cool. That really is great. You guys are so clever. ;-) - C

On Mon, Dec 15, 2003 at 02:47:01AM -0500, Jim Fulton wrote:

<shrug> Who knows. I'd rather be safe. It's not that hard. In any case, we would have needed the fix to handle old pickles correctly.

Asking a possibly redundant question... not being much of a ZODB hacker, I didn't follow this thread in detail... Q: Will Zope >= 2.8 be able to import .zexp's exported from zope <= 2.6.2? From Jim's comment above I would guess "yes", but I need to be sure. Some of us out here in the wild use import / export machinery pretty heavily (see e.g. ZSyncer), and we typically like to burn in a new zope version on a dev system for some time before upgrading the production systems... but during that time we need to keep syncing stuff from dev to production. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's FOOD-COVERED STRAIGHT FROM UNCLE! (random hero from isometric.spaceninja.com)

| Shane's "BTreeFolder" uses them, too. The old one. BTreeFolder2 uses the new BTrees. -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher You have mail.

On Thu, Dec 11, 2003 at 11:03:35PM -0500, Jim Fulton wrote:

3. The hard part is walking the object tree. You will need a function that, given an object, will return the oids of the objects it references. Perhaps Jeremy can help you with that.

Perhaps some of the zodb utilities contain code that might be useful as a starting point? fsrefs.py, checkbtrees.py...

5. You will want to keep a dictionary of the oids visited, just in case there are any cyles.

I believe the version of checkbtrees.py in cvs now does that. -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's PENGUIN FINALE! (random hero from isometric.spaceninja.com)

| Do you actually have more than one database connection? In other words, | is the error totally nonsensical or could the bug be that something is | getting stored in a foreign database connection? I'm just starting Zope by './bin/runzope' from a newly-created instance. No ZEO, no nothing. -- Sidnei da Silva <sidnei@awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher I came, I saw, I deleted all your files.