[ZODB-Dev] persistent broken problem

[Kaweh Kazemi]

Hi there,

started using ZODB3 with a new project of mine and I've encountered a problem, I couldn't find a solution for so far.

I'm adding instances of a class User to a storage (FileStorage for now). I did this in a small test program which worked fine. But then I started to refactor components out of the test program into a module hirarchie and I'm encountering "persistent broken" instances now.

1) First run: no database nor users created yet, creating a FileStorage, adding 3 users to the database will yield something like this (output within the tester after the work has been done):

	[<rp.framework.user.User object at 0x1016986e0>, <rp.framework.user.User object at 0x101698758>, <rp.framework.user.User object at 0x101698848>]

2) Second run: reopening previous database, adding 3 new users to the database will yield something like this (output within the tester after the work has been done):

	[<persistent broken rp.framework.user.User instance '\x00\x00\x00\x00\x00\x00\x00\x02'>, <persistent broken rp.framework.user.User instance '\x00\x00\x00\x00\x00\x00\x00\x04'>, <persistent broken rp.framework.user.User instance '\x00\x00\x00\x00\x00\x00\x00\x05'>, <rp.framework.user.User object at 0x1017ad410>, <rp.framework.user.User object at 0x1017adc80>, <rp.framework.user.User object at 0x1017adde8>]

3) Subsequent runs will add three users with the same output, just the number of persistent broken users is incorrect.

Now I understand that a class is "broken" if the import of that class during opening of the database fails. But I'm doing the runs without changes in between the runs, no changes to the module structure or code and running the exactly same tester (which just re-opens the database and add three new instances).

I digged deeper into ZODB.broken to better understand what is happening when the module is imported for a storage. It seems that while opening the database the import of the same module gives slightly different results than later in the script when adding new users.

I wonder if that's related to my module structure, which looks similiar to this:

	rp/
		__init__.py
		framework/
			__init__.py
			...
			user.py - this holds the class User, which is a subclass of Entity (see below)

		odb/
			__init__.py
			...
			entity.py - this holds the class Entity, which does a couple of general things for my implementation, which is a subclass of Persistent; class Entity has a class EntityMeta as it's __metaclass__ to handle some magic for me (very similar to what Elixier does for SQLAlchemy, but not that sophisticated)

I'm not doing anything in the __init__.py (so I'm not mangeling the module imports in any way, it's just a pure, very young structure I'm building up).

The tester is outside of the rp module and just does a "from rp.framework.user import User", always the exact same import and still I get the same issue.

I tried a couple of tricks in the __init__.py things, or changing sys.path, sys.modules, or even PYTHONPATH, but to no avail.

One thing I still wonder is, when exactly find_module is called? Is it called when opening the storage (which happens in my EntityMeta class, so during the creation of the Entity class - and that might ruin everything), or when an object is accessed the first time? Can this behaviour be controlled?

I'm quite certain I miss something quite obvious, but I'm stuck.

Any hints appreciated.

Cheers,
Kaweh