LocalFS-1-3-andreas and Zope 2.7.2
Hi, Has anyone else here noticed that with Zope 2.7.x and LocalFS-1-3-andreas, when FS-based objects are changes through LocalFS, changes are also stored to the ZODB (you'll see a transaction for any change made in the Undo tab of the root of your Zope instance) as well? The changes are made up as follows (from fsdump.py): data #00000 oid=000000000000727e class=OFS.Image.Pdata data #00001 oid=000000000000727f class=OFS.Image.Pdata data #00002 oid=000000000000727c class=OFS.Image.Pdata data #00003 oid=000000000000727d class=OFS.Image.Pdata data #00004 oid=000000000000727b class=OFS.Image.Pdata data #00005 oid=0000000000007282 class=OFS.Image.Pdata data #00006 oid=0000000000007283 class=Products.LocalFS.LocalFS.ObjectWrapper data #00007 oid=0000000000007280 class=OFS.Image.Pdata data #00008 oid=0000000000007281 class=OFS.Image.Pdata ...and packing removes them. Does anyone know what's causing this and what could be done to fix it? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Chris Withers wrote at 2005-1-27 23:05 +0000:
Has anyone else here noticed that with Zope 2.7.x and LocalFS-1-3-andreas, when FS-based objects are changes through LocalFS, changes are also stored to the ZODB (you'll see a transaction for any change made in the Undo tab of the root of your Zope instance) as well?
The changes are made up as follows (from fsdump.py):
data #00000 oid=000000000000727e class=OFS.Image.Pdata data #00001 oid=000000000000727f class=OFS.Image.Pdata data #00002 oid=000000000000727c class=OFS.Image.Pdata data #00003 oid=000000000000727d class=OFS.Image.Pdata data #00004 oid=000000000000727b class=OFS.Image.Pdata data #00005 oid=0000000000007282 class=OFS.Image.Pdata data #00006 oid=0000000000007283 class=Products.LocalFS.LocalFS.ObjectWrapper
data #00007 oid=0000000000007280 class=OFS.Image.Pdata data #00008 oid=0000000000007281 class=OFS.Image.Pdata
...and packing removes them.
This is funny indeed. When a transaction contains an object's data, then this means that either * this object was modified * this object was new *AND* is referenced by a modified persistent object * this object explicitely was assigned a "_p_oid" and was later modified In the first and second case, one would expect that the object is still reachable after the transaction. Thus packing should not remove it. It is highly unlikely, that LocalFS assigns oids explicitly. Thus, three should not apply. Did you show us a complete transaction? Or are more objects involved?
Does anyone know what's causing this and what could be done to fix it?
Debugging in an interactive Python session? -- Dieter
Dieter Maurer wrote:
...and packing removes them.
This is funny indeed.
I think that depends on whether you're maintaining the Zope server that has 1.5GB of disk space eaten up by junk ;-)
When a transaction contains an object's data, then this means that either
* this object was modified
* this object was new *AND* is referenced by a modified persistent object
* this object explicitely was assigned a "_p_oid" and was later modified
In the first and second case, one would expect that the object is still reachable after the transaction. Thus packing should not remove it.
It is highly unlikely, that LocalFS assigns oids explicitly. Thus, three should not apply.
Yes, but remember what LocalFS is doing is trying to spoof Zope into believing objects come from within the ZODB, while actually storing them on the file system. I have a hunch that it probably mixes OFS.Image.File into its "File" objects, and OFS.Image.File is doing something naughty (it usually does, it's got some VERY esoteric code in it ;-) which results in data being comitted to the ZODB. However, since LocalFS manages to not store any data in the ZODB, the stuff OFS.Image.File causes to get comitted probably gets packed away... Sound plausible? I'm still keen to hear of anyone else who can reproduce this...
Did you show us a complete transaction?
Yep, pretty sure I did.
Or are more objects involved?
Nope. All data in this project is supposed to be stored either in Oracle or on the file system (accessed through LocalFS).
Does anyone know what's causing this and what could be done to fix it?
Debugging in an interactive Python session?
Okay, but what am I looking for? Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Chris Withers wrote at 2005-1-31 10:22 +0000:
... Yes, but remember what LocalFS is doing is trying to spoof Zope into believing objects come from within the ZODB, while actually storing them on the file system.
I do not think that "LocalFS" is in any way interested to simulate objects in the ZODB. Instead, it emulates a hierarchy apparently rooted in a ZODB based object. But this is purely an acquisition concept and completely independent from the ZODB.
I have a hunch that it probably mixes OFS.Image.File into its "File" objects, and OFS.Image.File is doing something naughty (it usually does, it's got some VERY esoteric code in it ;-) which results in data being comitted to the ZODB.
Indeed, it contains a subtransaction commit. However, this should only lead to true ZODB writes when the object either already was persistent or is new and part of a modified persistent object.
...
Debugging in an interactive Python session?
Okay, but what am I looking for?
In the current transaction (obtained via "get_transaction()" for "_objects". You are interested in the code that adds objects to "_objects". The first object added to "_objects" is the persistent object that should not have become modified. -- Dieter
On Mon, Jan 31, 2005 at 07:45:24PM +0100, Dieter Maurer wrote:
I do not think that "LocalFS" is in any way interested to simulate objects in the ZODB. Instead, it emulates a hierarchy apparently rooted in a ZODB based object. But this is purely an acquisition concept and completely independent from the ZODB.
if you look at the LocalFS code (I have but not recently), it actually creates a temporary instance of e.g. OFS.Image.File on every request for a "file". IIRC these are supposed to be thrown away, but apparently something is broken... -- Paul Winkler http://www.slinkp.com
Paul Winkler wrote:
if you look at the LocalFS code (I have but not recently), it actually creates a temporary instance of e.g. OFS.Image.File on every request for a "file". IIRC these are supposed to be thrown away, but apparently something is broken...
FWIW, I can only confirm seeing ZODB growth when LocalFS objects are created through "Zope code" as opposed to just reading the objects and/or creating objects on the file system and just accessing them through LocalFS objects in Zope. Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Paul Winkler wrote at 2005-1-31 16:50 -0500:
On Mon, Jan 31, 2005 at 07:45:24PM +0100, Dieter Maurer wrote:
I do not think that "LocalFS" is in any way interested to simulate objects in the ZODB. Instead, it emulates a hierarchy apparently rooted in a ZODB based object. But this is purely an acquisition concept and completely independent from the ZODB.
if you look at the LocalFS code (I have but not recently), it actually creates a temporary instance of e.g. OFS.Image.File on every request for a "file". IIRC these are supposed to be thrown away, but apparently something is broken...
But there is no problem to create an instance of a class deriving from "Persistent" as long as this instance is not assigned to another (really) persistent object. As long as you do not do that, the instance has nothing to do with the ZODB. -- Dieter
participants (3)
-
Chris Withers -
Dieter Maurer -
Paul Winkler