[ZODB-Dev] Blobs, Copies, and Exports (Zope 3)
Christian Theune
ct at gocept.com
Fri Oct 24 18:10:59 EDT 2008
On Fri, 2008-10-24 at 11:45 -0600, Jeff Shell wrote:
> On Oct 24, 2008, at 8:36 AM, Christian Theune wrote:
> > Again: blobs export as expected.
> >
> > However, Jeff pickles/unpickles the subtree to get rid of the
> > '__parent__' attribute. This and blobs' inability to get copied around
> > in this specific manner make him end up with empty blobs.
>
>
> This is the real issue. What can be done to address it? This sounds
> like something that should be handled at/near the ZODB/Pickle/Copy
> (deep copy?) level.
>
> While doing some google searches on this issue, I saw others having
> same or similar concerns when using Blobs with other tools that did
> deep copies of objects, although their bugs seemed related more to the
> issue of having 'None' values in the readers and writers attributes of
> the Blob file (a bug which has been fixed).
>
> The initial traceback in this bug report is tied to that issue:
>
> https://bugs.launchpad.net/zope3/+bug/240381
>
> But then things swing around to actual content copying. Christophe
> Combelles last comment (from June) shows manually patching
> 'zope.location....locationCopy' to address his particular situation.
> 'loc' is the original 'located' object being copied.
>
> copied = unpickler.load()
> if hasattr(copied, '_data') and type(copied._data) is Blob:
> copied._data._create_uncommitted_file()
> targetfile = open(copied._data._current_filename(), 'w')
> sourcefile = loc.openDetached()
> chunk = sourcefile.read(1000000)
> while chunk:
> targetfile.write(chunk)
> chunk = sourcefile.read(1000000)
> targetfile.close()
> return copied
>
> That's obviously not a general solution, or the most efficient, but
> I'm guessing it's pretty close to what needs to happen...
>
> I just verified that with our CMS (built on Zope 3) and custom
> 'BlobFile' content type object, copying the BlobFile content type
> object copies all of the other persistent data of the object, but the
> Blob contents are indeed empty. This is using a copy/paste system
> built on top of zope.copypastemove.
There are two general approaches that I can imagine:
or a blob that is a copy of another, keep the original oid/tid pair
around from which it was copied implementing 'copy on write' semantics.
This has packing issues from what I can see.
The other would be to create a hard link to the file that matched the
content at the given point in time. This has transaction race conditions
from what I see.
I'd like to avoid copies though.
--
Christian Theune · ct at gocept.com
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1
Zope and Plone consulting and development
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : http://mail.zope.org/pipermail/zodb-dev/attachments/20081025/1b60afa9/attachment.bin
More information about the ZODB-Dev
mailing list