[ZODB-Dev] Blobs, Copies, and Exports (Zope 3)

[Jim Fulton]

A few high-level observations. (I don't have time to get into the  
weeds.)

1. The parent pointers used in Zope 3 are, obviously, a problem for  
ZODB exports.  You've already noticed this.
      A possible way to mitigate this would be to extend the export  
API to be able to omit objects from the export.
     Then, when copying a tree, you could tell it to omit the parent  
of the root. There would have to be some way
     to tell it what to use instead.

2. I doubt that blobs have been factored into ZODB exports. This is,  
obviously, an oversight.

3. I think that zope.fssync/zope.app.fssync might provide a better  
export/import technology, although they *may* require more work.  On  
the up site, they are more pluggable. We are, *finally*, about to  
start using zope.fssync in a production application.

Jim

On Oct 24, 2008, at 2:09 AM, Jeff Shell wrote:

> For the past few years we've been using an ugly, but working, trick to
> export data out of a Zope 3 instance without exporting every bit of
> content. Given an object to export, we use
> 'locationCopy' (zope.location.pickling) to get a deep copy, set the
> __parent__ attribute of the copied object to None, put it in the ZODB
> root (root[EXPORT_KEY] = detached_copy), and then commit the
> transaction. This is so that we have the 'oid' and connection needed
> for ZODB's 'exportFile' to work.
>
> If there's a better way of setting `__parent__` to None (or just not
> following the __parent__ ref during Export, without affecting the
> saved state in the database), I'd love to know. But that's an aside.
>
> I exported some content tonight that had Blobs in it. Upon import, I
> noticed that the blob files in the destination were empty (zero-
> length). Looking at the ZEXP file, I could see the BLOBSTART markers,
> but could see no binary data. Tracing down into the export, and then
> just playing around with 'locationCopy' to make other detached copies,
> I noticed that the Blob contents were never copied into the new file.
>
> What can I do here? I'm in the midst of a botched deployment of some
> content updates for a customer. I thought that copying something
> 'containing' a Blob would cause its contents to be copied. If this is
> an exercise for application layer code, how can I get plugged in and
> know when a Blob is being copied inside of a deep copy?
>
> Thanks,
>
> --
> Jeff Shell
> jeff at bottlerocket.net
>
> _______________________________________________
> For more information about ZODB, see the ZODB Wiki:
> http://www.zope.org/Wikis/ZODB/
>
> ZODB-Dev mailing list  -  ZODB-Dev at zope.org
> http://mail.zope.org/mailman/listinfo/zodb-dev

--
Jim Fulton
Zope Corporation