[Zope3-Users] Copy / Move ZODB Content From "Dev" Zope3Instance to "Production" Zope3 Instance?

[Jeremy Roberts]

Christopher Warner wrote:
> On Tue, 2008-01-22 at 18:16 +0100, Brian Sutherland wrote:
>> On Tue, Jan 22, 2008 at 12:08:10PM -0500, Christopher Warner wrote:
>>>> We do something similar but with our Zope2 instances. With Zope3 we
>>>> don't have a ZODB and do all distribution using Debian packages.
>>> So how does this answer the question? He's talking about ZODB instances
>>> themselves without having to use Postgres or any other data solution
>>> other than the Data.fs itself.
>>>
>>> My recommendation is to either export all of the data that you want as
>>> object and do a mass import. Or to simply copy around the Data.fs which
>>> would probably not be the best solution but unless someone responds with
>>> a better way :-); the only solution.
>> Perhaps you could have 2 Data.fs in your instance mounted at different
>> points. One for code and the other for data. Then you just copy the code
>> Data.fs around and leave the data Data.fs in place. 
> 
> Eventually you'd have to create data to test the code. I guess the real
> issue is once you're complete with all of the code if you should begin a
> move into production to input the data there. The only problem with this
> is that doing so really defeats the purpose of production. I guess it
> would make sense to have a "stage to production" sort of setup where
> before you hit production you have it in staging for data entry. Then
> from there push it into production.
> 
> Thanks,
> Christopher Warner



Thanks for the input Chris, and Brian.

It's true, I am looking for an easy way to move both content space 
objects and software space objects between ZODB storages.

I'd considered simply copying Data.fs files but I learned that there is 
a small but possible risk of making a copy while a transaction is 
incomplete, risking data integrity.

I then stumbled upon the repozo.py script which can safely handle making 
copies of Data.fs files while the database is in use. Unfortunately it 
doesn't seemed to be included in my linux Zope 3.3.1 install.

I am only just starting to play with eggs, but I did find that the 
repozo.py script is packaged with a recent ZODB3-3.8.0b3 egg install.

Even so, this approach (replicating entire Data.fs files) affords no 
control over the objects that are transfered between storages - it's all 
or nothing. This is not a deal breaker for many types of development 
situations, but I anticipate finding myself in other types of situations 
where it would be useful to be able to be able to transfer a few objects 
here and a few objects there - even if it required some premeditation on 
my part in terms of the z3 components available to me in the instances 
to make it possible.

My needs stem from my desire to provide the client with a dev instance, 
a production instance, and a sane controlled process for moving not only 
file system code (easy: darcs), but also content objects in ZODB or ZEO 
storage between the instances.


An illustration may, um... illustrate:

App is developed on the dev instance.
App works great on dev. woo!
App is launched to production using Data.fs replication.
New feature is requested.
Development of new feature proceeds on dev instance. (Meanwhile - site 
users are making new content objects on production.)
Client tests new feature by adding the site content objects to the dev 
instance ZODB that the new feature was designed to allow to be added.
New feature works great on dev, new content is on dev, now it's time to 
launch...

What now?
I have new content objects on production and dev.

This is all part of my desire to think responsibly about my development 
processes that involve my dev instance, a dev instance for the clients 
team of technologists, and a production instance.

-Jer

PS Is there a way to mount multiple Data.fs files in a Zope3 instance 
via configuration? I've only seen it done with code in the debugger prompt.

I have found description of how to acheive this with configuration in Z2 
(mount-point directive in zope.conf's zodb element), but not Z3...

If I had that capability, the only problem left for me would be the 
reference vs deep copy problem which occurs when using copy operations 
through the ZMI between storages. I had a friend recently point me 
towards the python __deepcopy__ api which I haven't looked at yet, and 
zc.copy.

As I know more...