[Zope] Error in Copy/Import during Migration

[Brian Brinegar]

I'm fairly well versed on the inner workings of Zope, but yesterday I've
run into a problem that I can grasp.

First, some background on what I'm doing.

I've got a 200GB Data.fs file which I'm splitting up into multiple
storages. The way I'm doing this is to create a zope instance which
mounts a new storage and the old storage and then a python script
connects to the application and uses a modified version of manage_clone
(which maintains ownership) to copy content into the new storage.

This works correctly for most things, but yesterday I tried to move a
folder with about 3.1GB of stuff in it.

On a side note, I ran into an issue with running out of /tmp space
during the clone. The _getCopy method of CopySourse exports an object to
a tempfile and then imports it to create the copy. When copying
something larger than available tmp space zope crashes. I worked around
this by explicitly setting tempfile.tempdir to somewhere with plenty of
free space. There may be an environment variable to do this as well.
Just an FYI.

Anyway, here's what I see. The copy starts, a tempfile is created (on
the client) and grows to the appropriate size (3.1GB). Then, a
Data.fs.temp appears on the new server and grows to the appropriate
size. However, when my script tries to commit the transaction I get an
EOF error:

Traceback (most recent call last):
  File "./migrate_site.py", line 45, in ?
    transaction.get().commit()
  File
"/var/local/zope/Zope-2.11.4-1/lib/python/transaction/_transaction.py",
line 325, in commit
    self._commitResources()
  File
"/var/local/zope/Zope-2.11.4-1/lib/python/transaction/_transaction.py",
line 427, in _commitResources
    rm.tpc_vote(self)
  File "/var/local/zope/Zope-2.11.4-1/lib/python/ZODB/Connection.py",
line 744, in tpc_vote
    s = vote(transaction)
  File "/var/local/zope/Zope-2.11.4-1/lib/python/ZEO/ClientStorage.py",
line 1004, in tpc_vote
    self._server.vote(id(txn))
  File "/var/local/zope/Zope-2.11.4-1/lib/python/ZEO/ServerStub.py",
line 262, in vote
    return self.rpc.call('vote', trans_id)
  File
"/var/local/zope/Zope-2.11.4-1/lib/python/ZEO/zrpc/connection.py", line
699, in call
    raise inst # error raised by server
EOFError

I don't seem to have any space issues on any devices, and the process
works for smaller site. Any input would be greatly appreciated.

Thanks,

-- 
Brian Brinegar
Web Services Coordinator
Engineering Computer Network