[Zope-dev] experiments & bugs with data.fs >2GB

[marc lindahl]

I've recently been testing a new setup with XFS
(http://oss.sgi.com/projects/xfs/1.0_release.html) and decided to try to
bloat data.fs before using this system for production.  The computer is set
up with zope 2.3.3 installed from source (the old fashioned way, with
everything in one directory), with a separate 18GB disk as /usr/local/zope.

I uploaded a couple of files -- seawolf disk one ISO image (about 650MB) and
SGI's XFS install ISO (about 300MB).  Both from another computer via ZMI via
a 10 baseT network.  (interestingly both loaded up OK, but the seawolf image
gave a client side 'page failed to load' error, while the other worked fine.
So I suppose 600MB is the hairy edge of filesize you can upload via ZMI)

Then proceeded to copy the two files and paste.  First time worked fine,
pushing data.fs to about 1.9GB.  Second time, I copied all 4, which should
push data.fs to about 4GB.  It seemed to reach about 2.29GB (perhaps copying
the first XFS ISO image), then gave this error:

--------------------------
               Zope Error

               Zope has encountered an error while publishing this resource.

               Error Type: OverflowError
               Error Value: long int too long to convert



               Troubleshooting Suggestions

                     The URL may be incorrect.
                     The parameters passed to this resource may be
incorrect. 
                     A resource that this resource relies on may be
encountering an error.

               For more detailed information about the error, please refer
to the HTML source for this page.

               If the error persists please contact the site maintainer.
Thank you for your patience.




Traceback (innermost last):
  File /usr/local/zope/Zope-2.3.3/lib/python/ZPublisher/Publish.py, line
223, in publish_module
  File /usr/local/zope/Zope-2.3.3/lib/python/ZPublisher/Publish.py, line
187, in publish
  File /usr/local/zope/Zope-2.3.3/lib/python/Zope/__init__.py, line 221, in
zpublisher_exception_hook
    (Object: Traversable)
  File /usr/local/zope/Zope-2.3.3/lib/python/ZPublisher/Publish.py, line
171, in publish
  File /usr/local/zope/Zope-2.3.3/lib/python/ZPublisher/mapply.py, line 160,
in mapply
    (Object: manage_pasteObjects)
  File /usr/local/zope/Zope-2.3.3/lib/python/ZPublisher/Publish.py, line
112, in call_object
    (Object: manage_pasteObjects)
  File /usr/local/zope/Zope-2.3.3/lib/python/OFS/CopySupport.py, line 229,
in manage_pasteObjects
    (Object: Traversable)
  File /usr/local/zope/Zope-2.3.3/lib/python/OFS/CopySupport.py, line 469,
in _getCopy
    (Object: copy_of_RH7.1-SGI-XFS-1.0.iso)
  File /usr/local/zope/Zope-2.3.3/lib/python/ZODB/ExportImport.py, line 218,
in importFile
  File /usr/local/zope/Zope-2.3.3/lib/python/ZODB/FileStorage.py, line 745,
in tpc_vote
    (Object: /usr/local/zope/Zope-2.3.3/var/Data.fs)
OverflowError: (see above)

--------------------------

Lines 744 and 745 are:

            pos=self._pos
            file.seek(pos)

it looks like file.seek doesn't like the long int?

Also, when going to the UNDO tab get this error:

-----------------------

               Zope Error

               Zope has encountered an error while publishing this resource.

               Error Type: OverflowError
               Error Value: long int too long to convert



               Troubleshooting Suggestions

                     The URL may be incorrect.
                     The parameters passed to this resource may be
incorrect. 
                     A resource that this resource relies on may be
encountering an error.

               For more detailed information about the error, please refer
to the HTML source for this page.

               If the error persists please contact the site maintainer.
Thank you for your patience.




Traceback (innermost last):
  File /usr/local/zope/Zope-2.3.3/lib/python/ZPublisher/Publish.py, line
223, in publish_module
  File /usr/local/zope/Zope-2.3.3/lib/python/ZPublisher/Publish.py, line
187, in publish
  File /usr/local/zope/Zope-2.3.3/lib/python/Zope/__init__.py, line 221, in
zpublisher_exception_hook
    (Object: Traversable)
  File /usr/local/zope/Zope-2.3.3/lib/python/ZPublisher/Publish.py, line
171, in publish
  File /usr/local/zope/Zope-2.3.3/lib/python/ZPublisher/mapply.py, line 160,
in mapply
    (Object: manage_UndoForm)
  File /usr/local/zope/Zope-2.3.3/lib/python/ZPublisher/Publish.py, line
112, in call_object
    (Object: manage_UndoForm)
  File /usr/local/zope/Zope-2.3.3/lib/python/Shared/DC/Scripts/Bindings.py,
line 324, in __call__
    (Object: manage_UndoForm)
  File /usr/local/zope/Zope-2.3.3/lib/python/Shared/DC/Scripts/Bindings.py,
line 354, in _bindAndExec
    (Object: manage_UndoForm)
  File /usr/local/zope/Zope-2.3.3/lib/python/App/special_dtml.py, line 236,
in _exec
    (Object: manage_UndoForm)
  File /usr/local/zope/Zope-2.3.3/lib/python/App/Undo.py, line 164, in
undoable_transactions
    (Object: Traversable)
  File /usr/local/zope/Zope-2.3.3/lib/python/ZODB/UndoLogCompatible.py, line
100, in undoInfo
    (Object: /usr/local/zope/Zope-2.3.3/var/Data.fs)
  File /usr/local/zope/Zope-2.3.3/lib/python/ZODB/FileStorage.py, line 868,
in undoLog
    (Object: /usr/local/zope/Zope-2.3.3/var/Data.fs)
OverflowError: (see above)


-------------------------

seems like the same thing:

                seek(pos-8)


I found one reference to large file support in Python:

http://www.python.org/doc/current/lib/posix-large-files.html

Could it be just that the default install script in the release needs to
enable large file support?  Or....???