[Zope] LocalFS and mega-uploads

[larry_prikockis@NatureServe.org]

Jim Penny jpenny@universal-fasteners.com  on Thu, 10 Jan 2002 16:51:37 -0500


	I  use an external method to do my upload, this is the salient part

	def upload(self, REQUEST):
	... code deleted ...

	The largest file I currently have here is 104,486,083 bytes.  This
is
	quite a bit bigger than my server's RAM!

	Jim Penny

On Thu, Jan 10, 2002 at 04:09:30PM -0500, larry_prikockis@NatureServe.org
wrote:
> ...from the "everything would be perfect if it weren't for those darn
users"
> category...
> 
> I'm running Zope 2.4.3 on Linux (RedHat 7.1 / Intel) and using the LocalFS
> product to allow users to easily upload/download files from their browser.
> For "normal sized" files (up to a few dozen Meg.) everything works as
> expected.  However, with bigger files- in particular a 172 Meg file
someone
> tried to upload- the system grinds to a halt as the server memory gets
> chewed up.  
> thanks--
> Larry

I've tried a number of different experiments (including using the upload
code kindly provided by Jim Penny above) with large files (my current test
case is a 170 Meg .ZIP file) and generally gotten the same frustrating
results.  Here's the traceback I get (after a minute or so of churning away
at the process)

----------
Error: [Errno 28] No space left on device and Traceback (innermost last):
  File /home/zoper/Zope-2.4.1-linux2-x86/lib/python/ZPublisher/Publish.py,
line 136, in publish
  File
/home/zoper/Zope-2.4.1-linux2-x86/lib/python/ZPublisher/HTTPRequest.py, line
409, in processInputs
  File /home/zoper/Zope-2.4.1-linux2-x86/lib/python2.1/cgi.py, line 517, in
__init__
  File
/home/zoper/Zope-2.4.1-linux2-x86/lib/python/ZPublisher/cgi_hotfix.py, line
20, in read_multi
  File /home/zoper/Zope-2.4.1-linux2-x86/lib/python2.1/cgi.py, line 519, in
__init__
  File /home/zoper/Zope-2.4.1-linux2-x86/lib/python2.1/cgi.py, line 616, in
read_single
  File /home/zoper/Zope-2.4.1-linux2-x86/lib/python2.1/cgi.py, line 638, in
read_lines
  File /home/zoper/Zope-2.4.1-linux2-x86/lib/python2.1/cgi.py, line 677, in
read_lines_to_outerboundary
IOError: [Errno 28] No space left on device

---------
I'm particularly puzzled by the "No space left on device" part... I have
several GB of free disk space, so that's not the issue.  Just for fun, I
tried this on another machine with 512Meg RAM both with Zope 2.4.3 and the
latest 2.5 beta... same results...

if this due to a bug in either cgi.py or somewhere else in the Zserver code?

I guess my solution for handling these extra-jumbo files will be to run a
sep. FTP server to allow uploading of the files.

I seem to be having more trouble with this than most other users, so I'm
wondering if there's still something silly I'm missing about the situation.
If the above traceback rings any bells for anyone, I'd love to compare
notes.

Larry