...Just to kind of wrap this thing up... Thanks to those who helped me figure this out. As several people have now suggested and confirmed, the problem was the partition I had /var/tmp on filling up during the upload process. Moving /var/tmp to a bigger partition cured the basic problem I was having. However, transferring of extremely huge files still put more of a load on the system than I would've liked. In the end, what I've ended up doing is setting up a separate standalone FTP server, locking it down as much as possible and using an html/script front-end to allow some degree of integration with the rest of my Zope site. It's not quite as nicely integrated as using Zserver's native http/ftp capabilities, but it does the job.
-----Original Message----- From: Jim Penny [mailto:jpenny@universal-fasteners.com] Sent: Wednesday, January 16, 2002 7:22 PM To: larry_prikockis@NatureServe.org Subject: Re: [Zope] LocalFS and mega-uploads
The /tmp path hypothesis does seem correct. I just started a megaupload, and found this--that space went down as the upload progressed, (as reported by df -k). But there were no unusual files in /tmp!
However, looking at lsof's output, I see: bytes vvvv python1.5 32012 zope 24u REG 3,1 56111104 448341 /tmp/@31823.224 (deleted)
So, it looks like cgi.py is being activated, and that it is stuffing the upload into a file that it is immediately unlinking. The upshot of this is that the file is not present to prying eyes. My script was then reading it from /tmp in 4k chunks.
This being the case, the best thing to do is to make sure that /tmp has a LOT of room available. You want to make sure that it is mounted on a separate partition from the data's destination and preferably from anything else. Otherwise, your hard drive will have to have at least twice as much space free (one for the /tmp copy and one for the destination copy) as the largest expected upload.
Note, it is fairly conventional to have /tmp mounted on /. Moreover, many administrators keep / intentionally small. You don't want to do this here: you can keep / small, but only if /tmp is mounted elsewhere!