[Zope] LocalFS and mega-uploads + CMF

[Jud Dagnall]

Hello-

I've got a similar problem, but would like to be able to use the CMF if
possible.
In my case, I need to have the (large) files end up on a separate FTP server
anyway, but I'd like to use the CMF for submission, testing and review of
files and metadata. Is there a straightforward way to have zope
move/copy/ftp files after review to an external server, or at least to a
centralized directory that could be mirrored?

Note: I'm using the ExtFile product now (without CMF). I set the directory
option to mirror the ZODB, and I rsync the .../approved directory daily.

Thanks,

Jud

--
Jud Dagnall
SGI Electronic Services
jdagnall@sgi.com


-----Original Message-----
From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of
larry_prikockis@NatureServe.org
Sent: Thursday, January 17, 2002 8:47 AM
To: zope@zope.org
Subject: RE: [Zope] LocalFS and mega-uploads


...Just to kind of wrap this thing up... Thanks to those who helped me
figure this out.  As several people have now suggested and confirmed, the
problem was the partition I had /var/tmp on filling up during the upload
process.

Moving /var/tmp to a bigger partition cured the basic problem I was having.
However, transferring of extremely huge files still put more of a load on
the system than I would've liked.  In the end, what I've ended up doing is
setting up a separate standalone FTP server, locking it down as much as
possible and using an html/script front-end to allow some degree of
integration with the rest of my Zope site.

It's not quite as nicely integrated as using Zserver's native http/ftp
capabilities, but it does the job.

> -----Original Message-----
> From: Jim Penny [mailto:jpenny@universal-fasteners.com]
> Sent: Wednesday, January 16, 2002 7:22 PM
> To: larry_prikockis@NatureServe.org
> Subject: Re: [Zope] LocalFS and mega-uploads
>
> The /tmp path hypothesis does seem correct.  I just started a
> megaupload,
> and found this--that space went down as the upload
> progressed, (as reported
> by df -k).  But there were no unusual files in /tmp!
>
> However, looking at lsof's output, I see:
>                                                  bytes
> 						 vvvv
> python1.5 32012     zope   24u   REG        3,1 56111104    448341
> /tmp/@31823.224 (deleted)
>
> So, it looks like cgi.py is being activated, and that it is stuffing
> the upload into a file that it is immediately unlinking.  The upshot
> of this is that the file is not present to prying eyes.  My script
> was then reading it from /tmp in 4k chunks.
>
> This being the case, the best thing to do is to make sure that /tmp
> has a LOT of room available.  You want to make sure that it is mounted
> on a separate partition from the data's destination and preferably
> from anything else.  Otherwise, your hard drive will have to have
> at least twice as much space free (one for the /tmp copy and one for
> the destination copy) as the largest expected upload.
>
> Note, it is fairly conventional to have /tmp mounted on /.  Moreover,
> many administrators keep / intentionally small.  You don't want to
> do this here:  you can keep / small, but only if /tmp is mounted
> elsewhere!

_______________________________________________
Zope maillist  -  Zope@zope.org
http://lists.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope-dev )