[Zope] Using Zope to manage and serve large files

Fri Oct 31 17:21:06 EST 2003

On Fri, 31 Oct 2003 23:47:36 +0200
Heimo Laukkanen <huima at iki.fi> wrote:

> Often when my customers have got used to Zope and especially Plone as 
> the UI to manage their website, questions go to using Zope and Plone
> for document management in intranet. It is old story that we all know
> how Zope sucks while serving large files compared to Apache. Serving
> large files from filesystem with Apache however takes out the whole
> point of using Zope - security mechanism etc.
> 
> Has anyone achieved to combine Zope and Apache/php to serve and manage
> 
> large files in a way that allows metadata and security of the files to
> 
> be managed in Zope, but actual serving happens from Apache so that 
> ZServer does not hog memory and processor.
> 

Well, the main problem is simply making sure that the customer cannot
possibly guess the filename.  So, use the Secure Hash Algorithm (SHA)
and your own site-based secrets.

That is, given a filename, calculate SHA(secret_1 + file_name +
customer_name + secret_2).  Save the file in a customer specific
(apache accessible) directory, using the SHA as the filename.  Then put
a dummy index.html in that folder, something like 
<html><head></head><body>No Peeking!</body></html>

Now, the customer something like 1 in 2^160 chance of finding the file
by probing; you have something that is fast and fairly cheap to
calculate.  The only practical way to get to a file is by something that
knows the secrets, and can do the right calculation.  Make this
calculation a part of zope, and you have your security bottleneck.
That is, use zope to authenticate and authorize, calculate the SHA, and
present either a direct link, or redirect to the actual file.

You do have to worry about the site secrets.  If they are
ever exposed, you would have to rehash the names of all files, but that
is not too big a deal to do periodically, anyway.  It does mean that you
need to keep a database of customer file names.  Do not put the SHA of
associated with the file name in the database.  Calculate from scratch
every time.

Jim Penny