[Zope-dev] Re: [Zope3-dev] proposal: serving static content faster

[zope at netchan.cotse.net]

> In fact, Zope puts large files (the threshold is around 256K - 512K) 
> into a temporary file before serving them, to free up application 
> threads.  It's a tremendous handicap.

I'm working on a product which serves files from the filesystem. The
data retrieval method is the usual:

def pushData(self, f, outstream):
    finished = False
    while not finished:
        block = f.read(blocksize)
        if len(block) < blocksize:
            finished = True
        outstream.write(block)

f is the file on the filesystem, outstream is the request object.

Testing with a 1Mbyte file (ab -n 12 -c 4), I get ~4.4 req/sec -
~4.7Mbyte/sec after a few iterations (the os caches the file).

Now, I change this method to the following:

def pushData(self, f, outstream, data='0'*65536):
    for i in range(16):
        outstream.write(data)

The test results are the same.

Now, if I disable the temporary file thing in ZServer/HTTPResponse.py
the performance goes up to around 6.9 req/sec - ~7Mbyte/sec. If I
restore my pushData method to it's original form it can still do ~6.2
req/sec - ~6.6Mbyte/sec.
In this case, practically every disk operation was served from the
cache.

It seems from these results, that ZServer's tempfile strategy causes
some (~20% if everything is cached) performance hit, but I think that
there should other bottleneck(s) beside this.

Regards,
Sandor