Re: WHY ? -- Performace in serving large media files from ZODB vs from external file system
Sedat Yilmazer writes:
In the performance for large image files discussion Dieter Wrote
Both will be quite slow when servered by ZServer.
What could be the reason for being "slow". Since the files are large, the setup time can be ignored ( time to locate the object) Once the object is found only thing that remains is the reading the file from DB ( should be a plain read) and push it to the connection. I do not see a reason for beeing slow on the db read once the data is located inside the DB. I have to admit that I did not make any measurements.
My remark was based on folkore that Zope is about an order of magnitude slower for static pages than Apache. It may be that the factor is lower for large files. However, when I chased errors in "HTTPResponse.HTTPResponse.write" (back in version 2.1.6), I recognized that such a write does not go straight to the HTTP connection but goes through an intermediate file. Thus, your large file is copied once at the server side before it is streamed out to the browser. And there is some Python code around that controls this buffering. Thus, I expect that ZServer is at least twice as slow as direct serving the file. Dieter
From: Dieter Maurer <dieter@handshake.de>
I have to admit that I did not make any measurements.
This is what we need, some real world numbers.
However, when I chased errors in "HTTPResponse.HTTPResponse.write" (back in version 2.1.6), I recognized that such a write does not go straight to the HTTP connection but goes through an intermediate file. Thus, your large file is copied once at the server side before it is streamed out to the browser. And there is some Python code around that controls this buffering. Thus, I expect that ZServer is at least twice as slow as direct serving the file.
I wonder why this temp file is used.... but just because a file is used doesn't make it twice as slow. Either the whole system needs to be analyzed or measured.
marc lindahl writes:
From: Dieter Maurer <dieter@handshake.de> I have to admit that I did not make any measurements.
This is what we need, some real world numbers. Go ahead! ... I wonder why this temp file is used.... but just because a file is used doesn't make it twice as slow. Either the whole system needs to be analyzed or measured. Sure, but it may well do, if you have a really fast network or access it locally.
The most efficient solution: read file in chunks and stream it onto the TCP connection. ZServer's solution: read the file in chunks, write to a temporary file. Read the temporary file in chunks and stream it onto the TCP connection. Additional overhead: Reading and writing the file once. Dieter
On Sat, Jul 07, 2001 at 10:31:32AM +0200, Dieter Maurer wrote:
marc lindahl writes:
From: Dieter Maurer <dieter@handshake.de> I have to admit that I did not make any measurements.
This is what we need, some real world numbers. Go ahead! ... I wonder why this temp file is used.... but just because a file is used doesn't make it twice as slow. Either the whole system needs to be analyzed or measured. Sure, but it may well do, if you have a really fast network or access it locally.
The most efficient solution: read file in chunks and stream it onto the TCP connection.
Yeah! APUE has good examples about what you said! I believe this examples can be found in chapter 3 and/or 4. Another important consideration is to keep block boundaries suitable aligned for even best performance.
ZServer's solution: read the file in chunks, write to a temporary file. Read the temporary file in chunks and stream it onto the TCP connection. Additional overhead: Reading and writing the file once.
Dieter
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
-- Moon, n.: 1. A celestial object whose phase is very important to hackers. See PHASE OF THE MOON. 2. Dave Moon (MOON@MC).
From: Dieter Maurer <dieter@handshake.de>
Go ahead!
I'm willing to contribute... anyone have any favorite numbers? I'll give some stuff a try... I'm actually more interested in size issues than speed issues... considering I just got a 933MHz PIII server brand new from Gateway for $500, my solution to speed is, buy a new machine, it's cheaper than the labor to optimize your software :)
Sure, but it may well do, if you have a really fast network or access it locally.
True, but without a system analysis, who knows if that's a real bottleneck?
The most efficient solution: read file in chunks and stream it onto the TCP connection.
ZServer's solution: read the file in chunks, write to a temporary file. Read the temporary file in chunks and stream it onto the TCP connection. Additional overhead: Reading and writing the file once.
Anyone know what Apache does for this case? And/or why ZServer doesn't tae the most efficient solution?
participants (3)
-
Dieter Maurer -
Gustavo Vieira Goncalves Coelho Rios -
marc lindahl