RE: [Zope] Re: WHY ? -- Performace in serving large media filesfrom ZODB vs from external file system
Hi Zopers In most of the systems ( specially where we have a server and performanace requirements) we have high end systems with fast disk access. In such system the problem most of the time is the CPU power/RAM/Internet Connection Speeds. If you have a system with a 2-8Mbps your basic bottleneck is at the CPU and RAM side, NOT at the Disc IO side. Lets assume that the total response time is setting up the client/server connection + (How is it in ZServer forks? ) request processing + finding the object in ZODB + Reading the object from ZODB + writing the data to temp file + pushing the file to client In this picture writting the file once more to the file system should not be a big part of the picture. In most systems IO is initiated by the CPU and completed by the hardware (DMA). If your channel to HD can not cope with that you must have the fastest Internet connection on the plannet. So as the last point. I would never ever giveup a consistent implementation if the performance inrease is less then %100. HW is so cheap that spending some days to get thing working may cost you alot more. Sedat -----Original Message----- From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of Dieter Maurer Sent: Saturday, July 07, 2001 11:32 AM To: marc lindahl Cc: zope@zope.org Subject: Re: [Zope] Re: WHY ? -- Performace in serving large media filesfrom ZODB vs from external file system marc lindahl writes:
From: Dieter Maurer <dieter@handshake.de> I have to admit that I did not make any measurements.
This is what we need, some real world numbers. Go ahead! ... I wonder why this temp file is used.... but just because a file is used doesn't make it twice as slow. Either the whole system needs to be analyzed or measured. Sure, but it may well do, if you have a really fast network or access it locally.
The most efficient solution: read file in chunks and stream it onto the TCP connection. ZServer's solution: read the file in chunks, write to a temporary file. Read the temporary file in chunks and stream it onto the TCP connection. Additional overhead: Reading and writing the file once. Dieter _______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On Sat, 7 Jul 2001 13:22:49 +0300, "Sedat Yilmazer" <sedat@kibele.com> wrote:
setting up the client/server connection + (How is it in ZServer forks? )
These two steps happen consecutively:
request processing + finding the object in ZODB +
However these are interleaved to a certain extent:
Reading the object from ZODB + writing the data to temp file + pushing the file to client
The total latency before the client recieves the *first* byte is: in the worker thread: * request processing * finding the object * reading the first 64k (iirc) chunk from ZODB * writing that chunk to a temp file via RESPONSE.write then back in the main medusa thread: * reading it back from the temp file * pushing it over the socket * network latency Im not sure how much bouncing via the filesystem affects the total latency for the first byte.. In the majority of cases the time to recieve the *last* byte is given by network bandwidth not server performance, as you suggest. Toby Dickenson tdickenson@geminidataloggers.com
participants (2)
-
Sedat Yilmazer -
Toby Dickenson