Hello everyone, I write this mail, because I could not find anything useful in the archives: I have a pretty big site with quiet a big database (PostGreSQL) in the background. When a site loads I can tell that after about 5 seconds the site is ready to download. The site downloads quickly thereafter if I am on a T1, DSL or Cable Modem connection. But since most of our users have 28.8k and 56k modem connection, we have to optimize for them too. When accessing the site over a modem, it takes a long time to load the site and then it pops up all at once (in IE (AOL) and Netscape). Is there a way that I can send parts of the HTML as it is generated, so that the customer starts seeing information before the entire site is loaded. Our site is about 50% slower than our competitors sites (we are serving 62kB and the competitor has up to 203kB and is 50% faster!!!) which use PHP and ColdFusion. I know that Zope is not slower. I strongly believe it is the HTML output which is not optimized (we are going to speed up the DB connection very soon by dedicating a NIC only for the DB communication). Technical Facts: - Zope 2.1.6+PCGI+Apache - Virtual Hosts for HTTP and HTTPS - PostGreSQL 7.0 (DB size: 50MB), ZPyGreSQLDA, UserDB - ZODB threads: 4 - Web Server: Pentium 500, RH 6.1, 322MB RAM (not all used), 100 MBit NIC - DB Server: Pentium 500, RH 6.1, 322MB RAM (plenty left to use), 100 MBit NIC - dedicated subnet If someone could give me some tips where I should start looking for speed holes, please let me know. Do you think it is the DB which could be so slow and I should cache search results more, are there ZServer/Apache options I can set or anything else in Zope I should have a look at? Regards, Stephan -- Stephan Richter CBU - Physics and Chemistry Web2k - Web Design/Development & Technical Project Management
Stephan Richter wrote:
I have a pretty big site with quiet a big database (PostGreSQL) in the background. When a site loads I can tell that after about 5 seconds the site is ready to download. The site downloads quickly thereafter if I am on a T1, DSL or Cable Modem connection. But since most of our users have 28.8k and 56k modem connection, we have to optimize for them too. When accessing the site over a modem, it takes a long time to load the site and then it pops up all at once (in IE (AOL) and Netscape). Is there a way that I can send parts of the HTML as it is generated, so that the customer starts seeing information before the entire site is loaded.
At the HTML level, are your pages in one big table? If so, Netscape will often need to wait until the whole contents of the table is downloaded before attempting to render it. To get around this, you need to specify the size of just about **everything** in the table. I think it is the same with IE too. Mozilla doesn't have that problem, with its incremental layout. Can you see the stream of data coming if you telnet to your server and type GET / HTTP/1.0 or whatever? Is it just that the rendering is slow because some part of the page is slow to load? -- Steve Alexander Software Engineer Cat-Box limited http://www.cat-box.net
At the HTML level, are your pages in one big table? If so, Netscape will often need to wait until the whole contents of the table is downloaded before attempting to render it.
To get around this, you need to specify the size of just about **everything** in the table.
No, we do not do that because of the NS 4.x pain.
I think it is the same with IE too. Mozilla doesn't have that problem, with its incremental layout.
No, IE builds everything as it comes in, like Mozilla.
Can you see the stream of data coming if you telnet to your server and type GET / HTTP/1.0 or whatever? Is it just that the rendering is slow because some part of the page is slow to load?
You got the hit. It does not stream the page out. It sits there and then suddenly pushes the entire site out. Do you know what settings I would need to tweak to fix that? Regards, Stephan -- Stephan Richter CBU - Physics and Chemistry Web2k - Web Design/Development & Technical Project Management
Stephan Richter wrote:
Can you see the stream of data coming if you telnet to your server and type GET / HTTP/1.0 or whatever? Is it just that the rendering is slow because some part of the page is slow to load?
You got the hit. It does not stream the page out. It sits there and then suddenly pushes the entire site out. Do you know what settings I would need to tweak to fix that?
lib/python/ZPublisher/BaseResponse.py def write(self,data): """\ Return data as a stream HTML data may be returned using a stream-oriented interface. This allows the browser to display partial results while computation of a response to proceed. The published object should first set any output headers or cookies on the response object. Note that published objects must not generate any errors after beginning stream-oriented output. """ Looks like it's more that just a tweak, though :-( -- Steve Alexander Software Engineer Cat-Box limited http://www.cat-box.net
On Sat, Jul 15, 2000 at 12:18:23PM -0700, Stephan Richter wrote:
I have a pretty big site with quiet a big database (PostGreSQL) in the background. When a site loads I can tell that after about 5 seconds the site is ready to download. The site downloads quickly thereafter if I am on a T1, DSL or Cable Modem connection. But since most of our users have 28.8k and 56k modem connection, we have to optimize for them too. When accessing the site over a modem, it takes a long time to load the site and then it pops up all at once (in IE (AOL) and Netscape). Is there a way that I can send parts of the HTML as it is generated, so that the customer starts seeing information before the entire site is loaded. Our site is about 50% slower than our competitors sites (we are serving 62kB and the competitor has up to 203kB and is 50% faster!!!) which use PHP and ColdFusion. I know that Zope is not slower. I strongly believe it is the HTML output which is not optimized (we are going to speed up the DB connection very soon by dedicating a NIC only for the DB communication).
Technical Facts: - Zope 2.1.6+PCGI+Apache - Virtual Hosts for HTTP and HTTPS - PostGreSQL 7.0 (DB size: 50MB), ZPyGreSQLDA, UserDB - ZODB threads: 4 - Web Server: Pentium 500, RH 6.1, 322MB RAM (not all used), 100 MBit NIC - DB Server: Pentium 500, RH 6.1, 322MB RAM (plenty left to use), 100 MBit NIC - dedicated subnet
If someone could give me some tips where I should start looking for speed holes, please let me know. Do you think it is the DB which could be so slow and I should cache search results more, are there ZServer/Apache options I can set or anything else in Zope I should have a look at?
Hi Stephan, This sounds like more of a HTML problem than a Zope problem, especially since the fast links have no trouble with the server. Zope obviously is fast enough to serve them. I bet your site heavily uses tables and images This will cause the browser to not display anything until it knows how to lay out the table. If your whole page is contained in one table, this means that your page won't pop up until the browser has seen the table end tag and knows the sizes of all images. You can help the browser a bit by making sure all IMG tags have a width and height attribute so the browser won't have to wait for the images to start to load. IIRC, Internet Explorer will even show the table without that information, and reflow the table if needed. It still will wait until it has the whole table. Only Mozilla reflows all incoming HTML on the fly, even before the end of the table is seen. When they the optimisations come in for Netscape 6 preview 3, it'll beat the hell out of Internet Explorer with this. So, if you want to make sure that your clients see something before all HTML is in, split up the table into a header and the rest, if possible. Then the browser has something to show while loading the rest of the page. This is what sites like www.cnet.com do; they show you a banner while waiting. -- Martijn Pieters | Software Engineer mailto:mj@digicool.com | Digital Creations http://www.digicool.com/ | Creators of Zope http://www.zope.org/ | ZopeStudio: http://www.zope.org/Products/ZopeStudio -----------------------------------------------------
<snip dicussion about large tables>
I take that back in the previous mail to Steve about the large table. We have one and I split the header from the rest as you suggested. But since it is not streaming the information, it will still pop up all at once. Any ideas? Steve just send me a file and a method but I do not understand what to get from it. I am checking the iste, whether I call RESPONSE.XXX somewhere in the middle of the page. Regards, Stephan PS: I contacted Illiad about an autogram and I will send the books to him to sign them and have the second one send back to you. Please send me your mailing address. -- Stephan Richter CBU - Physics and Chemistry Web2k - Web Design/Development & Technical Project Management
On Sat, Jul 15, 2000 at 01:19:33PM -0700, Stephan Richter wrote:
<snip dicussion about large tables>
I take that back in the previous mail to Steve about the large table. We have one and I split the header from the rest as you suggested. But since it is not streaming the information, it will still pop up all at once.
Any ideas? Steve just send me a file and a method but I do not understand what to get from it. I am checking the iste, whether I call RESPONSE.XXX somewhere in the middle of the page.
Because of transactions, Zope will build up the whole page before sending anything, regardless of what the page does. But the fact that clients on thick pipes don't have any latency problems, I still don't think Zope is to blame. Once Zope is done, ZServer get's the task of pushing the data to the client, and it should just be able to incrementily render (if so designed) if this data trickles over a slow, thin pipe. -- Martijn Pieters | Software Engineer mailto:mj@digicool.com | Digital Creations http://www.digicool.com/ | Creators of Zope http://www.zope.org/ | ZopeStudio: http://www.zope.org/Products/ZopeStudio -----------------------------------------------------
On Sat, 15 Jul 2000 13:19:33 -0700, Stephan Richter <srichter@cbu.edu> wrote:
I take that back in the previous mail to Steve about the large table. We have one and I split the header from the rest as you suggested. But since it is not streaming the information, it will still pop up all at once.
What you describe is true: Zope will not start sending the page until it is finished calculating. If one bit of your page takes a long time to calculate then the user will see *nothing* until that is complete. However, I don't think this limitation is affecting you. You say the page displays quickly when viewed over DSL - the connection bandwidth doesnt affect how long the page takes to calculate! A modem user will receive the first few bytes equally quickly - it just takes them longer the receive the rest. Toby Dickenson tdickenson@geminidataloggers.com
participants (4)
-
Martijn Pieters -
Stephan Richter -
Steve Alexander -
Toby Dickenson