[Zope] TAL page whitespace removal

[Floyd May]

Robert (Jamie) Munro wrote:
> Dieter Maurer wrote:
>> Robert (Jamie) Munro wrote at 2006-4-10 13:14 +0100:
>>> ....
>>> For example, I downloaded the www.plone.org home page and it was 47704
>>> bytes. I removed all whitespace from the beginning and end of lines, and
>>> then removed blank lines, and it was down to 35087 bytes - that's a more
>>> than 25% saving, and the output renders exactly the same in web
>>> browsers.
>> A much more efficient way is to activate "gzip" compression
>> for your pages. It not only handles white space efficiently
>> (and correct for e.g. pages with "pre" tags or similar CSS directives)
>> it also compresses other text. You not only gain 25 % but about 70 %
>> (in the size of requests).
> 
> gzip will add enormous processing overhead to the server. Striping
> spaces will add negligible overhead, likely less overhead than it saves.

Doubtful.  If high-throughput servers can encrypt and decrypt SSL 
traffic, gzip isn't going to be a problem.  The difference between 
running gzip versus htmltidy shouldn't even be significantly noticeable 
on a sufficiently-powered server.  I know that super-high throughput 
sites (e.g. Google) intentionally break some standards to save bandwidth 
(where omitting '</body></html>' could save gigabytes per day), but if 
you're in a situation like that, Zope probably isn't for you anyway.

>> You will not save RAM on the server side (of course), but
>> hopefully (and very likely) the scripts are not the major
>> user of RAM on your site.
> 
> I have written TAL that produces very large dumps of XML data in the
> past, even whole sites. It's a really nice way to dump data from a
> database (SQL or Zope DB), but Zope has to build the whole output in RAM
> before sending any of it, so it can cause the site to crash.

One solution I've found is to buffer the writes to REQUEST.RESPONSE by 
using a python script which the calls granular page templates rather 
than a single monolithic template, and outputting the results 25k at a 
time or so; it gives the rest of the server some time to catch up. 
However, the point you bring up has nothing to do with whether or not 
the output has significant whitespace in it -- double or triple the 
amount of data sent, and you're still in the same boat.  If you're this 
concerned about bandwidth, you're probably using the wrong tool for the job.

> I would
> hope in this kind of case that the TAL is the major user of RAM on the
> site, so any saving would be really good, but it all cases (except <pre>
> tags, which I would never use) it seems like a possibly significant gain.

There's a reason for TAL being rendered and stored before it's sent. 
Zope is an *Application Server*.  If there's an error in rendering, it 
gives the server an opportunity to handle this case gracefully in an 
*Application-Specific* way, rather than sending a half-finished page to 
the browser followed by an error message that will be rendered 
God-knows-how to the user.  If you render & store before sending, you 
can catch rendering errors and handle them gracefully.

-- 
Floyd May
Senior Systems Analyst
CTLN - CareerTech Learning Network
fmay at okcareertech.org