[Zope-dev] Adding gzip compression to HTTPResponse.py

[Adrian Hungate]

I have been working on a compression patch as a component of my PatchKit
product for some time now, but I have been hitting a few problems, and had
put it to one side while I worked on a couple of other products. If people
still want to see something like this, I can bring the code out of
moth-balls and start working on the problems.

Adrian...

--
The difficulty of tactical maneuvering consists in turning the devious into
the direct, and misfortune into gain.
- Sun Tzu


----- Original Message -----
From: "Brad Clements" <bkc@murkworks.com>
To: <zope-dev@zope.org>
Sent: Tuesday, February 05, 2002 10:34 PM
Subject: [Zope-dev] Adding gzip compression to HTTPResponse.py


> I'm looking for architectural suggestions for adding gzip compression to
> HTTPResponse for text types.
>
> First, I just wanted to compress xml-rpc output, since I'm returing lots
of table data as
> XML text (not objects), then loading that text/xml into a DOM for XSLT
processing.
>
> I hacked the attached code into HTTPResponse, at the end of setBody. It
works for
> xml-rpc responses and I suppose any text output, so long as the response
object has a
> header named "dogzip" set.
>
> I know "dogzip" is a stupid name, but this is just a testing thing.
>
> Representative compressions:
>
> compress oldlen  150366 new len 11926
> compress oldlen  204382 new len 14170
> compress oldlen  12746 new len 1364
>
> As you can see, very useful compressions for xml-rpc output.
>
> But for HTML output, what's really needed is I think a special kind of
Cache Object.
> One that combines HTTP Caching with Ram caching to keep gzip compressed
objects
> "in memory".
>
> Some HTML pages are really quite large, and gzip compression can make a
noticable
> difference. Just the javascript code sizes themselves are .. really big.
>
> For xml-rpc, obviously every response must be compressed if it's "worth
it", and I can
> see that having to set a response property on a per request basis is
appropriate for
> xml-rpc.
>
> But for text file objects, Page Templates and stuff.. How does setBody
work with Ram
> Cache objects? I have some ideas...
>
> Anyone think this is worthwhile?
>
> Also, RESPONSE.setBody really should have access to REQUEST.headers.
What's
> the clean way to do that? Just pass the request object to response
object's init
> method?
>
> Here's quick gzip compression hack-in, based on code posted by Neil
Schemenauer
>
> Thanks Neil.
>
> Added about line 265 in HTTPResponse.py in Zope 2.5 B3
>
>         try:
>             dogzip = self.headers['dogzip']
>             del self.headers['dogzip']
>             if dogzip and split(content_type,'/')[0] == 'text':
>                 body = self.body
>                 startlen = len(body)
>                 import zlib, struct
>                 _gzip_header = ("\037\213" # magic
>                                 "\010" # compression method
>                                 "\000" # flags
>                                 "\000\000\000\000" # time
>                                 "\002"
>                                 "\377")
>                 co = zlib.compressobj(6,zlib.DEFLATED,-zlib.MAX_WBITS,
>                                       zlib.DEF_MEM_LEVEL,0)
>                 chunks = [_gzip_header, co.compress(body),
>
co.flush(),struct.pack("<ll",zlib.crc32(body),startlen)]
>                 z = join(chunks,"")
>                 newlen = len(z)
>                 print "compress oldlen ",startlen,"new len",newlen
>                 if newlen < startlen:
>                     self.body = z
>                     self.setHeader('content-length', newlen)
>                     self.setHeader('content-encoding','gzip')
>         except:
>             pass
>
>
>
> Brad Clements,                bkc@murkworks.com   (315)268-1000
> http://www.murkworks.com                          (315)268-9812 Fax
> netmeeting: ils://ils.murkworks.com               AOL-IM: BKClements
>
>
> _______________________________________________
> Zope-Dev maillist  -  Zope-Dev@zope.org
> http://lists.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists -
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope )
>