[Zope-dev] Adding gzip compression to HTTPResponse.py

[seb bacon]

I don't have much useful to add - I just wanted to mention that I know
there are people out there who have succesfully used mod_gzip with Zope;
and that I *like* the name dogzip :-)

seb

On Tue, 2002-02-05 at 22:34, Brad Clements wrote:
> I'm looking for architectural suggestions for adding gzip compression to 
> HTTPResponse for text types.
> 
> First, I just wanted to compress xml-rpc output, since I'm returing lots of table data as 
> XML text (not objects), then loading that text/xml into a DOM for XSLT processing.
> 
> I hacked the attached code into HTTPResponse, at the end of setBody. It works for 
> xml-rpc responses and I suppose any text output, so long as the response object has a 
> header named "dogzip" set.
> 
> I know "dogzip" is a stupid name, but this is just a testing thing.
> 
> Representative compressions:
> 
> compress oldlen  150366 new len 11926
> compress oldlen  204382 new len 14170
> compress oldlen  12746 new len 1364
> 
> As you can see, very useful compressions for xml-rpc output.
> 
> But for HTML output, what's really needed is I think a special kind of Cache Object. 
> One that combines HTTP Caching with Ram caching to keep gzip compressed objects 
> "in memory".
> 
> Some HTML pages are really quite large, and gzip compression can make a noticable 
> difference. Just the javascript code sizes themselves are .. really big.
> 
> For xml-rpc, obviously every response must be compressed if it's "worth it", and I can 
> see that having to set a response property on a per request basis is appropriate for 
> xml-rpc.
> 
> But for text file objects, Page Templates and stuff.. How does setBody work with Ram 
> Cache objects? I have some ideas...
> 
> Anyone think this is worthwhile?
> 
> Also, RESPONSE.setBody really should have access to REQUEST.headers. What's 
> the clean way to do that? Just pass the request object to response object's init 
> method?
> 
> Here's quick gzip compression hack-in, based on code posted by Neil Schemenauer
> 
> Thanks Neil.
> 
> Added about line 265 in HTTPResponse.py in Zope 2.5 B3
> 
>         try:
>             dogzip = self.headers['dogzip']
>             del self.headers['dogzip']
>             if dogzip and split(content_type,'/')[0] == 'text':
>                 body = self.body
>                 startlen = len(body)
>                 import zlib, struct
>                 _gzip_header = ("\037\213" # magic
>                                 "\010" # compression method
>                                 "\000" # flags
>                                 "\000\000\000\000" # time
>                                 "\002"
>                                 "\377")
>                 co = zlib.compressobj(6,zlib.DEFLATED,-zlib.MAX_WBITS,
>                                       zlib.DEF_MEM_LEVEL,0)
>                 chunks = [_gzip_header, co.compress(body),
>                           co.flush(),struct.pack("<ll",zlib.crc32(body),startlen)]
>                 z = join(chunks,"")
>                 newlen = len(z)
>                 print "compress oldlen ",startlen,"new len",newlen
>                 if newlen < startlen:
>                     self.body = z
>                     self.setHeader('content-length', newlen)
>                     self.setHeader('content-encoding','gzip')
>         except:
>             pass
> 
> 
> 
> Brad Clements,                bkc@murkworks.com   (315)268-1000
> http://www.murkworks.com                          (315)268-9812 Fax
> netmeeting: ils://ils.murkworks.com               AOL-IM: BKClements
> 
> 
> _______________________________________________
> Zope-Dev maillist  -  Zope-Dev@zope.org
> http://lists.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists - 
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope )