RFC: ETag support for zope.browserresource
I've added ETag support for zope.browserresource in a branch: http://zope3.pov.lt/trac/changeset/115596 Does anybody have any comments/objections? If not, I'd like to merge this to trunk and release zope.browserresource 3.11.0. Marius Gedminas -- http://pov.lt/ -- Zope 3/BlueBream consulting and development
On Monday, August 09, 2010, Marius Gedminas wrote:
Does anybody have any comments/objections? If not, I'd like to merge this to trunk and release zope.browserresource 3.11.0.
Looks good to me. Regards, Stephan -- Entrepreneur and Software Geek Google me. "Zope Stephan Richter"
On Mon, Aug 9, 2010 at 3:25 PM, Marius Gedminas <marius@gedmin.as> wrote:
I've added ETag support for zope.browserresource in a branch: http://zope3.pov.lt/trac/changeset/115596
Does anybody have any comments/objections? If not, I'd like to merge this to trunk and release zope.browserresource 3.11.0.
+1 as long as there's a way to disable or configure how it's computed. There's some information about issues with ETags here: http://developer.yahoo.com/performance/rules.html#etags I see that your implementation uses last-modified + size, which should generally be fine. However if you're load-balancing across two different servers and the timestamps don't match then the ETag is useless. On a completely different note, I see that the File object reads the whole file into memory. Hum. Maybe RAM is cheaper than Disk these days and it doesn't matter, but reading whole files into memory generally raises a red flag for me. hoping-no-one-is-serving-iso-files-through-zope.browserresource-ly yours, -- Sidnei
On 08/10/2010 12:37 AM, Sidnei da Silva wrote:
On Mon, Aug 9, 2010 at 3:25 PM, Marius Gedminas<marius@gedmin.as> wrote:
I've added ETag support for zope.browserresource in a branch: http://zope3.pov.lt/trac/changeset/115596
Does anybody have any comments/objections? If not, I'd like to merge this to trunk and release zope.browserresource 3.11.0.
+1 as long as there's a way to disable or configure how it's computed. There's some information about issues with ETags here:
http://developer.yahoo.com/performance/rules.html#etags
I see that your implementation uses last-modified + size, which should generally be fine. However if you're load-balancing across two different servers and the timestamps don't match then the ETag is useless.
On a completely different note, I see that the File object reads the whole file into memory. Hum. Maybe RAM is cheaper than Disk these days and it doesn't matter, but reading whole files into memory generally raises a red flag for me.
IOPS aren't cheap nowadays. The IOPS/MB have had the issue that the MB kept increasing a lot in disks but the IOPS didn't catch up that much, so it may not even be a RAM issue but throughput/caching/IOPS. -- Christian Theune · ct@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development
On Mon, Aug 09, 2010 at 07:37:30PM -0300, Sidnei da Silva wrote:
On Mon, Aug 9, 2010 at 3:25 PM, Marius Gedminas <marius@gedmin.as> wrote:
I've added ETag support for zope.browserresource in a branch: http://zope3.pov.lt/trac/changeset/115596
Does anybody have any comments/objections? If not, I'd like to merge this to trunk and release zope.browserresource 3.11.0.
+1 as long as there's a way to disable or configure how it's computed.
How would you like that to be configured? Option #1: <browser:resource/resourceDirectory etags="off" /> ? Option #2: getMultiAdapter((resource, request), IETag).computeETag() ? This is maybe a bit problematic, because the actual File object that contains all the data--such as filename--doesn't implement any interfaces. And neither does FileResource.
There's some information about issues with ETags here:
http://developer.yahoo.com/performance/rules.html#etags
I see that your implementation uses last-modified + size, which should generally be fine. However if you're load-balancing across two different servers and the timestamps don't match then the ETag is useless.
Would you prefer a sha1 checksum?
On a completely different note, I see that the File object reads the whole file into memory.
And it does that twice for every request that results in a 200 response: once to auto-detect the content-type, the second time to return the actual data.
Hum. Maybe RAM is cheaper than Disk these days and it doesn't matter, but reading whole files into memory generally raises a red flag for me.
hoping-no-one-is-serving-iso-files-through-zope.browserresource-ly yours,
Marius Gedminas -- http://pov.lt/ -- Zope 3/BlueBream consulting and development
On Tue, Aug 10, 2010 at 12:42 PM, Marius Gedminas <marius@gedmin.as> wrote:
How would you like that to be configured?
Option #1:
<browser:resource/resourceDirectory etags="off" /> ?
Option #2:
getMultiAdapter((resource, request), IETag).computeETag() ?
This is maybe a bit problematic, because the actual File object that contains all the data--such as filename--doesn't implement any interfaces. And neither does FileResource.
So maybe Option #1?
I see that your implementation uses last-modified + size, which should generally be fine. However if you're load-balancing across two different servers and the timestamps don't match then the ETag is useless.
Would you prefer a sha1 checksum?
As a default? I'd say last-modified + size is OK.
On a completely different note, I see that the File object reads the whole file into memory.
And it does that twice for every request that results in a 200 response: once to auto-detect the content-type, the second time to return the actual data.
Ouch. That sounds pretty bad. I thought it kept the data in-memory (didn't look close enough at the source). -- Sidnei
On Tue, Aug 10, 2010 at 04:49:47PM -0300, Sidnei da Silva wrote:
On Tue, Aug 10, 2010 at 12:42 PM, Marius Gedminas <marius@gedmin.as> wrote:
How would you like that to be configured?
Option #1:
<browser:resource/resourceDirectory etags="off" /> ?
Option #2:
getMultiAdapter((resource, request), IETag).computeETag() ?
This is maybe a bit problematic, because the actual File object that contains all the data--such as filename--doesn't implement any interfaces. And neither does FileResource.
So maybe Option #1?
But then it's not easily overridable, globally, for a particular application, since each package may have its own browser:resource declarations. Option #3: IResourceETag(request).computeETag(path, timestamp, data) Feels a bit ad-hoc. Option #4: let people who want ultimate cache control use something else than zope.browserresource for serving static files. Feels a bit unfriendly (but Zope 3/Bluebream is usually pretty unfriendly...) Marius Gedminas -- http://pov.lt/ -- Zope 3/BlueBream consulting and development
On Tue, Aug 10, 2010 at 11:54:21PM +0300, Marius Gedminas wrote:
Option #3:
IResourceETag(request).computeETag(path, timestamp, data)
Feels a bit ad-hoc.
The more I think about this, the more I think it's best to configure etag policy by providing an adapter on the request. I think I'll go with some variation on this theme. Marius Gedminas -- http://pov.lt/ -- Zope 3/BlueBream consulting and development
On 2010-8-11 18:00, Marius Gedminas wrote:
On Tue, Aug 10, 2010 at 11:54:21PM +0300, Marius Gedminas wrote:
Option #3:
IResourceETag(request).computeETag(path, timestamp, data)
Feels a bit ad-hoc.
The more I think about this, the more I think it's best to configure etag policy by providing an adapter on the request.
I would suggest request and context, so it is more generally useful. I suspect plone.caching already has such an adapter. Wichert. -- Wichert Akkerman <wichert@wiggy.net> It is simple to make things. http://www.wiggy.net/ It is hard to make things simple.
On 10 August 2010 02:25, Marius Gedminas <marius@gedmin.as> wrote:
I've added ETag support for zope.browserresource in a branch: http://zope3.pov.lt/trac/changeset/115596
Does anybody have any comments/objections? If not, I'd like to merge this to trunk and release zope.browserresource 3.11.0.
No strong objections, really, but bear in mind that people will likely want to customise this. With plone.caching / plone.app.caching we have a framework that, among other things, deals with browser resources and sets etag, last-modified and other headers according to rules configured by the developer and/or administrator. I think that would just stomp on etags set by zope.browserresource, but worth bearing in mind that for bigger applications like Plone, we need a centralised, overarching and configurable strategy for cache headers. Martin
On Tue, Aug 10, 2010 at 08:49:53AM +0800, Martin Aspeli wrote:
On 10 August 2010 02:25, Marius Gedminas <marius@gedmin.as> wrote:
I've added ETag support for zope.browserresource in a branch: http://zope3.pov.lt/trac/changeset/115596
Does anybody have any comments/objections? If not, I'd like to merge this to trunk and release zope.browserresource 3.11.0.
No strong objections, really, but bear in mind that people will likely want to customise this.
Do you have any ideas about the API for customizing this?
With plone.caching / plone.app.caching we have a framework that, among other things, deals with browser resources and sets etag, last-modified and other headers according to rules configured by the developer and/or administrator.
Is it built on top of zope.browserresource, or is it entirely independent? I've grepped the sources of both plone.caching and plone.app.caching. They don't import zope.browserresource, so I guess the answer is "wndependent", unless I'm missing something. Your next sentence leads me to think I *am* missing something:
I think that would just stomp on etags set by zope.browserresource,
but then zope.browserresource's GET handler would reject _your_ etag in its If-None-Matches header check, so there may be a problem here?
but worth bearing in mind that for bigger applications like Plone, we need a centralised, overarching and configurable strategy for cache headers.
Marius Gedminas -- http://pov.lt/ -- Zope 3/BlueBream consulting and development
participants (6)
-
Christian Theune -
Marius Gedminas -
Martin Aspeli -
Sidnei da Silva -
Stephan Richter -
Wichert Akkerman