[Zope3-checkins] SVN: Zope3/branches/3.2/ Added new machinery that
allows published methods to just return
Jim Fulton
jim at zope.com
Fri Dec 23 14:51:23 EST 2005
Log message for revision 41002:
Added new machinery that allows published methods to just return
files.
Also:
- IResult is now a private interface
- When we look up an IResult, we use a multi-adapter call with the
request. This means that result adapters have access to the
request and response, which would allow a significant
simplification of the result API. This is why we made it
private now, so we can change it later.
Changed:
U Zope3/branches/3.2/doc/CHANGES.txt
U Zope3/branches/3.2/src/zope/app/configure.zcml
U Zope3/branches/3.2/src/zope/app/publisher/http.zcml
A Zope3/branches/3.2/src/zope/app/wsgi/configure.zcml
A Zope3/branches/3.2/src/zope/app/wsgi/fileresult.py
A Zope3/branches/3.2/src/zope/app/wsgi/fileresult.txt
U Zope3/branches/3.2/src/zope/app/wsgi/tests.py
U Zope3/branches/3.2/src/zope/publisher/http.py
U Zope3/branches/3.2/src/zope/publisher/httpresults.txt
U Zope3/branches/3.2/src/zope/publisher/interfaces/http.py
-=-
Modified: Zope3/branches/3.2/doc/CHANGES.txt
===================================================================
--- Zope3/branches/3.2/doc/CHANGES.txt 2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/doc/CHANGES.txt 2005-12-23 19:51:22 UTC (rev 41002)
@@ -110,32 +110,19 @@
- addMenuItem directive supports a `layer` attribute.
- - Added a new API, zope.publisher.interfaces.http.IResult. See
- the file httpresults.txt in the zope.publisher package for
- details.
+ - Changed the Publisher Response API.
- - Formalized the Publisher Response API.
+ + Large results can now ne handled efeciently by returning
+ files rather than strings. See the file httpresults.txt in
+ the zope.publisher package.
- + Until now the publisher made assumptions about the form of ouput of
- a publishing process. Either the called method returned a string
- (regular or unicode) or the response's write() method was used
- directly to write the data. Those models do not work well with some
- protocols. Thus, now the publisher deals with result objects. Those
- are generally not well defined, but for HTTP they must implement the
- IResult interface.
+ + The unused response.write method is no-longer supported.
+ HTTP responses provide two new methods that make reading the output
easier: `consumeBody()` and `consumeBodyIter()`. Either method can
be only called once. After that the output iterator is used and
empty.
- + The WSGI specification specifically has some provisions in it that
- supported our use of writing directly to the output stream. However,
- this method of providing an output is strongly discouraged. Instead,
- the application should return an iterable. Using the new IResult
- implementation in the HTTP publisher, we can now return such an
- iterable.
-
+ When a retry is issued in the publisher, then a new request is
created. This means that the request (including its response) that
were passed into `publish()` are not necessarily the same that are
Modified: Zope3/branches/3.2/src/zope/app/configure.zcml
===================================================================
--- Zope3/branches/3.2/src/zope/app/configure.zcml 2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/app/configure.zcml 2005-12-23 19:51:22 UTC (rev 41002)
@@ -71,6 +71,7 @@
<include package="zope.app.applicationcontrol" />
<include package="zope.app.dublincore" />
<include package="zope.app.introspector" />
+ <include package="zope.app.wsgi" />
<!-- Content types -->
Modified: Zope3/branches/3.2/src/zope/app/publisher/http.zcml
===================================================================
--- Zope3/branches/3.2/src/zope/app/publisher/http.zcml 2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/app/publisher/http.zcml 2005-12-23 19:51:22 UTC (rev 41002)
@@ -15,7 +15,7 @@
</content>
<class class="zope.publisher.http.DirectResult">
- <allow interface="zope.publisher.interfaces.http.IResult" />
+ <allow interface="zope.publisher.http.IResult" />
</class>
</configure>
Added: Zope3/branches/3.2/src/zope/app/wsgi/configure.zcml
===================================================================
--- Zope3/branches/3.2/src/zope/app/wsgi/configure.zcml 2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/app/wsgi/configure.zcml 2005-12-23 19:51:22 UTC (rev 41002)
@@ -0,0 +1,5 @@
+<configure xmlns="http://namespaces.zope.org/zope">
+ <adapter factory=".fileresult.FileResult" />
+ <adapter factory=".fileresult.TemporaryFileResult" />
+</configure>
+
Property changes on: Zope3/branches/3.2/src/zope/app/wsgi/configure.zcml
___________________________________________________________________
Name: svn:eol-style
+ native
Added: Zope3/branches/3.2/src/zope/app/wsgi/fileresult.py
===================================================================
--- Zope3/branches/3.2/src/zope/app/wsgi/fileresult.py 2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/app/wsgi/fileresult.py 2005-12-23 19:51:22 UTC (rev 41002)
@@ -0,0 +1,74 @@
+##############################################################################
+#
+# Copyright (c) 2005 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""IResult adapters for files.
+
+$Id$
+"""
+
+import tempfile
+from zope import component, interface
+import zope.publisher.interfaces.http
+import zope.publisher.http
+from zope.publisher.http import DirectResult
+from zope.security.proxy import removeSecurityProxy
+
+class FallbackWrapper:
+
+ def __init__(self, f):
+ self.close = f.close
+ self._file = f
+
+ def __iter__(self):
+ f = self._file
+ while 1:
+ v = f.read(32768)
+ if v:
+ yield v
+ else:
+ break
+
+ at component.adapter(file, zope.publisher.interfaces.http.IHTTPRequest)
+ at interface.implementer(zope.publisher.http.IResult)
+def FileResult(f, request):
+ f = removeSecurityProxy(f)
+ headers = ()
+ if request.response.getHeader('content-length') is None:
+ f.seek(0, 2)
+ size = f.tell()
+ f.seek(0)
+ headers += (('Content-Length', str(size)), )
+
+ wrapper = request.environment.get('wsgi.file_wrapper')
+ if wrapper is not None:
+ f = wrapper(f)
+ else:
+ f = FallbackWrapper(f)
+ return DirectResult(f, headers)
+
+# We need to provide an adapter for temporary files *if* they are different
+# than regular files. Whether they are is system dependent. Sigh.
+# If temporary files are the same type, we'll create a fake type just
+# to make the registration work.
+_tfile = tempfile.TemporaryFile()
+_tfile.close()
+_tfile = _tfile.__class__
+if _tfile is file:
+ # need a fake one. Sigh
+ class _tfile:
+ pass
+
+ at component.adapter(_tfile, zope.publisher.interfaces.http.IHTTPRequest)
+ at interface.implementer(zope.publisher.http.IResult)
+def TemporaryFileResult(f, request):
+ return FileResult(f, request)
Property changes on: Zope3/branches/3.2/src/zope/app/wsgi/fileresult.py
___________________________________________________________________
Name: svn:keywords
+ Id
Name: svn:eol-style
+ native
Added: Zope3/branches/3.2/src/zope/app/wsgi/fileresult.txt
===================================================================
--- Zope3/branches/3.2/src/zope/app/wsgi/fileresult.txt 2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/app/wsgi/fileresult.txt 2005-12-23 19:51:22 UTC (rev 41002)
@@ -0,0 +1,98 @@
+File results
+============
+
+The file results adapters provide adapters from Python file objects
+to and from temporary file objects to zope.publisher.http.IResult. They also have
+the property that they can handle security proxied files and unproxy
+them in the result. Finally, if the request has a wsgi.file_wrapper
+environment variable, then that is used to wrap the file in the
+result.
+
+Lets look at an example with a regular file object:
+
+ >>> from zope import component
+ >>> import zope.app.wsgi.fileresult
+ >>> component.provideAdapter(zope.app.wsgi.fileresult.FileResult)
+ >>> component.provideAdapter(zope.app.wsgi.fileresult.TemporaryFileResult)
+
+ >>> import tempfile
+ >>> dir = tempfile.mkdtemp()
+ >>> import os
+ >>> f = open(os.path.join(dir, 'f'), 'w+b')
+ >>> f.write('One\nTwo\nThree\nHa ha! I love to count!\n')
+ >>> from zope.security.checker import ProxyFactory
+ >>> from zope.publisher.http import IResult
+ >>> from zope.publisher.browser import TestRequest
+ >>> request = TestRequest()
+ >>> result = component.getMultiAdapter((ProxyFactory(f), request), IResult)
+ >>> for line in result.body:
+ ... print line
+ One
+ Two
+ Three
+ Ha ha! I love to count!
+
+ >>> result.headers
+ (('Content-Length', '38'),)
+
+
+We'll see something similar with a temporary file:
+
+ >>> t = tempfile.TemporaryFile()
+ >>> t.write('Three\nTwo\nOne\nHa ha! I love to count down!\n')
+ >>> result = component.getMultiAdapter((ProxyFactory(t), request), IResult)
+ >>> for line in result.body:
+ ... print line
+ Three
+ Two
+ One
+ Ha ha! I love to count down!
+
+ >>> result.headers
+ (('Content-Length', '43'),)
+
+
+If we provide a custom file wrapper:
+
+ >>> class Wrapper:
+ ... def __init__(self, file):
+ ... self.file = file
+
+ >>> request = TestRequest(environ={'wsgi.file_wrapper': Wrapper})
+ >>> result = component.getMultiAdapter((ProxyFactory(f), request), IResult)
+ >>> result.body.__class__ is Wrapper
+ True
+ >>> result.body.file is f
+ True
+
+ >>> result.headers
+ (('Content-Length', '38'),)
+
+ >>> result = component.getMultiAdapter((ProxyFactory(t), request), IResult)
+ >>> result.body.__class__ is Wrapper
+ True
+ >>> result.body.file is t
+ True
+
+ >>> result.headers
+ (('Content-Length', '43'),)
+
+Normally, the file given to FileResult must be seekable and the entire
+file is used. The adapters figure out the file size to determine a
+content length and seek to the beginning of the file.
+
+You can suppress this behavior by setting the content length yourself:
+
+ >>> request = TestRequest()
+ >>> request.response.setHeader('content-length', '10')
+ >>> f.seek(7)
+ >>> result = component.getMultiAdapter((ProxyFactory(t), request), IResult)
+ >>> print f.tell()
+ 7
+
+ >>> result.headers
+ ()
+
+Note, that you should really only use file returns for large results.
+Files use file descriptors which can be somewhat scarece resources on
+some systems. Only use them when you needs them.
Property changes on: Zope3/branches/3.2/src/zope/app/wsgi/fileresult.txt
___________________________________________________________________
Name: svn:eol-style
+ native
Modified: Zope3/branches/3.2/src/zope/app/wsgi/tests.py
===================================================================
--- Zope3/branches/3.2/src/zope/app/wsgi/tests.py 2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/app/wsgi/tests.py 2005-12-23 19:51:22 UTC (rev 41002)
@@ -15,8 +15,14 @@
$Id$
"""
+import tempfile
import unittest
+
+from zope import component, interface
from zope.testing import doctest
+
+import zope.app.testing.functional
+import zope.publisher.interfaces.browser
from zope.app.testing import placelesssetup
from zope.app.publication.requestpublicationregistry import factoryRegistry
from zope.app.publication.requestpublicationfactories import BrowserFactory
@@ -25,10 +31,78 @@
placelesssetup.setUp(test)
factoryRegistry.register('GET', '*', 'browser', 0, BrowserFactory())
+
+
+class FileView:
+
+ interface.implements(zope.publisher.interfaces.browser.IBrowserPublisher)
+ component.adapts(interface.Interface,
+ zope.publisher.interfaces.browser.IBrowserRequest)
+
+ def __init__(self, _, request):
+ self.request = request
+
+ def browserDefault(self, *_):
+ return self, ()
+
+ def __call__(self):
+ self.request.response.setHeader('content-type', 'text/plain')
+ f = tempfile.TemporaryFile()
+ f.write("Hello\nWorld!\n")
+ return f
+
+
+def test_file_returns():
+ """We want to make sure that file returns work
+
+Let's register a view that returns a temporary file and make sure that
+nothing bad happens. :)
+
+ >>> component.provideAdapter(FileView, name='test-file-view.html')
+ >>> from zope.security import checker
+ >>> checker.defineChecker(
+ ... FileView,
+ ... checker.NamesChecker(['browserDefault', '__call__']),
+ ... )
+
+ >>> from zope.testbrowser import Browser
+ >>> browser = Browser()
+ >>> browser.handleErrors = False
+ >>> browser.open('http://localhost/@@test-file-view.html')
+ >>> browser.headers['content-type']
+ 'text/plain'
+
+ >>> browser.headers['content-length']
+ '13'
+
+ >>> print browser.contents
+ Hello
+ World!
+ <BLANKLINE>
+
+Clean up:
+
+ >>> checker.undefineChecker(FileView)
+ >>> component.provideAdapter(
+ ... None,
+ ... (interface.Interface,
+ ... zope.publisher.interfaces.browser.IBrowserRequest),
+ ... zope.publisher.interfaces.browser.IBrowserPublisher,
+ ... 'test-file-view.html',
+ ... )
+
+
+"""
+
def test_suite():
+
+ functional_suite = doctest.DocTestSuite()
+ functional_suite.layer = zope.app.testing.functional.Functional
+
return unittest.TestSuite((
+ functional_suite,
doctest.DocFileSuite(
- 'README.txt',
+ 'README.txt', 'fileresult.txt',
setUp=setUp,
tearDown=placelesssetup.tearDown,
optionflags=doctest.NORMALIZE_WHITESPACE|doctest.ELLIPSIS),
Modified: Zope3/branches/3.2/src/zope/publisher/http.py
===================================================================
--- Zope3/branches/3.2/src/zope/publisher/http.py 2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/publisher/http.py 2005-12-23 19:51:22 UTC (rev 41002)
@@ -24,8 +24,9 @@
import logging
from tempfile import TemporaryFile
+from zope import component, interface
+
from zope.deprecation import deprecation
-from zope.interface import implements
from zope.publisher import contenttype
from zope.publisher.interfaces.http import IHTTPCredentials
@@ -34,7 +35,7 @@
from zope.publisher.interfaces.http import IHTTPPublisher
from zope.publisher.interfaces import Redirect
-from zope.publisher.interfaces.http import IHTTPResponse, IResult
+from zope.publisher.interfaces.http import IHTTPResponse
from zope.publisher.interfaces.http import IHTTPApplicationResponse
from zope.publisher.interfaces.logginginfo import ILoggingInfo
from zope.i18n.interfaces import IUserPreferredCharsets
@@ -251,7 +252,9 @@
values will be looked up in the order: environment variables,
other variables, form data, and then cookies.
"""
- implements(IHTTPCredentials, IHTTPRequest, IHTTPApplicationRequest)
+ interface.implements(IHTTPCredentials,
+ IHTTPRequest,
+ IHTTPApplicationRequest)
__slots__ = (
'__provides__', # Allow request to directly provide interfaces
@@ -593,8 +596,32 @@
d.update(self._cookies)
return d.keys()
+
+class IResult(interface.Interface):
+ """HTTP result.
+
+ WARNING! This is a PRIVATE interface and VERY LIKELY TO CHANGE!
+
+ The result provides the result in a form suitable for delivery to HTTP
+ clients.
+
+ IMPORTANT: The result object may be held indefinitely by a server and may
+ be accessed by arbitrary threads. For that reason the result should not
+ hold on to any application resources and should be prepared to be invoked
+ from any thread.
+ """
+
+ headers = interface.Attribute(
+ 'A sequence of tuples of result headers, such as '
+ '"Content-Type" and "Content-Length", etc.')
+
+ body = interface.Attribute(
+ 'An iterable that provides the body data of the response.')
+
+
+
class HTTPResponse(BaseResponse):
- implements(IHTTPResponse, IHTTPApplicationResponse)
+ interface.implements(IHTTPResponse, IHTTPApplicationResponse)
__slots__ = (
'authUser', # Authenticated user string
@@ -771,16 +798,21 @@
def setResult(self, result):
- r = IResult(result, None)
- if r is None:
- if isinstance(result, basestring):
- body, headers = self._implicitResult(result)
- r = DirectResult((body,), headers)
- elif result is None:
- body, headers = self._implicitResult('')
- r = DirectResult((body,), headers)
- else:
- raise TypeError('The result should be adaptable to IResult.')
+ if IResult.providedBy(result):
+ r = result
+ else:
+ r = component.queryMultiAdapter((result, self._request), IResult)
+ if r is None:
+ if isinstance(result, basestring):
+ body, headers = self._implicitResult(result)
+ r = DirectResult((body,), headers)
+ elif result is None:
+ body, headers = self._implicitResult('')
+ r = DirectResult((body,), headers)
+ else:
+ raise TypeError(
+ 'The result should be adaptable to IResult.')
+
self._result = r
self._headers.update(dict([(k, [v]) for (k, v) in r.headers]))
if not self._status_set:
@@ -937,7 +969,7 @@
class HTTPCharsets(object):
- implements(IUserPreferredCharsets)
+ interface.implements(IUserPreferredCharsets)
def __init__(self, request):
self.request = request
@@ -1009,7 +1041,7 @@
application to specify all headers related to the content, such as the
content type and length.
"""
- implements(IResult)
+ interface.implements(IResult)
def __init__(self, body, headers=()):
self.body = body
@@ -1023,3 +1055,4 @@
including content type and length.
"""
return DirectResult((body,), headers)
+
Modified: Zope3/branches/3.2/src/zope/publisher/httpresults.txt
===================================================================
--- Zope3/branches/3.2/src/zope/publisher/httpresults.txt 2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/publisher/httpresults.txt 2005-12-23 19:51:22 UTC (rev 41002)
@@ -30,67 +30,15 @@
Returning large amounts of data without storing the data in memory
------------------------------------------------------------------
-Starting in Zope 3.2, a published object (e.g. a view or view method)
-can return any object as long as it is is adaptable to
-zope.publisher.interfaces.http.IResult::
+To return a large result, you should write the result to a temporary
+file (tempfile.TemporaryFile) and return the temporary file.
+Alternatively, if the data you want to return is already in a
+(non-temporary) file, just open and return that file. The publisher
+(actually an adapter used by the publisher) will handle a returned
+file very efficiently.
- class IResult(Interface):
- """HTTP result.
+The publisher will compute the response content length from the file
+automatically. It is up to applications to set the content type.
+It will also take care of positioning the file to it's beginning,
+so applications don't need to do this beforehand.
- The result provides the result in a form suitable for delivery to HTTP
- clients.
-
- IMPORTANT: The result object may be held indefinitely by a server and may
- be accessed by arbitrary threads. For that reason the result should not
- hold on to any application resources and should be prepared to be invoked
- from any thread.
- """
-
- headers = Attribute('A sequence of tuples of result headers, such as'
- '"Content-Type" and "Content-Length", etc.')
-
- body = Attribute('An iterable that provides the body data of the'
- 'response.')
-
-The result object has headers and an iterable body. The ability to
-supply headers in a result is useful for adapters that compute headers
-by inspecting a the object being adapted.
-
-There is a helper class, zope.publisher.http.DirectResult that can be
-used to compute result objects.
-
-When an published object returns a string. the string is inspected to
-determine response headers (like content type and content length) and
-a result is created using DirectResult.
-
-If you want to return a large amont of data, you can create a result
-object yourself. A good way to do this is to copy the data to a
-temporary file and return an iterator to that::
-
- import tempfile
- file = tempfile.TemporaryFile()
-
- # ... write data to the file ...
-
- def fileiterator(file, bufsize=8192):
- while 1:
- data = file.read(bufsize)
- if data:
- yield data
- else:
- break
-
- file.close()
-
- return DirectResult(fileiterator(file),
- [('Content-Length', mydatalength),
- ('Content-Type', mydatatype),
- ])
-
-We should provide some helper objects that automate more of this, and
-we probably will in later revisions.
-
-IMPORTANT NOTE: the iterator that you pass to DirectResult must *not*
-use any application resources. When the iterator is called,
-application resoures may have been released or be in use by another
-thread.
Modified: Zope3/branches/3.2/src/zope/publisher/interfaces/http.py
===================================================================
--- Zope3/branches/3.2/src/zope/publisher/interfaces/http.py 2005-12-23 18:13:08 UTC (rev 41001)
+++ Zope3/branches/3.2/src/zope/publisher/interfaces/http.py 2005-12-23 19:51:22 UTC (rev 41002)
@@ -373,7 +373,7 @@
"""
def setResult(result):
- """Sets the response result value that is adaptable to ``IResult``.
+ """Sets the response result value to a string or a file.
"""
def consumeBody():
@@ -389,23 +389,3 @@
Note that this function can be only requested once, since it is
constructed from the result.
"""
-
-
-class IResult(Interface):
- """HTTP result.
-
- The result provides the result in a form suitable for delivery to HTTP
- clients.
-
- IMPORTANT: The result object may be held indefinitely by a server and may
- be accessed by arbitrary threads. For that reason the result should not
- hold on to any application resources and should be prepared to be invoked
- from any thread.
- """
-
- headers = Attribute('A sequence of tuples of result headers, such as'
- '"Content-Type" and "Content-Length", etc.')
-
- body = Attribute('An iterable that provides the body data of the'
- 'response.')
-
More information about the Zope3-Checkins
mailing list