[Zope3-dev] Re: [Zope-dev] Re: Proposal for optimized Blob handling

[Christian Theune]

Good point.

I'll re-read the spec and will try to integrate that.

Am Mittwoch, den 07.03.2007, 22:46 -0500 schrieb Chris McDonough:
> Note that one micro-optimization for PUT requests is to not use a  
> FieldStorage at all because the body is never mime-encoded anyway in  
> practice.
> 
> I have a monkey patch to do this now, which I turned into a patch for  
> the core, but took out because Phillipp whined at a sprint once.  ;-)
> 
> Here's the monkey patch...
> 
> def patch_httprequest_processinputs():
>      """ Patch HTTPRequest.processInputs to not do any processing on a
>      PUT request (it's pointless, and foils our on-the-fly encryption,
>      as it creates a new tempfile via FieldStorage). """
> 
>      # note that OTF encryption support only works for PUT requests
> 
>      import re
>      from ZPublisher.HTTPRequest import HTTPRequest
>      oldProcessInputs = HTTPRequest.processInputs
>      def newProcessInputs(
>          self,
>          # "static" variables that we want to be local for speed
>          SEQUENCE=1,
>          DEFAULT=2,
>          RECORD=4,
>          RECORDS=8,
>          REC=12, # RECORD|RECORDS
>          EMPTY=16,
>          CONVERTED=32,
>          hasattr=hasattr,
>          getattr=getattr,
>          setattr=setattr,
>          search_type=re.compile('(:[a-zA-Z][-a-zA-Z0-9_]+|\\.[xy]) 
> $').search,
>          ):
>          """Process request inputs
> 
>          We need to delay input parsing so that it is done under
>          publisher control for error handling purposes.
>          """
>          method=self.environ.get('REQUEST_METHOD','GET')
> 
>          if method == 'PUT':
>              # we don't need to do any real input processing if we  
> are handling
>              # a PUT request.  This is an optimization especially  
> because
>              # FieldStorage creates an additional tempfile if we  
> allow it to
>              # parse the body, and PUT uploads can tend to be large.
>              self._file = self.stdin
>              return
> 
>          return oldProcessInputs(self)
>      HTTPRequest.processInputs = newProcessInputs
> 
> 
> 
> 
> - C
> 
> 
> On Mar 7, 2007, at 9:57 PM, Philipp von Weitershausen wrote:
> 
> > Christian Theune wrote:
> >> Hi,
> >> [modified slightly from a similar proposal to zope3-dev to match Zope
> >> 2's publisher]
> >> I'm writing up a proposal for the ZODB to make even more efficient  
> >> Blob
> >> handling possible.
> >> This includes not copying the data from an uploaded file, but using a
> >> `link` operation when possible.
> >
> > I think this is a great idea.
> >
> > Am I the only person here who immediately associated "link" with  
> > the POSIX? Also, am I the only one who read "when possible" as  
> > "when on a POSIX system where link is available", in other words,  
> > "when not on Windows"? One starts to wonder...
> >
> >> However, the HTTPRequest class currently uses the default  
> >> implementation
> >> of the cgi module's FieldStorage.
> >> I propose to create a small subclass to override the `make_file`  
> >> method
> >> to use `NamedTemporaryFile` instead of `TemporaryFile` to allow  
> >> the file
> >> being accessible from a filename so I can apply a `link` operation.
> >
> > +1
> >
> >
> > -- 
> > http://worldcookery.com -- Professional Zope documentation and  
> > training
> > _______________________________________________
> > Zope-Dev maillist  -  Zope-Dev at zope.org
> > http://mail.zope.org/mailman/listinfo/zope-dev
> > **  No cross posts or HTML encoding!  **
> > (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce
> > http://mail.zope.org/mailman/listinfo/zope )
> >
> 
> _______________________________________________
> Zope3-dev mailing list
> Zope3-dev at zope.org
> Unsub: http://mail.zope.org/mailman/options/zope3-dev/ct%40gocept.com
> 
-- 
gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany
www.gocept.com - ct at gocept.com - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Dies ist ein digital signierter Nachrichtenteil
Url : http://mail.zope.org/pipermail/zope-dev/attachments/20070308/32d03666/attachment.bin