[Zope] Memory Requirements and SQL Results
Jim Kutter
jim@ebizq.net
Thu, 17 Jul 2003 13:18:42 -0400
This is a multi-part message in MIME format.
------_=_NextPart_001_01C34C87.78AB94F4
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
I'm using Zope to interface with a MySQL back end storing files as
BLOBs. For files in the neighborhood of 1MB+, I get MemoryError
exceptions when I call my 'insert' SQL method. The file gets uploaded to
Zope just fine, but inserting the body throws the exception.
=20
What is the best way to stuff potentially large amounts of data into an
SQL database via Zope without fear of a MemoryError?
=20
-jim
=20
Here is my code (this is a python script which is the action of the
input form - these are not in a product):
(insert is a SQL method which takes mime_type, filename, data, length as
parameters)
=20
request =3D container.REQUEST
response =3D request.RESPONSE
...
file =3D request.form['resource_file']
data =3D file.read()
request.set('mime_type', file.headers['Content-type'])
request.set('data', data)
size =3D len(data)
request.set('length', size)
filename =3D file.filename[file.filename.rindex('\\')+1:]
request.set('filename', filename)
context.insert()
------_=_NextPart_001_01C34C87.78AB94F4
Content-Type: text/html;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD><TITLE>Message</TITLE>
<META http-equiv=3DContent-Type content=3D"text/html; =
charset=3Dus-ascii">
<META content=3D"MSHTML 6.00.2800.1170" name=3DGENERATOR></HEAD>
<BODY>
<DIV><SPAN class=3D467370517-17072003><FONT face=3DArial size=3D2>I'm =
using Zope to=20
interface with a MySQL back end storing files as BLOBs. For =
files in the=20
neighborhood of 1MB+, I get MemoryError exceptions when I call my =
'insert' SQL=20
method. The file gets uploaded to Zope just fine, but inserting the body =
throws=20
the exception.</FONT></SPAN></DIV>
<DIV><SPAN class=3D467370517-17072003><FONT face=3DArial=20
size=3D2></FONT></SPAN> </DIV>
<DIV><SPAN class=3D467370517-17072003><FONT face=3DArial size=3D2>What =
is the best way=20
to stuff potentially large amounts of data into an SQL database via Zope =
without=20
fear of a MemoryError?</FONT></SPAN></DIV>
<DIV><SPAN class=3D467370517-17072003><FONT face=3DArial=20
size=3D2></FONT></SPAN> </DIV>
<DIV><SPAN class=3D467370517-17072003><FONT face=3DArial=20
size=3D2>-jim</FONT></SPAN></DIV>
<DIV><SPAN class=3D467370517-17072003><FONT face=3DArial=20
size=3D2></FONT></SPAN> </DIV>
<DIV><SPAN class=3D467370517-17072003><FONT face=3DArial size=3D2>Here =
is my code=20
(this is a python script which is the action of the input form - these =
are not=20
in a product):</FONT></SPAN></DIV>
<DIV><SPAN class=3D467370517-17072003><FONT face=3DArial =
size=3D2>(insert is a SQL=20
method which takes mime_type, filename, data, length as=20
parameters)</FONT></SPAN></DIV>
<DIV><SPAN class=3D467370517-17072003><FONT face=3DArial=20
size=3D2></FONT></SPAN> </DIV>
<DIV><SPAN class=3D467370517-17072003><FONT face=3DArial size=3D2> =
request =3D=20
container.REQUEST<BR> response =3D =
request.RESPONSE</FONT></SPAN></DIV>
<DIV><SPAN class=3D467370517-17072003><FONT face=3DArial size=3D2> =
...</FONT></SPAN></DIV>
<DIV><SPAN class=3D467370517-17072003><FONT face=3DArial size=3D2> =
file =3D=20
request.form['resource_file']</FONT></SPAN></DIV>
<DIV><SPAN class=3D467370517-17072003><FONT face=3DArial size=3D2> =
data =3D=20
file.read()<BR> request.set('mime_type',=20
file.headers['Content-type'])<BR> request.set('data', =
data)<BR> size=20
=3D len(data)<BR> request.set('length', size)<BR> filename =
=3D=20
file.filename[file.filename.rindex('\\')+1:]<BR> =
request.set('filename',=20
filename)<BR> context.insert()</FONT></SPAN><SPAN=20
class=3D467370517-17072003><FONT face=3DArial=20
size=3D2><BR></DIV></FONT></SPAN></BODY></HTML>
=00
------_=_NextPart_001_01C34C87.78AB94F4--