dtml-sqlvar problem with large strings?
Hello All, I am uploading and encoding files and then trying to insert the file content into a database. It works fine for small files (under about 20KB), but fails for larger ones with the error: Error Type: ProgrammingError Error Value: (-104, 'execute.isc_dsql_prepare: Dynamic SQL Error. SQL error code = -104. Unexpected end of command. ') See below for traceback etc. Using zope 2.6.4 on win32 with kinterbasdbda (not sure what version), firebird 1.0 It sounds like the sql command is getting trucated or something with the larger strings. Can anybody offer suggestions? Many Thanks, Josh ##################################### I have a form to upload a file: <dtml-var standard_html_header> <form method="post" action="upload_process" enctype="multipart/form-data"> <input type="file" name="quote_file"> <input type=hidden name=shipment_id value=&dtml-shipment_id;> <input type="submit"> <dtml-var standard_html_footer> A script to process the file: from Products.PythonScripts.standard import html_quote request = container.REQUEST RESPONSE = request.RESPONSE fil = request.quote_file data = fil.read() ct = fil.headers['Content-Type'] encdata = context.encode(data) print ct print len(data) context.insert(quote_file_data=encdata, quote_content_type=ct, shipment_id=request.shipment_id) return printed A zsql method: insert into quote_files (quote_file_data, quote_content_type, shipment_id) values (<dtml-sqlvar quote_file_data type=string>, <dtml-sqlvar quote_content_type type=string>, <dtml-sqlvar shipment_id type=int> ) External methods (encode_decode.py) for encoding and decoding the possibly binary file contents: def encode(bindata): from base64 import encodestring return encodestring(bindata) def decode(strdata): from base64 import decodestring return decodestring(strdata) My database table: CREATE TABLE QUOTE_FILES ( QUOTE_FILE_ID INTEGER NOT NULL, QUOTE_CONTENT_TYPE VARCHAR(50) CHARACTER SET NONE, SHIPMENT_ID INTEGER, FILENAME VARCHAR(500) CHARACTER SET NONE, QUOTE_FILE_DATA BLOB SUB_TYPE 3 SEGMENT SIZE 1 ); Exception traceback Time 2005/11/21 16:01:45.513 GMT+11 User Name (User Id) josh (josh) Request URL http://mrisydney:8080/Bigbird/shipments/admin3/quotes/upload_process Exception Type ProgrammingError Exception Value (-104, 'execute.isc_dsql_prepare: Dynamic SQL Error. SQL error code = -104. Unexpected end of command. ') Traceback (innermost last): . Module ZPublisher.Publish, line 98, in publish . Module ZPublisher.mapply, line 88, in mapply . Module ZPublisher.Publish, line 39, in call_object . Module Shared.DC.Scripts.Bindings, line 306, in __call__ . Module Shared.DC.Scripts.Bindings, line 343, in _bindAndExec . Module Products.PythonScripts.PythonScript, line 307, in _exec . Module None, line 16, in upload_process <PythonScript at /Bigbird/shipments/admin3/quotes/upload_process> Line 16 . Module Shared.DC.ZRDB.DA, line 428, in __call__ . Module Products.kinterbasdbDA.db, line 198, in query . Module Products.kinterbasdbDA.kinterbasdb, line 585, in execute ProgrammingError: (-104, 'execute.isc_dsql_prepare: Dynamic SQL Error. SQL error code = -104. Unexpected end of command. ') Display traceback as text REQUEST form quote_file <ZPublisher.HTTPRequest.FileUpload instance at 04B2FD6C> shipment_id '13096' cookies lazy items SESSION <bound method SessionDataManager.getSessionData of <SessionDataManager instance at 023D1718>> other AUTHENTICATION_PATH 'Bigbird' traverse_subpath [] SERVER_URL 'http://mrisydney:8080' shipment_id '13096' PUBLISHED <PythonScript at /Bigbird/shipments/admin3/quotes/upload_process> URL 'http://mrisydney:8080/Bigbird/shipments/admin3/quotes/upload_process' AUTHENTICATED_USER josh TraversalRequestNameStack [] quote_file <ZPublisher.HTTPRequest.FileUpload instance at 04B2FD6C> URL0 http://mrisydney:8080/Bigbird/shipments/admin3/quotes/upload_process URL1 http://mrisydney:8080/Bigbird/shipments/admin3/quotes URL2 http://mrisydney:8080/Bigbird/shipments/admin3 URL3 http://mrisydney:8080/Bigbird/shipments URL4 http://mrisydney:8080/Bigbird URL5 http://mrisydney:8080 BASE0 http://mrisydney:8080 BASE1 http://mrisydney:8080 BASE2 http://mrisydney:8080/Bigbird BASE3 http://mrisydney:8080/Bigbird/shipments BASE4 http://mrisydney:8080/Bigbird/shipments/admin3 BASE5 http://mrisydney:8080/Bigbird/shipments/admin3/quotes BASE6 http://mrisydney:8080/Bigbird/shipments/admin3/quotes/upload_process environ HTTP_ACCEPT_ENCODING 'gzip, deflate' CONTENT_TYPE 'multipart/form-data; boundary=---------------------------7d5bb2a201dc' PATH_TRANSLATED '\\Bigbird\\shipments\\admin3\\quotes\\upload_process' HTTP_ACCEPT 'image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/x-shockwave-flash, application/vnd.ms-excel, application/vnd.ms-powerpoint, application/msword, */*' GATEWAY_INTERFACE 'CGI/1.1' HTTP_ACCEPT_LANGUAGE 'en-au' REMOTE_ADDR '192.168.2.37' SERVER_PORT '8080' HTTP_USER_AGENT 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)' CONTENT_LENGTH '72706' channel.creation_time 1132549273 SERVER_PROTOCOL 'HTTP/1.1' PATH_INFO '/Bigbird/shipments/admin3/quotes/upload_process' HTTP_HOST 'mrisydney:8080' REQUEST_METHOD 'POST' SCRIPT_NAME '' SERVER_SOFTWARE 'Zope/(Zope 2.6.4 (binary release, python 2.1, win32-x86), python 2.1.3, win32) ZServer/1.1b1' HTTP_CACHE_CONTROL 'no-cache' CONNECTION_TYPE 'Keep-Alive' HTTP_REFERER 'http://mrisydney:8080/Bigbird/shipments/admin3/quotes/upload_form?shipment_ id=13096' SERVER_NAME 'mrisydney.sydney.mri.com.au'
--On 21. November 2005 16:30:10 +1100 Joshua Burvill <josh@mri.com.au> wrote:
Using zope 2.6.4 on win32 with kinterbasdbda (not sure what version), firebird 1.0
This sounds like a bug in the kinterbasedbda. Since this is DA is not used widely I would recommend to buzz the author of the DA directly. When you are on Windows you could consider switching to mxODBC in case your database supports ODBC. -aj
Hello All, I tried using zope 2.7.6 with kinterbasdbda 2.0 and I got the same error I think, but more informative :) ProgrammingError: (0, 'SQL statement of 2270651 bytes is too long (max 65535 allowed). Consider using parameters to shorten the SQL code, rather than passing large values as part of the SQL string.') So I guess it is the DA that disallows the long statement. So I will "Consider using parameters..." as it says. Otherwise I can look at switching to mxODBC. I wonder if that would disallow such long statements as well? Thanks for the help. Josh Traceback (innermost last): Module ZPublisher.Publish, line 101, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 39, in call_object Module Shared.DC.Scripts.Bindings, line 306, in __call__ Module Shared.DC.Scripts.Bindings, line 343, in _bindAndExec Module Products.PythonScripts.PythonScript, line 323, in _exec Module None, line 16, in upload_process <PythonScript at /external/dr3/test/quotes/upload_process> Line 16 Module Shared.DC.ZRDB.DA, line 454, in __call__ <SQL instance at 04168BC0> Module Products.kInterbaseDA.db, line 226, in query Module Products.kInterbaseDA.db, line 255, in _executeSQLAndReturnResultRows Module kinterbasdb, line 1592, in execute ProgrammingError: (0, 'SQL statement of 2270651 bytes is too long (max 65535 allowed). Consider using parameters to shorten the SQL code, rather than passing large values as part of the SQL string.') -----Original Message----- From: Andreas Jung [mailto:lists@andreas-jung.com] Sent: Monday, 21 November 2005 4:36 PM To: Joshua Burvill; zope@zope.org Subject: Re: [Zope] dtml-sqlvar problem with large strings? --On 21. November 2005 16:30:10 +1100 Joshua Burvill <josh@mri.com.au> wrote:
Using zope 2.6.4 on win32 with kinterbasdbda (not sure what version), firebird 1.0
This sounds like a bug in the kinterbasedbda. Since this is DA is not used widely I would recommend to buzz the author of the DA directly. When you are on Windows you could consider switching to mxODBC in case your database supports ODBC. -aj
--On 22. November 2005 08:42:32 +1100 Joshua Burvill <josh@mri.com.au> wrote: statement. So I will
"Consider using parameters..." as it says.
Otherwise I can look at switching to mxODBC. I wonder if that would disallow such long statements as well?
You could try it and report back :-) -aj
participants (2)
-
Andreas Jung -
Joshua Burvill