I learned my lesson for today, so I thought I'd share it. I am passing around large strings containing PDF and PS documents. Sometimes I make a call like document = container.PJdoc(container['proposal'].data) where the object "proposal" is a PDF file. I end up writing this data to a temporary file in my PJ class. I noticed early on that I could not simply write the whole string, but I worked around that by writing it in 1K blocks. I had my warning. Today my dumb kludge came back to bite me. We found that some of our proposals were rejected by the PDF parser. I started looking at the temporary files and noticed the problem ones were 98798 bytes in length. The PDF file sent was much larger. After some tests, I determined that asking for a slice from .data only returns a null string (zero length) after 98798 bytes. This meant that my little 1K write loop was happily going through the "string" but it was writing 0K blocks after awhile. So...I finally fixed the problem. Instead of all of the 1K writing junk, I simply cast whatever is passed to me into a real string. Then I can write the whole thing without any problems. I'm sure this is documented somewhere that I should have read before, but I'm hoping having it here too saves someone some grief. Thank you. --kyler
What version of python, zope, and os? I have been able to treat them as a pure string so far in my python products in linux even with several meg ones. Python 1.5.2, zope 2.3.2, and debian linux. If you are using windows writing to a temporary file could kill you anyways because of the difference ascii/binary difference on windows. Designing the webpages of tomorrow http://webme-eng.com Designing the MMORPGS of tomorrow http://worldforge.org On Thu, 21 Jun 2001, Kyler B. Laird wrote:
I learned my lesson for today, so I thought I'd share it.
I am passing around large strings containing PDF and PS documents. Sometimes I make a call like document = container.PJdoc(container['proposal'].data) where the object "proposal" is a PDF file.
I end up writing this data to a temporary file in my PJ class. I noticed early on that I could not simply write the whole string, but I worked around that by writing it in 1K blocks.
I had my warning.
Today my dumb kludge came back to bite me. We found that some of our proposals were rejected by the PDF parser. I started looking at the temporary files and noticed the problem ones were 98798 bytes in length. The PDF file sent was much larger.
After some tests, I determined that asking for a slice from .data only returns a null string (zero length) after 98798 bytes. This meant that my little 1K write loop was happily going through the "string" but it was writing 0K blocks after awhile.
So...I finally fixed the problem. Instead of all of the 1K writing junk, I simply cast whatever is passed to me into a real string. Then I can write the whole thing without any problems.
I'm sure this is documented somewhere that I should have read before, but I'm hoping having it here too saves someone some grief.
Thank you.
--kyler
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
On Thu, 21 Jun 2001 15:55:15 -0600 (MDT) you wrote:
What version of python, zope, and os?
Python 1.5.2 Zope 2.3.3 Mandrake 8.0 Linux 2.4.5-ac9 SMP The little test External Method I used to verify this was essentially this. def strlen(text): printed = 'len=%d\n' % len(text) for i in range(0, len(text), 1024): printed = printed + '%d: %d\n' % (i, len(text[i:])) The output is len=811008 0: 90112 1024: 89088 2048: 88064 3072: 87040 4096: 86016 5120: 84992 6144: 83968 7168: 82944 8192: 81920 9216: 80896 . . . 71680: 18432 72704: 17408 73728: 16384 74752: 15360 75776: 14336 76800: 13312 77824: 12288 78848: 11264 79872: 10240 80896: 9216 81920: 8192 82944: 7168 83968: 6144 84992: 5120 86016: 4096 87040: 3072 88064: 2048 89088: 1024 90112: 0 91136: 0 92160: 0 93184: 0 94208: 0 95232: 0 96256: 0 97280: 0 98304: 0 99328: 0 100352: 0 . . . 807936: 0 808960: 0 809984: 0 (Note that it breaks in a different place than I noticed before. I'm using a different file, I think.) All I have to do to make it work is throw "str()" around "text". --kyler
participants (2)
-
kosh＠aesaeion.com -
Kyler B. Laird