Using page templates to generate large XML files
Hi all. I need to generate a large XML file inside a Zope application; the file is for backup purposes, so it is stored on the server and can be downloaded later with whatever suitable tool. The data are taken from a database, and then they are used to create the XML. I assume there will be between 40.000 and 100.000 rows to be dumped. I thought i could use page template for this purpose, but I'm not sure if it is possible to send the output from a pt to a file, directly. Of course, this is a problem which can be easily solved. I wonder however if the PT has to generate the whole XML in just one step; in that case, maybe I should resort to use PT on one row at time, and write the data to different files. Thanks for your time. Regards Marco -- Marco Bizzarri http://notenotturne.blogspot.com/ http://iliveinpisa.blogspot.com/
--On 6. September 2008 08:26:17 +0200 Marco Bizzarri <marco.bizzarri@gmail.com> wrote:
Hi all.
I need to generate a large XML file inside a Zope application; the file is for backup purposes, so it is stored on the server and can be downloaded later with whatever suitable tool.
The data are taken from a database, and then they are used to create the XML. I assume there will be between 40.000 and 100.000 rows to be dumped.
I thought i could use page template for this purpose, but I'm not sure if it is possible to send the output from a pt to a file, directly. Of course, this is a problem which can be easily solved.
If you're using Zope 2.10 or higher, you can either use the "native" ZPT implementation from Zope 3 (see zope.pagetemplate module) or you use the Zope 2 implementation (which is basically only a wrapper around the Z3 implementation). In this case you might look at the unittests in testZopePageTemplates.py or otherwise you check the doctests of zope.pagetemplate. -aj
Marco Bizzarri wrote at 2008-9-6 08:26 +0200:
I need to generate a large XML file inside a Zope application; the file is for backup purposes, so it is stored on the server and can be downloaded later with whatever suitable tool.
The data are taken from a database, and then they are used to create the XML. I assume there will be between 40.000 and 100.000 rows to be dumped.
I thought i could use page template for this purpose, but I'm not sure if it is possible to send the output from a pt to a file, directly.
I fear, this is not a supported usecase: The standard page template implementation writes to a "StringIO" object (thus its internal infrastructure is prepared for incremental output generation) but the file to be written to it not controllable from outside. If your XML is not too complex, I would directly generate it (and not go through a PageTemplate). -- Dieter
--On 6. September 2008 20:34:21 +0200 Dieter Maurer <dieter@handshake.de> wrote:
Marco Bizzarri wrote at 2008-9-6 08:26 +0200:
I need to generate a large XML file inside a Zope application; the file is for backup purposes, so it is stored on the server and can be downloaded later with whatever suitable tool.
The data are taken from a database, and then they are used to create the XML. I assume there will be between 40.000 and 100.000 rows to be dumped.
I thought i could use page template for this purpose, but I'm not sure if it is possible to send the output from a pt to a file, directly.
I fear, this is not a supported usecase:
The standard page template implementation writes to a "StringIO" object (thus its internal infrastructure is prepared for incremental output generation) but the file to be written to it not controllable from outside.
Of course you can write the rendered result to a file. However the construction will occur in memory and might require lots of memory. No idea about the structure and the complexity of the XML document.... generating it through Python is very likely faster and will requires less resources. -aj
participants (3)
-
Andreas Jung -
Dieter Maurer -
Marco Bizzarri