I am testing the possibilities of delivering the content of XML Documents through the Zope environment. Unfortunately, some of the proposed file are rather large (up to 760KB) and just uploading them and viewing them on our current Zope server is prohibitively slow. Our server, running Z2 is a blazing P133 running NT 4.0 with 32 MB of RAM (I get the bottom feeders). Is the bottleneck the hardware; is there something I can do software-wise to improve performance; or is development not yet advanced enough to handle this scenario efficiently? Any opinions on this? Thanks, P. Johnson
You could check into the ExtFile product (search on zope.org) and see if that makes a perf difference. That said, just about anything on that setup is going to be slow!! Please excuse the os advocacy, but given that this is likely all you'd dream of running on this server anyways (ie it's not like it's also going to be your Exchange server), you might consider putting something that will leave more spare RAM and cpu time for Zope on that box, e.g. Linux or some other OS more suited towards the low end server hardware market. Or buy a nicer box to run this on. If you don't have existing unix skills in-house then throwing money at hardware might be less expensive than investing in human resources to get a Linux box going. (Whew, being even-handed is so much harder than being an OS bigot...;-) Sorry that's probably not what you wanted to hear. =) jim On Tue, 5 Dec 2000 paul_s_johnson@urscorp.com wrote:
I am testing the possibilities of delivering the content of XML Documents through the Zope environment. Unfortunately, some of the proposed file are rather large (up to 760KB) and just uploading them and viewing them on our current Zope server is prohibitively slow. Our server, running Z2 is a blazing P133 running NT 4.0 with 32 MB of RAM (I get the bottom feeders). Is the bottleneck the hardware; is there something I can do software-wise to improve performance; or is development not yet advanced enough to handle this scenario efficiently? Any opinions on this?
Thanks,
P. Johnson
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Definetly consider a stripped down linux install for this machine instead. just make sure almost nothing else is running as well (A lots ditros start all sorts of stuff up for you). Edward Muller - edwardam@home.com - edwardam@handhelds.org Grow a ponytail -- view it as your telepathic antenna to other Linux Kernel Developers. -- Jeff V. Merkey On Tue, 5 Dec 2000, Jim Hebert wrote:
You could check into the ExtFile product (search on zope.org) and see if that makes a perf difference.
That said, just about anything on that setup is going to be slow!! Please excuse the os advocacy, but given that this is likely all you'd dream of running on this server anyways (ie it's not like it's also going to be your Exchange server), you might consider putting something that will leave more spare RAM and cpu time for Zope on that box, e.g. Linux or some other OS more suited towards the low end server hardware market.
Or buy a nicer box to run this on. If you don't have existing unix skills in-house then throwing money at hardware might be less expensive than investing in human resources to get a Linux box going. (Whew, being even-handed is so much harder than being an OS bigot...;-)
Sorry that's probably not what you wanted to hear. =) jim
On Tue, 5 Dec 2000 paul_s_johnson@urscorp.com wrote:
I am testing the possibilities of delivering the content of XML Documents through the Zope environment. Unfortunately, some of the proposed file are rather large (up to 760KB) and just uploading them and viewing them on our current Zope server is prohibitively slow. Our server, running Z2 is a blazing P133 running NT 4.0 with 32 MB of RAM (I get the bottom feeders). Is the bottleneck the hardware; is there something I can do software-wise to improve performance; or is development not yet advanced enough to handle this scenario efficiently? Any opinions on this?
Thanks,
P. Johnson
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Paul, The XMLDocument type is rather 'expensive' and you may be able to 'get away' with using a simpler type such as DTMLDocument. If you don't need the DOMification of your XML when you put it into Zope this is something i'd look at.. I have done this with large documents and then used an external method to process them with XSLT, XPath etc. hth Phil phil.harris@zope.co.uk ----- Original Message ----- From: <paul_s_johnson@urscorp.com> To: <zope@zope.org> Sent: Tuesday, December 05, 2000 10:01 PM Subject: [Zope] Large XML files
I am testing the possibilities of delivering the content of XML Documents through the Zope environment. Unfortunately, some of the proposed file are rather large (up to 760KB) and just uploading them and viewing them on our current Zope server is prohibitively slow. Our server, running Z2 is a blazing P133 running NT 4.0 with 32 MB of RAM (I get the bottom feeders). Is the bottleneck the hardware; is there something I can do software-wise to improve performance; or is development not yet advanced enough to handle this scenario efficiently? Any opinions on this?
Thanks,
P. Johnson
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
"Parsed XML" (the successor to XML Document) should handle large documents less expensively. No code has been released yet, but something should be available soon (in a couple of weeks?)... Keep an eye on http://www.zope.org/Wikis/DevSite/Projects/ParsedXML and you may want to subscribe to the zope-xml mailing list. --- "Sardonic Slaughterer of Sacred Cows" Todd Coram - mailto:todd@digicool.com Digital Creations, Inc - http://www.digicool.com Publishers of Zope - http://www.zope.org
-----Original Message----- From: zope-admin@zope.org [mailto:zope-admin@zope.org]On Behalf Of Phil Harris Sent: Tuesday, December 05, 2000 6:03 PM To: paul_s_johnson@urscorp.com; zope@zope.org Subject: Re: [Zope] Large XML files
Paul,
The XMLDocument type is rather 'expensive' and you may be able to 'get away' with using a simpler type such as DTMLDocument.
If you don't need the DOMification of your XML when you put it into Zope this is something i'd look at..
I have done this with large documents and then used an external method to process them with XSLT, XPath etc.
hth
Phil phil.harris@zope.co.uk
----- Original Message ----- From: <paul_s_johnson@urscorp.com> To: <zope@zope.org> Sent: Tuesday, December 05, 2000 10:01 PM Subject: [Zope] Large XML files
I am testing the possibilities of delivering the content of XML Documents through the Zope environment. Unfortunately, some of the proposed file are rather large (up to 760KB) and just uploading them and viewing them on our current Zope server is prohibitively slow. Our server, running Z2 is a blazing P133 running NT 4.0 with 32 MB of RAM (I get the bottom feeders). Is the bottleneck the hardware; is there something I can do software-wise to improve performance; or is development not yet advanced enough to handle this scenario efficiently? Any opinions on this?
Thanks,
P. Johnson
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
_______________________________________________ Zope maillist - Zope@zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Hey Guys and Gals, Can anyone help me to Retrieve, Update, Insert image files into a Database in Zope. I am running zope on linux mandrake and use ZSever The database I am currently using is MySQL on MySQLDA (MySQLdb non-binary dist.) Mo.
On Tue, 5 Dec 2000 23:02:38 -0000, "Phil Harris" <phil.harris@zope.co.uk> wrote:
The XMLDocument type is rather 'expensive' and you may be able to 'get away' with using a simpler type such as DTMLDocument.
Simpler, but thats not necessarily an advantage. DTMLDocument will store the whole document in memory, but XMLDocument uses ZODB effectively so that it only loads the DOM nodes that are in use. If you are always using the *whole* document then this is no help, and simpler may indeed be better.
From: <paul_s_johnson@urscorp.com>
I am testing the possibilities of delivering the content of XML Documents through the Zope environment. Unfortunately, some of the proposed file are rather large (up to 760KB) and just uploading them and viewing them on our current Zope server is prohibitively slow. Our server, running Z2 is a blazing P133 running NT 4.0 with 32 MB of RAM (I get the bottom feeders). Is the bottleneck the hardware; is there something I can do software-wise to improve performance; or is development not yet advanced enough to handle this scenario efficiently? Any opinions on this?
For a machine of that size I suggest you use only one publisher thread (thats -T 1 on the command line), rather than the default of 4. Each thread gets a copy of the ZODB object cache, and you probably dont want to keep four copies of your 700k document in memory. Toby Dickenson tdickenson@geminidataloggers.com
participants (7)
-
Edward Muller -
Jim Hebert -
Mohan Baro -
paul_s_johnsonļ¼ urscorp.com -
Phil Harris -
Toby Dickenson -
Todd Coram