Threaded Database Access
I'm using Zope/Zserver 2.2.4, ZODBCDA v3.1.0b2 and SQL Server 2000. I've noticed that pages generated from database querries are definitely single threaded, and these pages are the bulk of my site. My poking around seems to indicate that ZODBCDA is the single-threaded culprit, and some emails from the list over the last couple months seem to confirm that. The volume of my site and the size of the database (hence query response times) are getting to the point that this is a concern. So I'm investigating my options, including: 1) Cluster the web site with multiple single processor machines. 2) Move to a database that Zope supports multi-threaded access to. 3) ? I've been considering a move to Oracle or PostgreSQL/MySQL for other reasons, but I'm not going to do it just for yucks. Extending and populating my site gives better return than porting the database. I'd like to make sure I'll get return on my time investment. I saw an document on the Zope web site from 1999 that talked about labeling Zope DAs Level 1,2 or 3 based on their sophistication, but I don't see any of the current DAs labeled as such. There was text in the Oracle and MySQL DA descriptions that seem to imply multi-threaded behaivor though. I'm looking for the info from the community on: 1. Is the ZODBCDA single-threaded and might that change anytime soon? 2. Can someone confirm that the Oracle, PostgreSQL or MySQL DAs are multi-threaded. 3. Does the multi-threading require using the Zope Session facility or some other feature? (My site uses FSSession.) Thanks in advance for any input. Jeff Nielsen / UgoFast http://www.UgoFast.com Jeff@UgoFast.com
ZPsycopgDA (for postgres) is definitely multi-threaded. I think zPopyDA is too. Another solution might be to setup your site using ZEO and several single threaded ZEO Clients. Then put a load balancer in front of it. Then the single-threadedness is much less of an issue, since you have multiple instances handling the requests. You might need more hardware (or at least more RAM) to make this pay off though. -Casey On Thursday 02 May 2002 12:43 am, Jeff Nielsen / UgoFast allegedly wrote:
I'm using Zope/Zserver 2.2.4, ZODBCDA v3.1.0b2 and SQL Server 2000. I've noticed that pages generated from database querries are definitely single threaded, and these pages are the bulk of my site. My poking around seems to indicate that ZODBCDA is the single-threaded culprit, and some emails from the list over the last couple months seem to confirm that.
The volume of my site and the size of the database (hence query response times) are getting to the point that this is a concern. So I'm investigating my options, including: 1) Cluster the web site with multiple single processor machines. 2) Move to a database that Zope supports multi-threaded access to. 3) ?
I've been considering a move to Oracle or PostgreSQL/MySQL for other reasons, but I'm not going to do it just for yucks. Extending and populating my site gives better return than porting the database. I'd like to make sure I'll get return on my time investment.
I saw an document on the Zope web site from 1999 that talked about labeling Zope DAs Level 1,2 or 3 based on their sophistication, but I don't see any of the current DAs labeled as such. There was text in the Oracle and MySQL DA descriptions that seem to imply multi-threaded behaivor though.
I'm looking for the info from the community on: 1. Is the ZODBCDA single-threaded and might that change anytime soon? 2. Can someone confirm that the Oracle, PostgreSQL or MySQL DAs are multi-threaded. 3. Does the multi-threading require using the Zope Session facility or some other feature? (My site uses FSSession.)
Thanks in advance for any input.
Jeff Nielsen / UgoFast http://www.UgoFast.com Jeff@UgoFast.com
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
--On 02 May 2002 10:08 -0400 Casey Duncan <casey@zope.com> wrote:
ZPsycopgDA (for postgres) is definitely multi-threaded. I think zPopyDA is too.
http://www.zope.org/Members/glpb/solaris says:
The report also investigates in a cursory way the effects of adding a database adapter to the equation. A test was carried out to compare the effect of changing the threading model on a site doing queries against an Oracle database using DCOracle2. It was found that queries that involved writes on the database were forced to be completed in serial (this is unsuprising), whilst read requests could be carried out in parallel. In addition, ZSQL Methods have the ability to cache the rows they fetch from a database for a given query. The results show that caching of SQL queries can produce a significant performance boost.
HTH Paul -- The Library, Tyndall Avenue, Univ. of Bristol, Bristol, BS8 1TJ, UK E-mail: paul.browning@bristol.ac.uk URL: http://www.bris.ac.uk/
Thanks for the info guys. It looks like these might do the trick. -----Original Message----- From: Paul Browning [mailto:paul.browning@bristol.ac.uk] Sent: Thursday, May 02, 2002 10:55 AM To: Casey Duncan Cc: Jeff Nielsen / UgoFast; zope-dev Subject: Re: [Zope-dev] Threaded Database Access --On 02 May 2002 10:08 -0400 Casey Duncan <casey@zope.com> wrote:
ZPsycopgDA (for postgres) is definitely multi-threaded. I think zPopyDA is too.
http://www.zope.org/Members/glpb/solaris says:
The report also investigates in a cursory way the effects of adding a database adapter to the equation. A test was carried out to compare the effect of changing the threading model on a site doing queries against an Oracle database using DCOracle2. It was found that queries that involved writes on the database were forced to be completed in serial (this is unsuprising), whilst read requests could be carried out in parallel. In addition, ZSQL Methods have the ability to cache the rows they fetch from a database for a given query. The results show that caching of SQL queries can produce a significant performance
boost.
HTH Paul -- The Library, Tyndall Avenue, Univ. of Bristol, Bristol, BS8 1TJ, UK E-mail: paul.browning@bristol.ac.uk URL: http://www.bris.ac.uk/
I have a product out that i designed to allow querying of an ODBC database from any platform using ODBCSocketServer on the database machine (As this has been turned commercial, and the opensource code is now not being maintained, I am developing a complete replacement - out on SourceForge soon). I have used this quite successfully in some tests that I have run, and this is certainly multithreaded on the server and client sides. Look for ZDataQueryKit ( http://www.zope.org/Members/haqa/ZDataQueryKit ) and ODBC Socket Server (You'll have to search sourceforge for the new project for that as I have lost the URL.) Drop me a line if you are interested in my new ODBCServer project, which will include some major protocol enhancements, including schema queries, and result compression. Adrian... ----- Original Message ----- From: "Jeff Nielsen / UgoFast" <Ugo@speakeasy.net> To: <zope-dev@zope.org> Sent: Thursday, May 02, 2002 5:43 AM Subject: [Zope-dev] Threaded Database Access
I'm using Zope/Zserver 2.2.4, ZODBCDA v3.1.0b2 and SQL Server 2000. I've noticed that pages generated from database querries are definitely single threaded, and these pages are the bulk of my site. My poking around seems to indicate that ZODBCDA is the single-threaded culprit, and some emails from the list over the last couple months seem to confirm that.
The volume of my site and the size of the database (hence query response times) are getting to the point that this is a concern. So I'm investigating my options, including: 1) Cluster the web site with multiple single processor machines. 2) Move to a database that Zope supports multi-threaded access to. 3) ?
I've been considering a move to Oracle or PostgreSQL/MySQL for other reasons, but I'm not going to do it just for yucks. Extending and populating my site gives better return than porting the database. I'd like to make sure I'll get return on my time investment.
I saw an document on the Zope web site from 1999 that talked about labeling Zope DAs Level 1,2 or 3 based on their sophistication, but I don't see any of the current DAs labeled as such. There was text in the Oracle and MySQL DA descriptions that seem to imply multi-threaded behaivor though.
I'm looking for the info from the community on: 1. Is the ZODBCDA single-threaded and might that change anytime soon? 2. Can someone confirm that the Oracle, PostgreSQL or MySQL DAs are multi-threaded. 3. Does the multi-threading require using the Zope Session facility or some other feature? (My site uses FSSession.)
Thanks in advance for any input.
Jeff Nielsen / UgoFast http://www.UgoFast.com Jeff@UgoFast.com
_______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
participants (4)
-
Adrian Hungate -
Casey Duncan -
Jeff Nielsen / UgoFast -
Paul Browning