March 2001 - Zope-Dev

DB connection pooling
by Erik Stephens 02 Mar '01

02 Mar '01

Hello all, I've been working on an implementation of connection pooling for the MySQLDA. Basically, Zope connections will be pooled by connection string to reduce the number of unecessary connections to MySQL. The number of connections will be reduced in 2 respects: 1) Multiple threads will share the connection pool and 2) Zope connection objects that use the same connection will be pooled. A slight bit of run-time overhead will be introduced from having to look up connection objects by connection string, but the trade-off will be well worth it, at least for our setup. Correct me if I'm wrong, but without pooling, the number of connections that could be created is: <num Zope servers> * <num Zope threads> * <num connection objects>. With pooling, this number becomes: <num Zope servers> * <num distinct connection strings> * X. X can range from 1 to <num Zope threads>, depending on how high load is. Question to y'all is this: Is this a worthwhile endeavor? Would others find this useful? Or are we just weird? The new release of the postgresql dba announce today mentioned "automatic management of multiple phisical connections to the DB (one per thread) to speed up queries". Is this connection pooling in some form? Thanks, Erik

1 0

Re: [Zope-dev] TransWarp preview release (AOP tools only)
by Phillip J. Eby 02 Mar '01

02 Mar '01

At 03:53 PM 3/2/01 -0500, R. David Murray wrote: >On Fri, 2 Mar 2001, Shane Hathaway wrote: >> "Phillip J. Eby" wrote: >> > One reason I've been racing like mad to get the preview release out is so >> > that, after seeing the usefulness of TW for the Zope 3 component >> > architecture, y'all might be interested in considering implementations for >> > module persistence besides proxying. :) IMHO, a "rollback importer" >> > approach ala unittest might be better suited for TW-ish components, and >> > have fewer weird side effects for persistent modules generally. I don't >> > like unittest's mechanism for *doing* the rollback, and Zope would need a >> > thread-specific version of sys.modules, as well as a mechanism to purge the >> > cache of any objects which were loaded after any of the purged modules, but >> > I think the basic idea is sound. >> >> Hmm. I'm afraid I don't see how TW helps with module persistence. > >I could be wrong, but I didn't read what Phillip wrote as saying he was >suggesting TransWarp as a player in the Module Persistence implementation, >but rather that he wanted people to see the value of TransWarp first hand >so that they would want to make sure that Module Persistence and TransWarp >could play together.... > Yes, that's exactly what I meant. :)

2 1

Re: [Zope-dev] dtml-in: next-batches
by Dieter Maurer 02 Mar '01

02 Mar '01

Brett Carter writes: > I'm running Zope 2.2.2, and I'm trying to use the batching stuff for > the dtml-in tag, however, I can't seem to access any attributes off > the 'next-batches' or 'previous-batches' mapping object that's > supposed to be available, with the attributes 'batch-start-index', > 'batch-end-index', and 'batch-size'. > .... > I wrote an external method to dir the 'next-batches' object, and it > seems the only attributes available are: > ['data', 'items', 'query_string', 'start_name_re'] The attributes your are looking for are in fact keys of the mapping 'data'. Dieter

1 0

Re: [Zope-dev] Increased system CPU cycles under heavy load?
by Dieter Maurer 02 Mar '01

02 Mar '01

Erik Stephens writes: > Does anyone know what kinds of operations fall under 'system'? Task > switching, paging, these kinds of things? The computer can execute instruction being part of the operating system (usually priviledged code) or of user processes. The first would be accounted for as system, the latter as user time. System time includes (as you expect) scheduling, but also memory management (swapping in and out, paging in and out), IO activity, inter-process communication and all other operating system services. Dieter

1 0

New PostgreSQL Database Adapters: psycopg and ZpsycopgDA
by Federico Di Gregorio 02 Mar '01

02 Mar '01

Hi *, the guys here at initd (http://initd.org/) arre happy to announce the release of psycopg, a Python/PostgreSQL driver module and ZPsycopgDA, a Zope Database Adapter based on the psycopg driver. Now, I'll answer the questions you asked... (_what_ questions? well... follow me...) Q. Why another postgresql dba? we already have pygres and popy! A. That's true. We had some ideas about a performant, thread-safe driver and we wanted to hack a little bit. the resulting driver is slim, fast, with clean code and some nice features (see below.) Writing a Zope DBA was a logical consequence only because we use it. Q. Should I switch to psycopg and zpsycopgda? A. Not if you are happy with your current driver [and i hope it is popy ;-)] The only real reason is if you want the advanced features you can find only in psycopg: 1/ level-2 thread safety 2/ advanced type-casting system 3/ automatic management of multiple phisical connections to the DB (one per thread) to speed up queries 4/ extremely clean and maintainable code also, in a future release (0.5) zpsycopgda will support db and table management from zope management screens. Q. Hey! you're from MIXAD LIVE, is psycopg a popy fork? A. No, psycopg is 100% clean code. we had some ideas on how a db driver should workin a heavily multithreaded environment and we wanted to test them. that's all. Q. Where can I download it? A. http://initd.org/Software/psycopg (tarball and precompiled debian packages) ciao and good hacking, federico -- Federico Di Gregorio MIXAD LIVE Chief of Research & Technology fog(a)mixadlive.com Debian GNU/Linux Developer & Italian Press Contact fog(a)debian.org Qu'est ce que la folie? Juste un sentiment de liberté si fort qu'on en oublie ce qui nous rattache au monde... -- J. de Loctra

2 2

Re: [Zope-dev] dtml-in: next-batches
by Brett Carter 02 Mar '01

02 Mar '01

I've also discovered that setting overlap = 0 actually causes everything to overlap by 1. I didn't see anything in the collector about this either... -Brett >>>>> "Brett" == Brett Carter <brett(a)kavi.com> writes: Brett> I'm running Zope 2.2.2, and I'm trying to use the batching stuff for Brett> the dtml-in tag, however, I can't seem to access any attributes off Brett> the 'next-batches' or 'previous-batches' mapping object that's Brett> supposed to be available, with the attributes 'batch-start-index', Brett> 'batch-end-index', and 'batch-size'. However, my example code (see Brett> below) blows up with a key error when trying to access any of these. Brett> Is this a bug, or am I just misunderstanding how to use this thing? Brett> I wrote an external method to dir the 'next-batches' object, and it Brett> seems the only attributes available are: Brett> ['data', 'items', 'query_string', 'start_name_re'] Brett> -Brett Brett> Brett> <dtml-in "mystuff.retseq()" size=5 start=start> Brett> <dtml-var sequence-item> Brett> </dtml-in> Brett> Brett> <dtml-in "mystuff.retseq()" size=5 start=start previous> Brett> <a href="<dtml-var absolute_url>/batch_me?start=<dtml-var previous-sequence-start-index>">Previous</a> Brett> </dtml-in> Brett> <dtml-in "mystuff.retseq()" size=5 start=start next overlap=-1> Brett> <dtml-try> Brett> <dtml-in next-batches> Brett> <dtml-var batch-start-index> Brett> </dtml-in> Brett> <dtml-except> Brett> <pre> Brett> <dtml-var error_type> Brett> <dtml-var error_value> Brett> <dtml-var error_tb> Brett> </pre> Brett> </dtml-try> Brett> <a href="<dtml-var absolute_url>/batch_me?start=<dtml-var next-sequence-start-index>">Next</a> Brett> </dtml-in> Brett> Brett> _______________________________________________ Brett> Zope-Dev maillist - Zope-Dev(a)zope.org Brett> http://lists.zope.org/mailman/listinfo/zope-dev Brett> ** No cross posts or HTML encoding! ** Brett> (Related lists - Brett> http://lists.zope.org/mailman/listinfo/zope-announce Brett> http://lists.zope.org/mailman/listinfo/zope )

1 0

Re: [Zope-dev] SessionDataManager oddness
by Chris McDonough 02 Mar '01

02 Mar '01

Not really. In this case it seems appropriate, in others its not really necessary. I hope my suggestion works. ;-) ----- Original Message ----- From: "Matt Hamilton" <matth(a)netsight.co.uk> To: "Chris McDonough" <chrism(a)digicool.com> Sent: Friday, March 02, 2001 10:43 AM Subject: Re: [Zope-dev] SessionDataManager oddness > On Fri, 2 Mar 2001, Chris McDonough wrote: > > > Try putting the code which adds the session data manager from your class' > > __init__ into a manage_afterAdd method of your class, e.g.: > > > > def manage_afterAdd(self, item, container): > > ob = SessionDataManager( > > id='session_data_mgr', title='', path=None, timeout_mins=20) > > self._setObject('session_data_mgr', ob) > > > > manage_afterAdd is called automagically by the _setObject machinery at init > > time, but it gets the object in its acquisition-wrapped context. > > OK, thanks, I'll give that a try. For future reference, is there where I > should be putting in all code to add objects to a new instance of a > folderish class? Currently it is in __init__ (I add a couple of > BTreeFolders, and a ZCatalog). > > -Matt > > -- > Matt Hamilton matth(a)netsight.co.uk > Netsight Internet Solutions, Ltd. Business Vision on the Internet > http://www.netsight.co.uk +44 (0)117 9090901 > Web Hosting | Web Design | Domain Names | Co-location | DB Integration > > >

1 0

SessionDataManager oddness
by Matt Hamilton 02 Mar '01

02 Mar '01

Dear All, I am trying to add session tracking to a Product of mine. I am trying to add a SessionDataManager to a Folderish object in the __init__ method of the class, but having troubles. I think I am importing everything: from Globals import HTMLFile from Globals import Persistent from Globals import default__class_init__ import Acquisition import AccessControl.Role import OFS.Folder from Products.ZCatalog import ZCatalog from Products.BTreeFolder import BTreeFolder from Products.CoreSessionTracking.SessionDataManager import SessionDataManager from Products.CoreSessionTracking.SessionDataContainer import SessionDataContainerMount from Products.CoreSessionTracking.SessionData import SessionData try: from Products.CoreSessionTracking.SessioningInterfaces import SessionDataManagerInterface except: # pre-2.3 Zopes don't have an Interfaces module class SessionDataManagerInterface: pass then in my __init__ method: ob = SessionDataManager( id='session_data_mgr', title='', path=None, timeout_mins=20) self._setObject('session_data_mgr', ob) However I get an attribute error when trying to add an intance of the product. _sdc not found. I have traced it using pdb, and from what I can find out it appears as if it fails in _setObject. More specifically SessionDataContainerMount seems to not returning the correct thing. I have tried tracing the normal addition of a SessionDataManager in Zope and it works fine. SessionDataContainerMount doesn't have an __init__ method, but it looks like when I add a S.D.M via the normal web interface __of__ gets called, but when my product tries to add it __del__ gets called. Any ideas? I am at the end of my Zope internals knowledge.... if there is anything else I can provide to help diagnose, let me know. TIA, Matt -- Matt Hamilton matth(a)netsight.co.uk Netsight Internet Solutions, Ltd. Business Vision on the Internet http://www.netsight.co.uk +44 (0)117 9090901 Web Hosting | Web Design | Domain Names | Co-location | DB Integration

2 1

ZDebug conflicts with MailHost?
by Itai Tavor 02 Mar '01

02 Mar '01

Hi, On Zope 2.3.0 and 2.3.1b1, the following method produces an error when ZDebug is installed; If I remove ZDebug, it works fine. <dtml-sendmail mailhost=MailHost> To: itai(a)optusnet.com.au From: itai(a)optusnet.com.au Subject: Test This is a test </dtml-sendmail> With ZDebug the content of the sendmail tag is ignored; sendmail.send is called with all arguments empty. Traceback: Error type: TypeError Error value: len() of unsized object Traceback (innermost last): File /opt/Zope-2.3.0-linux2-x86/lib/python/ZPublisher/Publish.py, line 222, in publish_module File /opt/Zope-2.3.0-linux2-x86/lib/python/ZPublisher/Publish.py, line 187, in publish File /opt/Zope-2.3.0-linux2-x86/lib/python/Zope/__init__.py, line 221, in zpublisher_exception_hook (Object: ApplicationDefaultPermissions) File /opt/Zope-2.3.0-linux2-x86/lib/python/ZPublisher/Publish.py, line 171, in publish File /opt/Zope-2.3.0-linux2-x86/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: mailtest) File /opt/Zope-2.3.0-linux2-x86/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: mailtest) File /opt/Zope-2.3.0-linux2-x86/lib/python/OFS/DTMLMethod.py, line 189, in __call__ (Object: mailtest) File /opt/Zope/lib/python/Products/ZDebug/DTMLLocator.py, line 103, in __call__ (Object: mailtest) File /opt/Zope-2.3.0-linux2-x86/lib/python/DocumentTemplate/DT_String.py, line 538, in __call__ (Object: mailtest) File /opt/Zope/lib/python/Products/ZDebug/DTMLDebug.py, line 307, in debug_render_blocks File /opt/Zope/lib/python/Products/ZDebug/DTMLDebug.py, line 239, in debugException (Object: MailHost) File /opt/Zope/lib/python/Products/ZDebug/DTMLDebug.py, line 305, in debug_render_blocks File /opt/Zope-2.3.0-linux2-x86/lib/python/Products/MailHost/SendMailTag.py, line 188, in render (Object: MailHost) File /opt/Zope-2.3.0-linux2-x86/lib/python/Products/MailHost/MailHost.py, line 222, in send (Object: MailHost) File /opt/Zope-2.3.0-linux2-x86/lib/python1.5/smtplib.py, line 465, in sendmail File /opt/Zope-2.3.0-linux2-x86/lib/python1.5/smtplib.py, line 349, in mail File /opt/Zope-2.3.0-linux2-x86/lib/python1.5/smtplib.py, line 118, in quoteaddr File /opt/Zope-2.3.0-linux2-x86/lib/python1.5/rfc822.py, line 451, in parseaddr File /opt/Zope-2.3.0-linux2-x86/lib/python1.5/rfc822.py, line 496, in getaddrlist File /opt/Zope-2.3.0-linux2-x86/lib/python1.5/rfc822.py, line 504, in getaddress File /opt/Zope-2.3.0-linux2-x86/lib/python1.5/rfc822.py, line 484, in gotonext TypeError: (see above) -- -- Itai Tavor -- "Je sautille, donc je suis." -- itai(a)optusnet.com.au -- - Kermit the Frog -- -- -- -- "If you haven't got your health, you haven't got anything" --

1 0

dtml-in: next-batches
by Brett Carter 02 Mar '01

02 Mar '01

I'm running Zope 2.2.2, and I'm trying to use the batching stuff for the dtml-in tag, however, I can't seem to access any attributes off the 'next-batches' or 'previous-batches' mapping object that's supposed to be available, with the attributes 'batch-start-index', 'batch-end-index', and 'batch-size'. However, my example code (see below) blows up with a key error when trying to access any of these. Is this a bug, or am I just misunderstanding how to use this thing? I wrote an external method to dir the 'next-batches' object, and it seems the only attributes available are: ['data', 'items', 'query_string', 'start_name_re'] -Brett <dtml-in "mystuff.retseq()" size=5 start=start> <dtml-var sequence-item> </dtml-in> <dtml-in "mystuff.retseq()" size=5 start=start previous> <a href="<dtml-var absolute_url>/batch_me?start=<dtml-var previous-sequence-start-index>">Previous</a> </dtml-in> <dtml-in "mystuff.retseq()" size=5 start=start next overlap=-1> <dtml-try> <dtml-in next-batches> <dtml-var batch-start-index> </dtml-in> <dtml-except> <pre> <dtml-var error_type> <dtml-var error_value> <dtml-var error_tb> </pre> </dtml-try> <a href="<dtml-var absolute_url>/batch_me?start=<dtml-var next-sequence-start-index>">Next</a> </dtml-in>

1 0