September 2006 - Zope-Dev

In Defense of ZClasses
by Christopher Lozinski 28 Sep '06

28 Sep '06

At the risk of going down in history (and Google Searches) as the man who supports ZClasses, I think that someone deserves to come to their defense. I think that there is a different tool for every job. Sometimes I think Plone is the best solution, sometimes Zope 2 is the best solution. Sometimes Zope 3is the best solution, and sometimes ZClasses are the best solution. When are ZClasses the best solution? Frequently a ZClass is the fastest way to build an application. I can put up a simple list of types in ZClasses and DTML so incredibly quickly. Frankly developing a file system based python application is just way too much overhead for bringing up simple web applications quickly. In fact I have pushed ZClasses pretty far. If you are building an application, and you are a single developer, and you are comfortable with a tree based data model, then ZClasses is probably still your fastest development tool. I was recently told that ZClasses now work with Zope 2.8.* Maybe it was 2.9.* Frankly I am not smart enough to understand the recent evolution of the Zope framework. But I suspect that ZClasses can be made to work for a while more on top of the newest releases of Zope, and that is good enough for me. Someone will be motivated enough to make it work, and I will take advantage of the resulting open source code. I also think that there is a bright future for ZClasses, in the niches that I am interested in. Many years ago, I used to use versions to develop production and development code on a single server. I was in heaven. It was so easy. It was so productive. Then came zcatalogs and all of that broke. Pretty soon, I think someone, maybe even me, will be motivated enough to make ZClasses work with MVCC. Multivalued Concurrency Control, so that we can get rid of our development servers, and as a single developer just run a simple cheap development/production server. Life will be great. But I see a brighter future for ZClasses as well. The demand for my services is booming. I have way more business than I can handle. Someone recently did a search for a plone resume on Google, and I got 6 of the top placements. We have heard of search engine optimization, but this is just crazy good. Of course managing a staff of recruiters is the hardest part. So soon I will need to run multiple servers with data being shared among them. I will need a distributed schema, where people enter their resumes on one job board, and have them distributed to several other job boards. Zope 3 has a single server schema, but I have no idea if I will be able to expand it to a distributed Schema. With Zope 2, and ZClasses, it is very clear to me how I can build a distributed schema that moves classes around between servers. Most of the work is already done for me, another huge chunk, the schema part, I have done. So I think Zclasses serve my needs wonderfully. And I think that I can upgrade them with MVCC to make a single developer even more productive. And I think that I can build powerful distributed applications on top of them. That will be so much easier than trying to build and move python code on the file system and restarting dozens of servers. And it only takes a few of us using ZClasses, and making money off them to keep ZClasses quite alive. So I hope that this defense of ZClasses encourages you not to abandon them. Use whichever tool is best for your application, and trust that others will make the same wise decisions. Flame Shield is now up. I look forward to your replies. Regards Chris

9 10

buildbot failure in Zope branches 2.10 2.4 Windows 2000 zc-bbwin2
by buildbot＠zope.com 28 Sep '06

28 Sep '06

The Buildbot has detected a failed build of Zope branches 2.10 2.4 Windows 2000 zc-bbwin2. Buildbot URL: http://buildbot.zope.org/ Build Reason: changes Build Source Stamp: 7900 Blamelist: andreasjung,baijum,ctheune,dobe,faassen,fdrake,gintautasm,hdima,jim,jukart,mkerrin,opetznick,poster,srichter,wosc BUILD FAILED: failed compile sincerely, -The Buildbot

1 0

buildbot failure in Zope branches 2.10 2.4 Linux zc-buildbot
by buildbot＠zope.com 28 Sep '06

28 Sep '06

The Buildbot has detected a failed build of Zope branches 2.10 2.4 Linux zc-buildbot. Buildbot URL: http://buildbot.zope.org/ Build Reason: changes Build Source Stamp: 7900 Blamelist: andreasjung,baijum,ctheune,dobe,faassen,fdrake,gintautasm,hdima,jim,jukart,mkerrin,opetznick,poster,srichter,wosc BUILD FAILED: failed test sincerely, -The Buildbot

1 0

buildbot failure in Zope trunk 2.4 Windows 2000 zc-bbwin6
by buildbot＠zope.com 28 Sep '06

28 Sep '06

The Buildbot has detected a failed build of Zope trunk 2.4 Windows 2000 zc-bbwin6. Buildbot URL: http://buildbot.zope.org/ Build Reason: changes Build Source Stamp: 7897 Blamelist: andreasjung,baijum,ctheune,dobe,faassen,fdrake,gintautasm,hdima,jim,jinty,jukart,mkerrin,opetznick,poster,regebro,srichter,tseaver,wosc,yuppie BUILD FAILED: failed compile sincerely, -The Buildbot

1 0

buildbot failure in Zope trunk 2.4 Linux zc-buildbot
by buildbot＠zope.com 28 Sep '06

28 Sep '06

The Buildbot has detected a failed build of Zope trunk 2.4 Linux zc-buildbot. Buildbot URL: http://buildbot.zope.org/ Build Reason: changes Build Source Stamp: 7897 Blamelist: andreasjung,baijum,ctheune,dobe,faassen,fdrake,gintautasm,hdima,jim,jinty,jukart,mkerrin,opetznick,poster,regebro,srichter,tseaver,wosc,yuppie BUILD FAILED: failed test sincerely, -The Buildbot

1 0

Zope 2.9.5 coming soon
by Andreas Jung 27 Sep '06

27 Sep '06

FYI: Zope 2.9.5 will be released this weekend. Cheers, Andreas -- ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany Web: www.zopyx.com - Email: info(a)zopyx.com - Phone +49 - 7071 - 793376 E-Publishing, Python, Zope & Plone development, Consulting

1 0

Re: [Zope-DB] KeyError from Shared.DC.DA.DA
by Chris Withers 25 Sep '06

25 Sep '06

This probably merits the Zope Dev list too, since this is part of "core zope"... cheers, Chris FAIRS, Dan, GBM wrote: > Hi, > > Under load, we're seeing KeyErrors from the _cached_result method of the DA > class in the code used to clear out cached results. We're using Python > 2.3.5, Zope 2.8.5, mxODBCZopeDA-1.0.8 running on RHEL3 connecting to a > remote Microsoft SQL Server 2000 database. > > Note this is a different issue to Collector #1745. > > Here's the salient part of the traceback: > > File "/opt/intranet/cbfmnet/zope_home/Products/Membership/Member.py", line > 118, in _getDBValue > values = self._getDBValues(attributes=[attribute]) > File "/opt/intranet/cbfmnet/zope_home/Products/Membership/Member.py", line > 96, in _getDBValues > , userid=login_elements[1] > File "/opt/intranet/cbfmnet/zope/lib/python/Shared/DC/ZRDB/DA.py", line > 453, in __call__ > result=self._cached_result(DB__, (query, self.max_rows_)) > File "/opt/intranet/cbfmnet/zope/lib/python/Shared/DC/ZRDB/DA.py", line > 375, in _cached_result > if int(cache[q][0]) == key: > KeyError: ("execute dbo.EmployeeByNT 'DOMAIN', 'user'\n", 1000, '\nDBConnId: > None') > > The ZSQL Method EmployeeByNT looks like this (it's actually an FSZSQL method > in a skin): > > <dtml-comment> > connection_id: DB_Connection > arguments:domain userid > cache_time: 60 > max_cache: 50000 > </dtml-comment> > execute dbo.EmployeeByNT <dtml-sqlvar domain type="string">, <dtml-sqlvar > userid type="string"> > > We do have other ZSQL methods, but this one is by far the most heavily used. > > > Here's (most of) the offending method, _cached_result (I've snipped some > code which just appends the db connection id to the query. Note also that > Bucket is also defined as lambda: {} at the top of DA.py): > > # Try to fetch from cache > if hasattr(self,'_v_cache'): cache=self._v_cache > else: cache=self._v_cache={}, Bucket() > cache, tcache = cache > max_cache=self.max_cache_ > now=time() > t=now-self.cache_time_ > if len(cache) > max_cache / 2: > keys=tcache.keys() > keys.reverse() > while keys and (len(keys) > max_cache or keys[-1] < t): > key=keys[-1] > q=tcache[key] > del tcache[key] > if int(cache[q][0]) == key: > del cache[q] > del keys[-1] > > if cache.has_key(query): > k, r = cache[query] > if k > t: return r > > # call the pure query > result=apply(DB__.query, pure_query) > if self.cache_time_ > 0: > tcache[int(now)]=query > cache[query]= now, result > > return result > > In this code, 'tcache' is a dict which maps integers representing the time > to queries (the result of calling int(t) where t is the floating-point > result from time.time()) , and 'cache' maps queries to a 2-tuple consisting > of the floating-point timestamp of when the method was called, and the > result of the query. > > I've been trying to figure out a circumstance which could cause the KeyError > we're seeing in production, without success. > > Here are some observations which may give some clues: > > The result of time.time() is converted to an integer and used as a key into > tcache. AFAICS, this means that only 1 entry per second will be stored in > tcache, even though every query will be stored in cache. It also looks like > cache could be the source of a memory leak. > > For example: > - 'SELECT a FROM b' and 'SELECT c FROM b' run in the same 1-second window > - 'SELECT c FROM b' will end up stored in tcache. > - Both queries end up in cache. > - 'SELECT a FROM b' will only ever be removed from cache if 'SELECT a FROM > b' is executed once more later that second - and then 'SELECT c FROM b' will > remain. > > We only see the KeyError after 'some time' in the application. Presumably > this is the time it takes for the caches to fill and start actively removing > entries as per the while loop. (We restarted the cluster last night, and > have yet to see the errors return). > > The SQL query we are running maps an NT credential (domain and userid) to an > internal user id from our corporate database. To give an idea of the usage > pattern of this ZSQL method, it gets run at least once per request, with > application serving approximately 150,000 - 180,000 requests per day, with > 12,000 - 14,000 distinct users (and hence different arguments to the ZSQL > method). > > I'm going to put some logging code around the line that causes the KeyError > next time we do a release. However - has anyone seen this before? Can anyone > see what might be going wrong here? > > Cheers, > Dan -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk

1 0

buildbot failure in Zope branches 2.10 2.4 Linux zc-buildbot
by buildbot＠zope.com 25 Sep '06

25 Sep '06

The Buildbot has detected a failed build of Zope branches 2.10 2.4 Linux zc-buildbot. Buildbot URL: http://buildbot.zope.org/ Build Reason: changes Build Source Stamp: 7856 Blamelist: andreasjung,wosc,yuppie BUILD FAILED: failed test sincerely, -The Buildbot

2 1

buildbot failure in Zope branches 2.10 2.4 Windows 2000 zc-bbwin2
by buildbot＠zope.com 25 Sep '06

25 Sep '06

The Buildbot has detected a failed build of Zope branches 2.10 2.4 Windows 2000 zc-bbwin2. Buildbot URL: http://buildbot.zope.org/ Build Reason: changes Build Source Stamp: 7856 Blamelist: andreasjung,wosc,yuppie BUILD FAILED: failed compile sincerely, -The Buildbot

1 0

Apologies for slowing down svn
by Dario Lopez-Kästen 25 Sep '06

25 Sep '06

Hello, uh... I just realised that I have messed up. I was going to co the Zope2 trunk but after wondering why it never finished, I discovered that I somehow managed to co *all* of zope2, including all the branches, etc. :-P I am not doing that any more. I apologise for any incovenience this may have caused during the past 20 minutes. I am not sure it has but, just in case... /dario -- -- ------------------------------------------------------------------- Dario Lopez-Kästen, IT Systems & Services Chalmers University of Tech. Lyrics applied to programming & application design: "emancipate yourself from mental slavery" - redemption song, b. marley

1 0