- Zope - Zope lists

Good Object Stores to Load ratio?
by Jake 10 Mar '05

10 Mar '05

I have read a lot of Plone/Zope books but never found a any advice on a good ratio when it comes to the Recent Database Activity Chart. My Zope intances (10+ sites Plone/CMF) gets over a 100,000 hits a day and I have it running on two threads each with 60,000 cache items. I run an adbanner product that does a lot of writes (views/clicks) so I know it will never get too low, but after a day or so I see the following in the chart. 2,802 - Object stores Total 1,406 - Object loads Total 6,905 - Connections Total Which gives me about a 2:1 ratio when it comes Stores:Loads which is about as good as I can get it. So, my question is, is that good? bad? Anyone care? What are others seeing? Thoughts? Jake -- http://www.ZopeZone.com

2 2

zserver non responsive
by Darian V Schramm 10 Mar '05

10 Mar '05

Hello List, Zserver stopped responding the other day. The only relevent errors I can find are in the zeo.log, and the event.log zeo.log: 2005-03-07T05:33:53 ERROR(200) zrpc-conn:127.0.0.1:41661 Error caught in asyncore Traceback (most recent call last): File "/home/isarda-s/zope/273/lib/python/ThreadedAsync/LoopCallback.py", line 120, in poll File "/home/isarda-s/Python-2.3.4//lib/python2.3/asyncore.py", line 390, in handle_read_event self.handle_read() File "/home/isarda-s/zope/273/lib/python/ZEO/zrpc/smac.py", line 145, in handle_read File "/home/isarda-s/Python-2.3.4//lib/python2.3/asyncore.py", line 356, in recv raise socket.error, why error: (110, 'Connection timed out') event.log: 2005-03-07T05:01:57 ERROR(200) zrpc-conn:localhost:9999 Error caught in asyncore Traceback (most recent call last): File "/home/isarda-s/Python-2.3.4//lib/python2.3/asyncore.py", line 69, in read obj.handle_read_event() File "/home/isarda-s/Python-2.3.4//lib/python2.3/asyncore.py", line 390, in handle_read_event self.handle_read() File "/home/isarda-s/zope/273/lib/python/ZEO/zrpc/smac.py", line 145, in handle_read File "/home/isarda-s/Python-2.3.4//lib/python2.3/asyncore.py", line 356, in recv raise socket.error, why error: (9, 'Bad file descriptor') Zope runs behind apache using zeo as filestorage. I tried restarting zope and zeo, and still the nothing from zserver. We ended up restarting the machine and it started working again, but I'd like to get to the bottom of what caused the downtime. The machine had been up for 6 months before this happened and everything had been running smoothly prior to this issue. There was an issue with an apache log growing too large (2GB limit) but I don't think this would effect zope. Here's the specs: Zope 2.7.3-0 python 2.3.4 Plone 1.1 (yeah, its old) -- Darian Schramm darian at abstractedge dot com

2 1

parameter passing syntax
by prabuddha ray 10 Mar '05

10 Mar '05

Thanks fo rthe support In my dtml document the DTML-IN tag used to fill the data in my 2nd combobox in my form i'm using a ZSQL method with one param districtname. this value i've to get from the submit of the same form upon selecting from 1st combobox 'districtname'. the dtml-in tag i used goes like this: <dtml-in getDistrictUser (districtname=REQUEST.form.districtname)> <dtml-var username> </dtml-in> error on saving changes: document template parse error, invalid attribute name getDistrictUser(districtname for tag "<dtml-in...." both the parameter name n the first combobox name is "districtname" . i made them similar thinking it wud take the request variable automatically. plz help me out sir.

2 1

Troubleshooting Zope 100% CPU utilization
by Matthew X. Economou 10 Mar '05

10 Mar '05

After running properly for about an hour or two, my Zope instance stops responding to HTTP requests and begins consuming all available CPU time. How should I go about troubleshooting this problem? Specs: Digital AlphaServer 2000 5/250 FreeBSD/alpha 5.3-RELEASE Python 2.3.4 Zope 2.7.4 Plone 2.0.5 I have already looked at the following: 1. I recently imported a number of objects from a Zope instance running on a completely different architecture (i386 to alpha). i386 is big-endian, alpha is little-endian. Could this be a problem when exporting and importing objects? 2. I examined the Zope web access and event log files, but I can find no common event prior to when the Zope instance stops responding. There are no error messages in the Z2.log file, neither are there any errors in the event.log file (in fact, the last line is "Zope Ready to handle requests"). Running Zope in debug mode does not result in additional logging beyond this. 3. I traced the Zope process. There are a number of calls to poll(2) and the SIGPROF signal handler (which resets itself, apparently, as I see follow-up calls to gettimeofday(2) and sigprocmask(2)). These calls repeat. 4. I ran lsof on the Zope process. Oddly enough, there are still a few established TCP connections between Zope and the Apache web server in front of Zope. There are several other TCP connections in the state "no PCB, CANTSENDMORE, CANTRCVMORE". What else should I check? Best wishes, Matthew -- "The challenge of a moral life is to do nothing that requires forgiveness." - Roger Ebert in his review of _The Woodsman_

3 2

ZSQL result passing
by burley＠zonnet.nl 10 Mar '05

10 Mar '05

In a DTML method I'd like to pass the result from an ZSQL method to aPython script. I've tried the following (which doesn't work but explainsmy idea): Having a ZSQL my_zsql_members_call simply defined as: SELECT * FROM members; I'd hoped that in: <dtml-let members=my_zsql_members_call> <dtml-in my_zsql_iterator_call prefix="seq"> <dtml-var my_python_call(item=seq_item,members=members)> </dtml-in> </dtml-let> the my_zsql_members_call would return a *list* of multiple rows (or kind ofsequence object) of members information data to be used thereafter in the <dtml-in> iteration. Obviously, this is not the case. Technically, my problem can simply be restated as follows: how do I retrieve a multiple row ZSQL result and bind it to a single variable so that it can be passed to and iterated upon in a Python script? Thanks. -- _____________________________________________________________________ Versatel ADSL Gratis. De voordelen van gratis internet met de snelheid van ADSL. Zonder abonnementskosten en zonder vast contract. Je betaalt alleen voor de tijd online. Nu zonder aansluitkosten en met gratis modem. Bestel snel op www.versatel.nl.

3 3

Re: Copy from Data.fs to APE
by Gabriel Genellina 10 Mar '05

10 Mar '05

At Thursday 10/3/2005 03:16, markus.luckey(a)sdm.de wrote: >I'm currently trying to copy my data from DATA.FS to APE with no success. >Import does not work, copy doesn't either. >Therefore I search google for it and found your thread. Do you have any >hint for me, helping me copy my files? >But be patient, i'm a newbe :-) Try this, it's my test example. I hope it's clear enough. --- begin testexp.py --- """ This script tries to read a Folder from a FileStorage (data.fs) and write it to a different filesystem-based Ape storage. How to run: create a Folder inside the root (/test) and put something there. The script will create a directory in [ZOPE]/var/test-export which should contain a copy of the test folder. """ import os, sys import Zope from Products.Ape import apelib from apelib.zodb3.storage import ApeStorage from apelib.zope2.mapper import create_fs_mapper from apelib.fs.connection import FSConnection from apelib.zodb3.resource import StaticResource from apelib.zodb3.db import ApeDB from ZODB.FileStorage import FileStorage from ZODB.DB import DB from OFS.Folder import Folder from Acquisition import aq_base path_to_datafs=os.path.join(CLIENT_HOME, 'data.fs') print 'Setting up FileStorage for reading in', path_to_datafs fs_storage = FileStorage(path_to_datafs, create=0) dbfs = DB(fs_storage) print 'Open FileStorage' connfs = dbfs.open() rootfs = connfs.root()['Application'] # Setup export directory path_export=os.path.join(CLIENT_HOME, 'test-export') print 'Verifying export directory in', path_export if os.path.exists(path_export): print 'Deleting previous contents' import shutil shutil.rmtree(path_export,1) # Setup ApeStorage print 'Creating mapper' mapper, conns = create_fs_mapper(path_export) resource = StaticResource(mapper) print 'Creating ApeStorage' ape_storage = ApeStorage(resource, conns, clear_all=1) dbape = ApeDB(ape_storage, None) print 'Open ApeStorage' connape = dbape.open() if not connape.root().has_key('Application'): print 'Creating root object' from OFS.Application import Application connape.root()['Application'] = Application() # may be any container, Folder() is fine print 'Getting root object of ApeStorage' rootape = connape.root()['Application'] # just to verify I can create something inside the export directory print 'Creating a test2 folder inside APE' id = 'test2' obj = Folder() obj.id = obj.title = id rootape._setObject(id, aq_base(obj)) # copy test folder print 'Reading test folder' id = 'test' obj = rootfs._getOb(id) print 'Obtaining a copy of test folder' dup = obj._getCopy(rootape) print 'Writing copy in APE' rootape._setObject(id, dup) print 'Closing all' dbape.close() dbfs.close() print 'Done' --- end testexp.py --- Gabriel Genellina Softlab SRL

1 0

RE: [Zope] Troubleshooting Zope 100% CPU utilization
by Matthew X. Economou 10 Mar '05

10 Mar '05

Paul, Thank you for your suggestions. Even with the trace log enabled, I do not see a pattern, although I am now able to pinpoint when Zope begins spinning. I also did the Google search, which prompted me to attach to the Zope instance with GDB. (I should have thought of that sooner.) While it doesn't list any threads, GDB gives me the following backtrace: (gdb) bt #0 0x00000001601bd320 in _get_curthread () from /usr/lib/libc_r.so.5 #1 0x00000001601c1be4 in _thread_leave_cancellation_point () from /usr/lib/libc_r.so.5 #2 0x00000001601b70e8 in select () from /usr/lib/libc_r.so.5 #3 0x0000000160ca4644 in ldap_int_select () from /usr/local/lib/libldap_r-2.2.so.7 #4 0x0000000160c8c9a4 in wait4msg () from /usr/local/lib/libldap_r-2.2.so.7 #5 0x0000000160c8c440 in ldap_result () from /usr/local/lib/libldap_r-2.2.so.7 #6 0x0000000160c5ce50 in l_ldap_result2 () from /usr/local/lib/python2.3/site-packages/_ldap.so #7 0x00000001200cf9ac in PyCFunction_Call () #8 0x00000001200186f4 in PyObject_Call () #9 0x0000000120080228 in PyEval_GetFuncDesc () #10 0x000000012007c7a8 in PyEval_EvalCode () #11 0x000000012007dcb8 in PyEval_EvalCodeEx () I happen to be using LDAPUserFolders 2.5beta2 and python-ldap 2.0.6. What confuses me is the fact that I am running the same versions of these two modules on the old Zope server (a PC-based server from which I migrated). I may try rebuilding Python with a larger thread stack, but a few more Google searches are in order. As always, any additional suggestions are welcome. Thanks again for all of your help. Best wishes, Matthew -- "The challenge of a moral life is to do nothing that requires forgiveness." - Roger Ebert in his review of _The Woodsman_

1 0

SESSION updates don't always take
by Chris Kratz 10 Mar '05

10 Mar '05

Hello all, We have run into intermittent problems where the updates to items in the SESSION object are not always saved. I've boiled the problem down to the following set of steps. 1. The SESION object stores a list of items, in the test case, it is a list with a single number. 2. We use a function to mutate that list directly by passing in a reference to the list, ie function(SESSION['listName']) 3. The function makes changes to the list directly. It seems that when a single user is using the system, this works consistently about 99% of the time. But, for some reason, when a second test case running on a different machine/browser starts up, almost always we get an immediate inconsistency between what should be in the session and what is actually there in the first test case. Now, it will happen running a solo version of the test case as well, just far more rarely. Note that running two test cases inside of tabs in firefox will immediately become inconsistent because they share the same session object within zope. But running two browsers (ie & firefox) or just using 2 different machines as the clients will trigger the problem. I also found that if I make any other unrelated change to the session object using SESSION.set() it appears to fix the problem. My hunch is that by manipulating the list directly without using session.set, we are causing a situation where sometimes the SESSION variable doesn't get marked as dirty and saved. Are there any other ideas out there why we would be seeing the behavior we are seeing? Thanks, -Chris Zope 2.7.3 on python 2.3.4 Linux kernel 2.6.8 ************************************************************************** WARNING, this test case may stress the server it runs against due to auto-reloading once per second. ************************************************************************** 1. Create DTML document by copying in the contents attached. 2. Create a python script with the contents attached. 3. View the page in two completely different browsers (to trigger separate zope sessions) 4. The pages will run automatically reloading once a second and checking what is in the session to what it thinks should be in the session. I get an inconsistency within 50 runs on at least one of the browsers. Often within 10 runs one of them finds an inconsistency. -------------- testSessionBug (DTML Doc) --------------  <dtml-if numTests><dtml-else> <dtml-call "REQUEST.set('numTests', 0)"> <dtml-call "SESSION.set('numTests', [0])"> </dtml-if> <html><head> </head><body> <h2><dtml-var title_or_id></h2> <p> <dtml-if "SESSION.has_key('numTests') and numTests<>0 and int(SESSION['numTests'][0])<>int(numTests)"> <b>Found Session List inconsistency</b> <br> SESSION=<dtml-var "SESSION['numTests'][0]"><br> Should be=<dtml-var numTests> <dtml-else> <dtml-var numTests> complete without error.  <dtml-call "REQUEST.set('numTests', int(numTests)+1)">  <dtml-call "writeToSession(REQUEST.SESSION['numTests'], numTests)">  <d-call "SESSION.set('test',1)">  <script Language="javascript"> setTimeout('self.location="testSessionBug?numTests=<dtml-var numTests>"', 1000); </script> </dtml-if> <br><br> <a href="testSessionBug">Start Over</a> </p> </body></html> -------------- writeToSession (python script) -------------- ## Script (Python) "writeToSession" ##bind container=container ##bind context=context ##bind namespace= ##bind script=script ##bind subpath=traverse_subpath ##parameters=valList, val ##title= ## valList[0]=val -- Chris Kratz

3 5

Re: Re: [Zope] parameter prob remains
by prabuddha ray 10 Mar '05

10 Mar '05

seems this worked.jus a matter of []. thanks yuan. but I'm totally at loss.no compilation errors, but when i test the districtname request parameter is missing. error display: Error Type KeyError Error Value 'districtname' which is quite obvious. plz tip me how to set the districtname requset var for the 1st time.sud be an ifelse thing plz tell i;m stuck. On Thu, 10 Mar 2005 Hong Yuan wrote : >Try this: > ><div tal:repeat="district >python:context.getDistrictUser(district_name=request.form['districtname'])"> > >-- HONG Yuan >Homemaster Trading Co., Ltd. >No. 601, Bldg. 41, 288 Shuangyang Rd. (N) >Shanghai 200433, P.R.C. >Tel: +86 21 55056553 >Fax: +86 21 55067325 >E-mail: hongyuan(a)homemaster.cn > > >prabuddha ray wrote: > >>Sorry to say but i dint get the clue. >>AJ plz tell me which python dictionay method will do it. >> >> >>On Sat, 05 Mar 2005 Andreas Jung wrote : >> >>> >>>request.form is usually a mapping/dictionary but *not* a >>>record-style object...means you >>>need to access through the Python dictionary methods *wink* >>> >>>-aj >> >> >>> >>>--On Samstag, 5. März 2005 5:50 Uhr +0000 prabuddha ray >>><buddha_2001(a)rediffmail.com> wrote: >>> >>>>thnx 4 the tip. but i got the following error when testing >>>> >>>>Error Type >>>>AttributeError Error Value >>>>'dict' object has no attribute 'districtname' >>>> >>>>I don have anything like 'dict' . >>>> >>>>the stmt was ; >>>><div tal:repeat="district >>>>python:context.getDistrictUser(district_name=request.form.districtname)"> >>>> >>> >>_______________________________________________ >>Zope maillist - Zope(a)zope.org >>http://mail.zope.org/mailman/listinfo/zope >>** No cross posts or HTML encoding! ** >>(Related lists - >>http://mail.zope.org/mailman/listinfo/zope-announce >>http://mail.zope.org/mailman/listinfo/zope-dev ) > >

1 0

dtml-tree: wrong branchesexpanded/collapsed
by Ulla Theiss 10 Mar '05

10 Mar '05

Hello List, I've a problem with dtml-tree. When I click on the first "plus sign" the branches may be expanded, or nothing may be happen or one of the other branches may be expanded. The same behaviour occurs with the attempt to collapse or expand the other branches. Dtml-code: <dtml-tree branches="getBranches"> branch: <dtml-var expr="__str__"> Product-code: class TestBranch: def __init__(self, str= "a"): self.str = str def getBranches(self): if not "x" in self.str: return [TestBranch(self.str + "x")] else: return None def __str__(self): return self.str def getBranches(self): return [TestBranch("a"), TestBranch("b"), TestBranch("c")] Do you have an idea, what I make wrong? Many thanks in advance, Ulla.

2 6