December 2001 - Zope-Dev

Re: [Zope-dev] RAMCacheManager and gzip
by Martijn Jacobs 12 Dec '01

12 Dec '01

> There is zlib support in Python, and it is used in Medusa already. ZServer > does not call it, but the compression stuff is there. > > There is one good reason not to use zlib though, Micro$oft Internet > Distorter does not support content-encoding: gzip and for content-encoding: > compress it expects data to be compressed using the Unix compress tool... (I > think someone reported this already... oh well) Hmm, how does mod_gzip compress it then? In every request I get from IE includes the "accept-encoding: gzip" string, and apache is sending it gzip-compressed back. Do you mean that the mod_gzip is compressing it different then zlib? mart(ijn)

3 3

Re: [Zope-dev] recipe for trapping SIGSEGV and SIGILL signals on solaris
by Joseph Wayne Norton 12 Dec '01

12 Dec '01

At Tue, 11 Dec 2001 10:42:46 -0500, Matthew T. Kromer wrote: > > > >#0 0xef5b9810 in _lwp_sema_wait () > >(gdb) info threads > > 19 Thread 10 0xef5b9810 in _lwp_sema_wait () > > 18 Thread 9 0xef5b9810 in _lwp_sema_wait () > > 17 Thread 8 0xef5b9810 in _lwp_sema_wait () > > 16 Thread 7 (LWP 8) subtract_refs (containers=0x13dec8) > > at ./Modules/gcmodule.c:166 > > > > Aha! See? > Matthew - I performed the operations that you recommended and here are the results (see below). The problem seems to be with the value of the tp_traverse field. I am not aware of any "T" type python object. I'm wondering if this is an extension class type (just a guess). I searched through all of the *.c files in zope, etc. but I as not able to find any type of name "T". I also ran across a bug posting at sourceforge ... http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id… This bug report looks very similiar. - j #0 0xef5b9810 in _lwp_sema_wait () (gdb) info threads 19 Thread 10 0xef5b9810 in _lwp_sema_wait () 18 Thread 9 0xef5b9810 in _lwp_sema_wait () 17 Thread 8 0xef5b9810 in _lwp_sema_wait () 16 Thread 7 (LWP 8) subtract_refs (containers=0x13dec8) at ./Modules/gcmodule.c:166 15 Thread 6 0xef647cac in _swtch () 14 Thread 5 0xef5b9810 in _lwp_sema_wait () 13 Thread 4 (LWP 0) 0xef647b7c in _swtch () 12 Thread 3 0xef647cac in _swtch () 11 Thread 2 (LWP 2) 0xef5b9958 in _signotifywait () 10 Thread 1 (LWP 6) 0xef5b7488 in _poll () 9 LWP 9 0xef5b6a24 in door_restart () 8 LWP 8 subtract_refs (containers=0x13dec8) at ./Modules/gcmodule.c:166 7 LWP 7 0xef5b9810 in _lwp_sema_wait () 6 LWP 6 0xef5b7488 in _poll () 5 LWP 5 0xef5b9814 in _lwp_sema_wait () 4 LWP 4 0xef5b9810 in _lwp_sema_wait () 3 LWP 3 0xef5b9810 in _lwp_sema_wait () 2 LWP 2 0xef5b9958 in _signotifywait () * 1 LWP 1 0xef5b9810 in _lwp_sema_wait () (gdb) thread 16 [Switching to Thread 7 (LWP 8)] #0 subtract_refs (containers=0x13dec8) at ./Modules/gcmodule.c:166 ./Modules/gcmodule.c:166: No such file or directory. (gdb) print *((PyObject *) gc)->ob_type $1 = {ob_refcnt = 18213696, ob_type = 0x2d70b0, ob_size = 0, tp_name = 0x1 "T", tp_basicsize = 1328272, tp_itemsize = 4156348, tp_dealloc = 0x125865c, tp_print = 0x3c1b04, tp_getattr = 0, tp_setattr = 0, tp_compare = 0x29, tp_repr = 0x3adeb0, tp_as_number = 0xf66198, tp_as_sequence = 0xdf3fa0, tp_as_mapping = 0x0, tp_hash = 0x1, tp_call = 0x144490 <PyMethod_Type>, tp_str = 0x3f0a1c, tp_getattro = 0x125865c, tp_setattro = 0x3c1b04, tp_as_buffer = 0x0, tp_flags = 158561192, tp_doc = 0x29 "", tp_traverse = 0x4c4f4144, tp_clear = 0xd908c0, tp_richcompare = 0x1151300, tp_weaklistoffset = 0} (gdb) print *((PyObject *) 0x2d70b0)->ob_type $2 = {ob_refcnt = 2977968, ob_type = 0xff5b80, ob_size = 0, tp_name = 0x1 "T", tp_basicsize = 1328272, tp_itemsize = 4155228, tp_dealloc = 0x125865c, tp_print = 0x3c1b04, tp_getattr = 0, tp_setattr = 0, tp_compare = 0x29, tp_repr = 0, tp_as_number = 0x1212b48, tp_as_sequence = 0xbf8d30, tp_as_mapping = 0xffffffff, tp_hash = 0x1, tp_call = 0x144490 <PyMethod_Type>, tp_str = 0x4ab2cc, tp_getattro = 0x1089d5c, tp_setattro = 0x4ab30c, tp_as_buffer = 0x0, tp_flags = 0, tp_doc = 0x29 "", tp_traverse = 0, tp_clear = 0x122d140, tp_richcompare = 0x11ccd70, tp_weaklistoffset = -1} gdb) x 0x4c4f4144 0x4c4f4144: Cannot access memory at address 0x4c4f4144.

3 2

DA connection pooling thoughts
by Brad Clements 12 Dec '01

12 Dec '01

Sorry if this is FAQ.. I've been reading the petrilli's DA how-to, and thinking about Thunked_TM. For Interbase, I'm thinking of trying a DB connection pool that threads can "checkout". That is, each DB connection is serialized to be used by one thread only, but there can be multiple DB connections in a Zope process. We'd maintain a pool of connections from 1 to n.. and each thread making a DA request checks out a DB connection from the pool (if available) and uses it.. I'm trying to figure out of Zope already does this, that is, has one DB connection per thread if I took out Thunked_TM from the DB itself. Or is it that there are multiple DA's, and the DA uses on internal DB for all the DA's ? If Zope will re-entrantly call a DA and it's corresponding DB without Thunked_TM in- place, then having a connection pool (up to one for each thread) might be worthwhile.. I remember seeing something on the list about per-thread state information being available somewhere.. an oid jar or something? Anyone have pointers? Thanks Brad Clements, bkc(a)murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax netmeeting: ils://ils.murkworks.com AOL-IM: BKClements

2 1

Re: [Zope-dev] recipe for trapping SIGSEGV and SIGILL signals on solaris
by Joseph Wayne Norton 12 Dec '01

12 Dec '01

Matt - Well, your patch seems fine in our testing environment. Unfortunately, we do not see any restarts in the testing environment ... always in production. I had to rebuild our entire software base because we are using other products that use extensions class and they are not included under the main zope installation. It caused a bus error the first time (with only running wo_pcgi.py). As I mentioned in my prior e-mail, I modified the patch slightly to exactly match the struct in Python's object.h. Please review this patch. I will apply the patch in production tomorrow morning, 12/13, (Japan Standard Time or GMT+9) and monitor the system. If zope does not restart during the day, then I think you have fixed the problem. I'm using Zope 2.4.3 and Python 2.1.1 with pymalloc disabled on the solaris platform. thanks and regards, - joe n. p.s. I looked **briefly** at the Zope 2.5 source and this patch will not be compatible since there doesn't seem to be a standard among the different extension classes on whether to include or not include the COUNT_ALLOCS define. The cAccessControl class seems to be the exception. *** ExtensionClass.h.bak Fri Nov 16 10:37:11 2001 --- ExtensionClass.h Wed Dec 12 15:10:03 2001 *************** *** 136,154 **** PySequenceMethods *tp_as_sequence; PyMappingMethods *tp_as_mapping; ! /* More standard operations (at end for binary compatibility) */ hashfunc tp_hash; ternaryfunc tp_call; reprfunc tp_str; getattrofunc tp_getattro; setattrofunc tp_setattro; ! /* Space for future expansion */ ! long tp_xxx3; ! long tp_xxx4; char *tp_doc; /* Documentation string */ #ifdef COUNT_ALLOCS /* these must be last */ int tp_alloc; --- 136,169 ---- PySequenceMethods *tp_as_sequence; PyMappingMethods *tp_as_mapping; ! /* More standard operations (here for binary compatibility) */ hashfunc tp_hash; ternaryfunc tp_call; reprfunc tp_str; getattrofunc tp_getattro; setattrofunc tp_setattro; ! ! /* Functions to access object as input/output buffer */ ! PyBufferProcs *tp_as_buffer; ! ! /* Flags to define presence of optional/expanded features */ ! long tp_flags; char *tp_doc; /* Documentation string */ + /* call function for all accessible objects */ + traverseproc tp_traverse; + + /* delete references to contained objects */ + inquiry tp_clear; + + /* rich comparisons */ + richcmpfunc tp_richcompare; + + /* weak reference enabler */ + long tp_weaklistoffset; + #ifdef COUNT_ALLOCS /* these must be last */ int tp_alloc; *************** *** 302,308 **** { PyExtensionClassCAPI->Export(D,N,&T); } /* Convert a method list to a method chain. */ ! #define METHOD_CHAIN(DEF) { DEF, NULL } /* The following macro checks whether a type is an extension class: */ #define PyExtensionClass_Check(TYPE) \ --- 317,330 ---- { PyExtensionClassCAPI->Export(D,N,&T); } /* Convert a method list to a method chain. */ ! /* MTK -- make it pad the type structure out -- presumes only use is in ! ** type structure initialization ! */ ! #ifdef COUNT_ALLOCS ! #define METHOD_CHAIN(DEF) 0,0,0,0,0,0,0,0,{ DEF, NULL } ! #else ! #define METHOD_CHAIN(DEF) 0,0,0,0,{ DEF, NULL } ! #endif /* The following macro checks whether a type is an extension class: */ #define PyExtensionClass_Check(TYPE) \ *************** *** 336,342 **** #define PURE_MIXIN_CLASS(NAME,DOC,METHODS) \ static PyExtensionClass NAME ## Type = { PyObject_HEAD_INIT(NULL) \ 0, # NAME, sizeof(PyPureMixinObject), 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, \ ! 0, 0, 0, 0, 0, 0, 0, DOC, {METHODS, NULL}, \ EXTENSIONCLASS_BASICNEW_FLAG} /* The following macros provide limited access to extension-class --- 358,364 ---- #define PURE_MIXIN_CLASS(NAME,DOC,METHODS) \ static PyExtensionClass NAME ## Type = { PyObject_HEAD_INIT(NULL) \ 0, # NAME, sizeof(PyPureMixinObject), 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, \ ! 0, 0, 0, 0, 0, 0, 0, DOC, METHOD_CHAIN(METHODS), \ EXTENSIONCLASS_BASICNEW_FLAG} /* The following macros provide limited access to extension-class

2 1

Medusa Asyncore patch
by Matthew T. Kromer 12 Dec '01

12 Dec '01

People having problems with medusa on Linux and Solaris (particularly solaris) can try running this patch: http://www.zope.org/Members/matt/patchMedusa-asyncore.py Run it from your Zope root. It will copy asyncore.py from the python lib and patch it using the "patch" command.

3 2

ZBabel Object incompatibility with Zope 2.5 b2
by Joachim Werner 11 Dec '01

11 Dec '01

Hi! This time I need some specialists to help ... Here is some code from ZBabel Objects. ZBabel objects are folders that hold localized versions of anything you want to internationalize and pass getting attributes and calling objects to one of these localized versions depending on your language choice. In Zope 2.4 this works fine. But with Zope 2.5 b2 there are recursion problems: class ZBabelObject(Folder): '''ZBabelObject class ''' ..... def __getattr__(self, name): '''__getattr__(self, name) --> try to get an attribute from the localized object; return attr''' if name[0] != '_' and not self._v_alreadyLooking: try: self._v_alreadyLooking = 1 trueObject = self._getTrueObject() if hasattr(trueObject, name): return Rewrapper((trueObject.getId(), name)) <= HERE THE REWRAPPER IS CALLED # return getattr(trueObject, name) finally: self._v_alreadyLooking = 0 return Base.__getattr__(self, name) Look at the code above: the __getattr__ code calls the Rewrapper to make sure that the object gets the right context. If you replace the line by the commented-out one, the attributes work, but calling the object will not work. E.g., an image width or height will be passed fine, but calling the image will cause an authorization request even for managers. It seems as though __call__ will look for some security assertions via __getattr__ (?), but can't find them due to the missing wrapper. I have no clue. Here is the Rewrapper code: class Rewrapper(Base): def __init__(self, path): self._path = path def __of__(self, parent): ob = parent for p in self._path: ob = getattr(ob, p) <= HERE WE GET A RECURSION ERROR return ob When the code is run unchanged with 2.5, we get an infinite recursion (well, Zope halts it after a few recursions) at the marked line, which is bad. Does anybody have a clue why that happens NOW, and what has change in that respect from 2.4 to 2.5? Maybe there are alternative implementations? The aim is that the localized object just behaves like a normal image, DTML Method, whatever, but actually maps to different contained object versions. BTW: The full code is available here: http://cvs.iuveno-net.de/CVS/cvsweb.cgi/ZBabel/ZBabelObject.py Joachim

2 3

more than one user: zope 'hangs'
by Stephan Herschel 11 Dec '01

11 Dec '01

Hello, We have got a funny problem in Zope: if user A issues a long-running query user B also has to wait in the meantime - zope just waits for A's query to finish. We are using interbase as our database and GV Interbase Database Adapter 0.9.6 on a linux system. In the README it says 'threadsafety ... is set to 1 - threads may share the module but not connections' Somehow i suspect the problem lies there. Can anyone give me a hint on this problem? Thanks in advance, Stephan

3 2

More signal 11 restarts....
by Daniel Duclos 11 Dec '01

11 Dec '01

I have a zope that is dumping signal 11 every 40 minutes or so. I have tried recompile python 2.1.1 with-threads without-pymalloc, recompile Zope with it, recompile ZPAtterns, recompile and instal MYSQL for Python 0.9.1, upgraded to Zope 2.4.3, all this on a Debian Linux box. Nothing changed... still restarting... Anybody, please, has any ideia on this matter? Please, let me know if there's any relevant info that I forgot to mention abot my case! Thanks in advance!! -- daniel lobato duclos -- daniduc(a)hiper.com.br -- http://www.hiperlogica.com.br -----------------------------------------------------------------------------

14 37

recipe for trapping SIGSEGV and SIGILL signals on solaris
by Joseph Wayne Norton 11 Dec '01

11 Dec '01

Hello. We are facing zope restarts on the solaris 5.6 platform with zope 2.4.3 and python 2.1.1. I put together a script based some information on an old posting to the apache mailing list. The following shell/perl script allows one to get a core file from a dying zope child process and also allow the zope to restart without any side effects. The script .... #!/bin/sh PATH=$PATH:/usr/local/bin export PATH cd /tmp for PID in `ps -u zfs -f -o pid,comm,args | fgrep z2.py | cut -d' ' -f1` do export PID truss -f -l -t\!all -S SIGSEGV,SIGILL -p $PID 2>&1 \ | perl -pe 'system("gcore $ENV{'PID'} && sleep 5 && kill -9 $ENV{'PID'}"), exit($ENV{'PID'}) if /(SIGSEGV|SIGILL)/;' & done Step 1: modify script to match your environment. Step 2: execute script Step 3: wait for core file to be dumped in /tmp. Step 4: analyze with gdb where $PID is the pid of the dumped process #bash gdb /path/to/bin/python /tmp/core.$PID #0 0xef5b9810 in _lwp_sema_wait () (gdb) where #0 0xef5b9810 in _lwp_sema_wait () #1 0xef647ea0 in _park () #2 0xef647b84 in _swtch () #3 0xef6468a4 in cond_wait () #4 0xef6467c8 in _ti_pthread_cond_wait () #5 0x50220 in PyThread_acquire_lock (lock=0xd9d878, waitflag=1) at Python/thread_pthread.h:313 #6 0x51f18 in lock_PyThread_acquire_lock (self=0xda39b8, args=0x0) at ./Modules/threadmodule.c:67 #7 0x35db4 in fast_cfunction (func=0xda39b8, pp_stack=0xed40f828, na=0) at Python/ceval.c:2994 #8 0x33ca0 in eval_code2 (co=0x267848, globals=0x51ec4, locals=0x0, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:1951 : : It seems that we are facing trouble due to the thread library on solaris (unless the truss command has introduced a side-effect). Anyone else facing similiar troubles? .... or maybe I should post this to a python mailing list. - joe

2 1

How makesetup deals with Zope ?
by Pascal Samuzeau 11 Dec '01

11 Dec '01

Hi, Next week I've tried to install the ParsedXML, under my Zope. Unfortunately, the installation failed. After some hours of hard worker, I've only change the permission of the makesetup file with : chmod 644 /usr/local/zope/Zope-2.4.3-linux2-x86/lib/python2.1/config/makesetup So, I have 2 questions about it 1-- Even I thought to see what makesetup is doing for and what it is doing, is there anybody to explain me in more details, for my personal knowledge ? 2-- In fact, the real question of this mail. The makesetup file should'nt installed with the "x" bit on, when we installed Zope ? Sincerily PS -------------------------------------------------- Oreka ! Nous sommes l'internet moins cher ! Surfez 25% moins cher avec http://www.oreka.com

1 0