[Zope-Coders] ParsedXML spinning

[Martijn Faassen]

Hi there,

And this is my followup to Matt; again sorry for the weird followup 
due to my mail lossage.


> I have not been able to produce  a crash; however I was easily able to 
> get Zope to spin to death, regardless of what ZOPE_SECURITY_POLICY is 
> set to.  The __getattr__ of ParsedXML is my primary culprit; I suspect 
> it has some very bad behavior when there is complex structure.  The 
> difference of enabling or not the C access control I think has to do 
> with running out of stack space, but without being able to reproduce 
> this...

It is very unlikely you are seeing the same problem here -- I get no
spinning at all, and fast crashes involving simple DOM access to even
relatively small documents.

> When I instrumented the __getattr__ function in ParsedXML I saw a ton of 
> references to im_self -- this is what the security policy is doing to 
> check to see if it is trying to control access to a method or not.  
> However, I'm not familiar enough with ParsedXML to know what to tweak to 
> try to do things like cache negative references.  When I threw the 
> profiler at it, 50% of the time was spent spinning in that getattr 
> inside ParsedXML.

Which version of ParsedXML did you use? It sounds like you're using
an *old* version of ParsedXML, before I took over maintenance.
This had a *very* inefficient Zope interface layer in DOMProxy.py which
I threw out completely and replaced with a new DOMProxy.py that doesn't
use __getattr__ at all. So if you're talking about this __getattr__,
it's gone, and I've never actually tested a version that had it with
Zope 2.5.

The only place __getattr__ is used now is in DOM/Core.py and it's use
there seems rather innocuous.

Either get the version I released or the latest CVS version:

http://www.zope.org/Members/faassen/ParsedXML

http://cvs.zope.org/Products/ParsedXML/ParsedXML.tar.gz?tarball=1

I thought I'd made this essential difference clear before, but apparently
it wasn't yet.

> > I could use some assistance on this. :) And it seems very likely to me
> > that this is pointing at a real crashbug in the new Zope security
> > module..

> While it is possible that there is some actual bad code in the security 
> routines, I think what is happening instead is just very very bad 
> acquisition behavior which causes a very deeply nested stack and 
> ultimately a crash.  This may or not be due to some nuance of the C code 
> being exactly the same as the python implementation.

These are not the symptoms I'm seeing. While there may be still acquisition
issues left (ParsedXML is over-full with it), please do test with a newer
version of the code..

Regards,

Martijn