[Zope-dev] memory leak

[Marco Nova]

This message is in MIME format. Since your mail reader does not understand
this format, some or all of this message may not be legible.

------_=_NextPart_000_01C0C32E.1DA0FA98
Content-Type: text/plain;
	charset="iso-8859-1"

I hope I hurt nobody attaching a 3 Kbyte zip file.

> -----Original Message-----
> From: Karl Anderson [mailto:karl@digicool.com]
> 
> Dyon Balding <dyon@devcoder.com.au> writes:
> 
> > On Tue, Apr 10, 2001 at 05:27:28PM +0200, Marco Nova wrote:
> > > Hello guys,
> > > 
> > > It seems that I've some memory leak problems and I want 
> to understand where
> > > I've to look for.
> > > 
> > > Scenario:
> > > 
> > > I've an IIS web server that generates an XML page (taking 
> data from
> > > SqlServer);
> > > The Zope web server must retrieve the XML page, manage 
> the data and render
> > > them in HTML (I cant only apply a XSL stylesheet to XML).
> > > 
> > > The Facts:
> > > 
> > > I've written a external method that - using httplib - 
> reads the XML page and
> > > return a string that is the raw XML sources, than I've 
> written another
> > > external method that use ExpatBuilder of 
> Products.ParsedXML.DOM to parse the
> > > XML an generate a list of [(id - {mapping})] which will 
> be accessed with a
> > > dtml-in tag, so in the sequence-key I've access to the ID and with
> > > sequence-item I can access to the mapping object.
> 
> Are you parsing the XML and creating this list from the DOM objects,
> and then throwing them away?  Or are you parsing the XML and creating
> this list from your pyexpat interface code, and never accessing the
> DOM objects?

I'm not sure, I think that the ExpatBuilder creates the DOM tree which I use
for gather information for my list (look at tarkresults.py in the attacched
zip file).

> In either case, it seems like you shouldn't be using ParsedXML's
> ExpatBuilder, which is an interface to PyExpat which is made to create
> ParsedXML's DOM objects.  You should be using your own PyExpat
> interface to parse this information from your XML input, without the
> overhead of generating a DOM tree that you're just going to throw
> away.  The interface is documented in the XML-SIG topic area on
> python.org, and our ExpatBuilder should be a good example on how to
> interface with our PyExpat installation - just don't build the DOM if
> you don't use it.

Ok, I will try, thanks.Anyway if my code is correct we have some bugs around
Zope or ParsedXML that could be of help for someother.

> Regardless of what you *should* be doing :), or if I'm misreading you,
> if you aren't keeping references to the DOM objects being created,
> then they should be reclaimed by the GC, and you do seem to have DOM
> objects lying around.  I'd like to see your code, is it short enough
> to post?  Or you can chuck it into the collector or ParsedXML tracker.
> 
> For debugging purporses, I'd suggest simplifying by supplying an XML
> string directly, rather than from an external server.

Attached you will find a zip file containing:

- res.xml, an xml file generated by my web server, you should import it as a
dtml document named res;
- tarkresults.py, an external method (id:process_results,
modulename:tarkresults, functionname:process_results);
- check_qs.dtml, a dtml method (id:check_qs);
- grid_view.dtml, a dtml method (id:grid_view);
- grid_page.dtml, a dtml method (id:grid_page);

then accessing to grid_view, you should be able to reproduce the behaviour.


Best Regards,

- mn


------_=_NextPart_000_01C0C32E.1DA0FA98
Content-Type: application/octet-stream;
	name="tarkresults.zip"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename="tarkresults.zip"

UEsDBBQAAAAIAEhGiyoGPc5wVQMAAPcLAAAOAAAAdGFya3Jlc3VsdHMucHmdVlFr2zAQfg/kP4jm
wQ4NptvYS1kGGzVj0CalK6NQilEsOdXm2EaSQ7ux/76TLMly4rhpH+LIku7uu9N9n5zxcoP+XC6/
IbapSi4RDGcovrlZ3oxHmVq85iWpUymia8wFJXdXl9HF8spuj58qLL/WLCeUj0fjEaEZ4lRyRrc0
KUpCE0mfZFiufk3PxyOEspIjNY1YgWAySh/BdAETQi8jxDK9HqnH7XNF0XyuN97Gd7fJYnkRnyv/
NS/abT9xXlNlbRaCQCGZKCgVL1MqRMKpqHMJf2nJSaiMZogRBWmC0OaZsFSiOfr7z3u/DxgJHmCW
ET2rkIN9YtHr8B34E4Pf7urk4Cbjy/gqXphcjFHXTOL1Am90kDAoYBTMUAAhUs4qycpCvRalpCKY
tg4c7F03KoWeE2lzmTofND8IQ5Q1TzWQtKyeOVs/SvWy5mVdqYGQWNYiQMcgOrWVdQtrKr9IgLiq
JQ3V6kuYoKBBVa9ylmJdkZ6o/vor4rX2lLCXbc2mXgceAIIlHXKj149KmmT9yWL5OBSg5rnzP6EF
gYbTb3rstzbM6gXDpcZ/M7lPKBEKmmczIGhz7jtM6pJ9nyx9XN/jCMKA0K/DfB40PG4LwQgylodP
1lSKEVWlfl1AVhgsJ7xKuUINFKmRv37fFea0kIkvPTvK0ys8XcX0nBynnH2K4wTnlWIzQ3sdPXXu
HPhjpMfKjtfs+1h8ekNoS7TZq6RoANxp8L0p88G+GTRWz+CtCXpM3iXy1jH5MIsHyD+MxfVzL/G9
K7Tb0UwkgNxcmzBKVJ805X0rha0XUwET9p0H4ayfT1ZynjZ5InjaoIClJGdCMen+QU1UXMDY/ziJ
/JdQ14OUqZYCEVXq0+YH1K5Yh9ZzUzOTo6kAWETwqzfAwjin6g8aTmL+21EBNvfs2jnHLeVC3xro
02d08j76eILgJF62Wyn4ygp9OHO9A19swJZukYAHZ4CMFVucg5zZeG1PmiI31aK5sAdxhLMNBn5C
NZw3z5cVrLU6EatafXn1qZcpcmsMQS0Ej81N07bu3fZ9p13Hzq29P3yFMJeI8zUkCEiTxkj3gauk
zaG5TnzbplkjXFVAwhCFjMycx6m/WZG0oatjr5/9YVKv2y0epWzk8eg/UEsDBBQAAAAIAJ1RjCom
ia5p5AIAAIkaAAAHAAAAcmVzLnhtbO2ZW2+bMBTH3yv1O1h+aKRpDZfceiFU3aRJk6a9rI95cbCT
eCGY2oet+fY7mCRtNwZobNOywgPg4/+xfU5+2MQENw+bmHwR2kiVTKnXdykRSaS4TJZTmsHi/ILe
hKcnATC9ftT5/REl80zGfEoHLkUBwSMw9/F9JvQ2DJzD7a4qjU1RZq8C51DYVWphshgMiVSWALZ4
SclaJth2ypYib73Q7Q/UR0pzIlHhjfe9P1MkbCPCWy9w7E2JgAsTaZkChoOjfVoqa02BMCgrriWC
NJvHMmK5vx2VT8N3Hz5hoI/2MjfB5cFlRMOF1AYCZ2et6YczEKE/cVzP8S4vL571ZetK3I3KdCSK
vNHwfQJCJyzGH8vayzwilW61XK7gR6dDVZnfUqsstT5DGtqCHzj2WjouYJAZK8ef86PSGzsqay1N
A1+QTMdT6uRYYrTMwXTbPGA6MCneuI8azOjVjOloJb+IGQpmhWDmT2aFAJPGF991EDgFXdXUjSqo
84+IugsaGgwr4f8bduhzFsN1lliT4GdLuP6rBI7qCBy1I3BYQeDgiAjEOMQmhW0H4G8GcFgH4LAV
gJMK/oYdfy+ev0kNfpN2059fgd+ow+/F4+f5ddOf3w5ArwLAcTMA71YSQzYEVoLk/3TIVwmrXSnF
k4wg06IhpvCksdxC1OKx4d7tuNdxfJQce3Uce+04dis4njTj+Mk9QcKgIbF7/Bm5s04dnkeIp1uH
p9sKz4pl/k3DWbYhjZ8zA4RxLjiZs2iNl4VWG5Lj0qF5jGjWvQC0XP8HPyczarjvaCdAuz7HSq1N
vr26JakWANvXRAJZMXN1enJOvP2rADGgNJIpE7u07+K6zjUDnHkfwOwU/dOThtjjeSWTJQFFDNsS
NlcZEDgMTDxEIs0NDEGAniF5XJz03nq9fvdYHONj4Q3qZuxBywejYm+UHdOO/C8R6I0cd/hvEzih
IdOC/QnoaNiIm9xgv/jkX5YsqOE3UEsDBBQAAgAIABpTjCpl3gp1bAAAABYBAAANAAAAY2hlY2tf
cXMuZHRtbLNJKcnN0S3Ny0ktLlZQCnINDHUNDtHLSCyOz06t1FAvLFbXVLLj5VIAAhuw0uTEnByE
wuLUErAiHUOwMht9JOOAfKAIWCAzTUEpXi8zr0QDqjEapClW08aQdMMz05AMxulwYtwNNFldHau7
AVBLAwQUAAIACACiVIwqQOnNGvkAAADDAQAADgAAAGdyaWRfdmlldy5kdG1sZVHLTsMwELxHyj9Y
vqRFghS4xv0KbghZxl6aqK5tdt1Q8fWsQxIi4ZNndvY127l88fejQUHZBGfQ6Z4Z3YNxgMe6qqtu
kljjvbA92LP+pOPCesh1JfhdzE1ng2eNQEo+HuQvbaNn+DwjjF+MnmaUzAk0Dd+gZJHdlegcmioR
4AioZBPiaJJttrF3Q5zWtA7G9oWJJXjj0Qmtkvq14Umat6UzWO0HykomjBaIypxXn2m3pIj9Rmrj
NbBWP3gIuyV3L9e1h7BW5M1TGsJJTJusO4mIqTdByYMszmJWk2ulQbdafsLB6ZJSCrdz5b8/m7s5
wP8bfcSY+UY/UEsDBBQAAgAIAJhUjCqFkqsSvwEAABkEAAAOAAAAZ3JpZF9wYWdlLmR0bWyNU8tu
2zAQPDtA/2FDIH4gUGUjyKUQlVOuAfo6FYFBiSuLKEUp5Fop+vWlKOth1wWq02p3ljM75CaSKh2p
Ahy+HdHkGDkSltIPN0mOhtB2EYlMI2S1lWg527KQC5X41O5j8F9CElqh1cFwRnXDhvRtFIGqxAFB
ChJQN2ggisamcHyJ6lASZ7vt9o7Bu5JUDj9nzH2LPUWLjrJn7PUO/JWSUuMJFXoElBYLziSSUHrf
Knx/enM8CSO0wgLb/1iNNigj8dfq9X7H0uUZ6Mvz5+/PX7956OqVpWzOoKoDtK4ROXL2yKCcwmGC
HZvPycDZnLOSqPkUxxOHv4Gfe4e29fZfZDPhMI2Dlx+Fa56INz4MDi4LTuWxyoyfbqnkTLOSXmcS
i7nWzEKuUXhNGgs6GyMmOfgcj0ZPlg8XPXfdlxYhX9SGwKnfGGYtuvmFVULP5jCiQq+mQ3Ztc7p/
X+uQX/wPQ3PMtMoFqdr494YpRHC1+NIpgfVUQ6nG/GaU2Au67kocrJ/+Okwy7BRbr68+qQ3cwQNw
DtsN1BbmGDSye1bdavUbFs46Go3OwRx1uYM95GInk7+2+9Q5yk7i0d/z1j9QSwECFgsUAAAACABI
RosqBj3OcFUDAAD3CwAADgAAAAAAAAABACAAgIEAAAAAdGFya3Jlc3VsdHMucHlQSwECFgsUAAAA
CACdUYwqJomuaeQCAACJGgAABwAAAAAAAAABACAAgIGBAwAAcmVzLnhtbFBLAQIWCxQAAgAIABpT
jCpl3gp1bAAAABYBAAANAAAAAAAAAAEAIACAgYoGAABjaGVja19xcy5kdG1sUEsBAhYLFAACAAgA
olSMKkDpzRr5AAAAwwEAAA4AAAAAAAAAAQAgAICBIQcAAGdyaWRfdmlldy5kdG1sUEsBAhYLFAAC
AAgAmFSMKoWSqxK/AQAAGQQAAA4AAAAAAAAAAQAgAICBRggAAGdyaWRfcGFnZS5kdG1sUEsFBgAA
AAAFAAUAJAEAADEKAAAAAA==

------_=_NextPart_000_01C0C32E.1DA0FA98--