[Zope-Dev] Zope 2.1.6 Memory issue

[Jason Spisak]

Shane Hathaway writes:

Well, I reduced the 'target size' from 20,000 object down to 1000, and the
memory seems to be climbing.


> Jason Spisak wrote:
> > >
> > > After minimizing your in-memory cache, look at the manage_debug screen
> > > again.  Are the refcounts still high?  I bet they are...
> > >
> > 
> > No.  The ref counts drop to hundreds.
> 
> That's a big relief.  Okay, now we know that this isn't a memory leak,
> it's only related to the fact that your site requires ZODB to load a
> very large number of objects into memory in order to perform normal
> operations.
> 
> Here's something to think about: whenever an object is added to or
> removed from a folder, all objects in all neighboring folders and below
> are traversed.  The manage_beforeDelete and manage_afterAdd hooks are
> called on each object.

I am hoping to use ZPatterns to stem this rapidly out of control,
ObjectManger not scalable, issue.

> 
> Also, there is still the issue that Python doesn't seem to be releasing
> memory back to the system.  I wonder whether that capability has been
> implemented in Python.
> 

It would really put a damper on things if is wasn't.  However, when I
stopped my Zope process each time, it didn't seem to give me back what 
Ihad started with.  Maybe there is a leak in the kernel I am using.  It's
2.2.12-smp.  Anyone?

> > Also, is see reference to 161 Products.GadflyDA.gadfly.kjparser.ParseRule,
> > and I'm not even useing Gadfly!
> > 
> > What the heck is that about?  Anyone?
> 
> Chances are some other product is "reaching in" to Gadfly.  Do you have
> the Zope tutorial installed?  It uses Gadfly.
> 

No.  This is a Data.fs that came from Zope 1.  I didn't even think to get
Gadfly out of the Products directory.

> > 
> > Thing's seem to be a bit nicer now that I took out an External Method that
> > returns a DateTime object the Catalog for Indexing.  However, there's only
> > about 14,000 of objects with that property in existsnce in my DB, and they
> > don't change often.
> > 
> > Here's the code:
> > 
> > def calendar_day(self)
> >     return int(self.calendar_date)/1000000
> > 
> > Used to return the unique integer day.
> 
> That code actually doesn't return a DateTime instance, just an integer.
> 

Yeah, I didn't really think is was that. ;)

> Keep me informed.

You bet.  And a thousand thanks are waiting in my cache for you.

Jason Spisak
CIO
HireTechs.com
6151 West Century Boulevard
Suite 900
Los Angeles, CA 90045
P. 310.665.3444
F. 310.665.3544

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.