hi all, i still can't make out how zodb works. i see that it increases it's sizes after a while. if i pack, it shrinks back. what is really happening? i read in previous threads, all unreferenced objects are kept until packed. will cache garbage collection do this? will frequent garbage collection (30 sec) help? the site i'm maintaining has a phenomenal zodb size increase. when i pack, it goes down from 550M to 20M. this is in one day. i wonder how zope.org tackles this? how can i do this automatically? read zope admin and content manager guide, but there's only brief rundowns. any pointers, docs, howtos apreciated. tq -- -- ------------------------------------------------------ http://www.kedai.com.my/kk Am I Evil?
----- Original Message ----- From: "-" <kedai@kedai.com.my> To: <zope@zope.org> Sent: Monday, February 14, 2000 10:13 PM Subject: [Zope] zodb Q
i still can't make out how zodb works. i see that it increases it's sizes after a while. if i pack, it shrinks back. what is really happening? i read in previous threads, all unreferenced objects are kept until packed. will cache garbage collection do this? will frequent garbage collection (30 sec) help?
The FileSystem Storage basically just keeps appending to the Data.fs file whenever an object is changed. This is why doing a hit counter using a folder property or something similar is not good, because the file grows with each hit. I have found that CatalogAware objects seem to add up quickly. If you're using CatalogAware ZClasses and changing them/reindexing them a lot, this might explain your file growing so quickly. Without knowing your application, it's hard to tell what can be changed to avoid the frequent file bloat... Kevin
On Tue, 15 Feb 2000, Kevin Dangoor wrote:
----- Original Message ----- From: "-" <kedai@kedai.com.my> To: <zope@zope.org> Sent: Monday, February 14, 2000 10:13 PM Subject: [Zope] zodb Q
The FileSystem Storage basically just keeps appending to the Data.fs file whenever an object is changed. This is why doing a hit counter using a folder property or something similar is not good, because the file grows with each hit.
I have found that CatalogAware objects seem to add up quickly. If you're using CatalogAware ZClasses and changing them/reindexing them a lot, this might explain your file growing so quickly.
Without knowing your application, it's hard to tell what can be changed to avoid the frequent file bloat...
Kevin
your guess on catalog aware is right on the dot. the site (www.emedia.com.my) is mainly a news site. articles are stored via a pubZclass and is catalog aware. i've been updating the catalog hourly to stop numerical error when doing searches. will updating the catalog help reduce the size? i see that it does not. it may have caused the increase. if i stop updating the catalog, users will see the random numerical key error. how to sync the catalog only when necessary, instead of reindexing. how does zope.org tackle this. i assume, zope.org does have a catalog aware classes somewhere. thanks -- ------------------------------------------------------ http://www.kedai.com.my/kk Am I Evil?
----- Original Message ----- From: "-" <kedai@kedai.com.my> To: "Kevin Dangoor" <kid@kendermedia.com>; <bak@nstp.com.my>; <zope@zope.org> Sent: Monday, February 14, 2000 10:44 PM Subject: Re: [Zope] zodb Q
your guess on catalog aware is right on the dot. the site (www.emedia.com.my) is mainly a news site. articles are stored via a pubZclass and is catalog aware. i've been updating the catalog hourly to stop numerical error when doing searches.
I haven't had this problem with my Catalog. I have about 1,700 CatalogAware objects and searching them seems to work just fine. They don't change very frequently though.
will updating the catalog help reduce the size? i see that it does not. it may have caused the increase. if i stop updating the catalog, users will see the random numerical key error. how to sync the catalog only when necessary, instead of reindexing.
As long as you call reindex_object after you make a change to any of your CatalogAware objects, the catalog shouldn't really get out of sync. It's been working fine for me... Kevin
On Wed, 16 Feb 2000, Kevin Dangoor wrote:
I haven't had this problem with my Catalog. I have about 1,700 CatalogAware objects and searching them seems to work just fine. They don't change very frequently though.
hmm.. i got only about 500 catalog aware objects now, but kept seeing the effect of catalog not in sync. the article class does change every morning. but the thing that worries me most is the phenomenal zodb size increase. it almost reached 2G early this week. and i'm running on linux and i'm not too keen to find out what will happen if zodb reaches 2G. can any of you zope gurus point me to docs, or answers? i can't keep packing every day. maybe it's my ZClass, or dtml, ..i'm not sure, and i'm very worried. or maybe it's time to consider DC support.
As long as you call reindex_object after you make a change to any of your CatalogAware objects, the catalog shouldn't really get out of sync. It's been working fine for me...
the call to reindex_object is in place, after every new instance of the article class.
Kevin
thks -- ------------------------------------------------------ http://www.kedai.com.my/kk Am I Evil?
participants (2)
-
- -
Kevin Dangoor