Large jumps of memory use increase, and seeking overall understanding of memory use
All, (I am posting to both the general Zope and Plone lists, I hope that's all right. My questions are more directly Zope issues, but in case someone who only looks only at the Plone lists has a different insight to offer...) I'm the co-admin Alex Kirk was referring to in his recent post "Random Crashes/Freezes on FreeBSD 5.4-RELEASE" on the Zope list. The particular instance he was referring to was not a "random" crash but simply Zope running out of memory (with the standard 512M limit on FreeBSD, that we have not increased so far -- we have 2Gb of physical memory). I have had one truly unexplained unresponsive Zope on that system so far, but was not able to characterize it well enough to ask an intelligent question on the list about that one. I fully understand that Zope has to be restarted regularly and intend to use monit to do so, but I would like to better understand what is going on in any case, and hopefully be able to lower the required frequency of such restarts. I have searched the lists (posts like "[Zope-dev] Very severe memory leak" and "[ plone.user] Memory leaks even with latest-everything minimalistic site?") and get the general idea, but what I have read so far does not directly explain what I am observing. I have a test machine, but for starters, I have done some analysis of memory consumption of our live site to see what normal traffic produces. I have used a script to capture the size of the Zope process every minute (from ps) and merged these results with the access log. What I see does not make sense to me in terms of object cache, or growth of refcounts. My only (entirely unverified) hypothesis is that modules like PIL grab large chunks of memory once in a while. I provide specific examples below. First, the system: Plone 2.5.1, CMF-1.6.2, Zope (Zope 2.9.5-final, python 2.4.3, freebsd5), Five 1.3.7, Python 2.4.3 (#2, Jul 2 2006, 16:14:54) [GCC 3.4.2 [FreeBSD] 20040728], PIL 1.1.5 FreeBSD 5.4 My object cache size was initially at the default of 5000 objects per threads (4 threads default). I reduced this to 2000 just to see, but as I expected, the overall memory use behavior was the same. I have made some minor customizations to Plone and added contributed as well as my own products. My use of CMFReportTool might explain some of what I see, but definitely not all. Overall, my custom memory-use/access-log shows long periods with no increase in memory at all, but there are some big jumps at times (during a one minute tick) by 5, 10, even 25 Mb! As pretty much expected, Zope never gives much memory back to the OS; that part I understand, however I would like to get it to a state where it mostly plateaus and then only slowly creeps up. Some specific examples: With memory use at 417640 Kb, no requests at all in the previous minute, and a single request for an image_view_fullscreen in that whole minute, memory use jumped by 5064 Kb to 422704. The image was definitely not that large! (access log shows size returned was 1137). While on that topic, I realize that the ATImage has to be loaded in the object cache at some point, and that if it's a large image (in the order of mega bytes), I might see the memory use jump by that much. Now assume that the cache was already full when the image was loaded, that a lot of other later requests fill the cache with much smaller objects (so the image is not cached anymore) and then later the image is requested again, my guess is that the cache should already have the space it needs to load it, i.e. should I expect to see another memory jump by the size of the image, and if so, why (where did the space it originally occupy go that it can't be reused)? Can I easily check if large objects are being split in smaller chunks as they should by the way? Other example: A few of the large jumps I noticed have as one of the request during that minute a PDF listing of some documents. I use CMFReportTool to generate the PDF. There seems to be a possible correlation there, however, the resulting PDF document is about 150K, and these memory jumps can be as large as 10 Mb! I'll have a look at my code, CMFReportTool's and ReportLab's, but that seems a bit excessive to produce that document, so I wonder if something else is going on there. Of course, there is a catalog query required to get the documents to list, but I generate the report with the metadata only ( i.e. so nothing should be loaded in the object cache). I expect some increase in memory the first time a module is loaded (like ReportLab, although doesn't that happen when Zope is launched as part of product initialization?), but that would not explain continued increases on further requests. There are also many cases where the same listing was requested with no increase in memory at all (although that may be due to caching -- I'll have to look into that; I use cacheFu but have turned Squid off for now). Other example: There are however cases where I just can't see any reason for the sudden jumps. In one case there was a ~15M increase (from 477996 to 493016) during a one minute tick, with only a few very benign requests (GETs for the sento_form, contact-info form, and a couple of very small ATDocuments. There were no memory increases for the two minutes preceding and following that occurrence. Yet another: There was a 27Mb increase in one minute (the largest during that log), nowhere close to a Zope restart (the memory use was at 374376 prior to the jump). In that case there were a lot of requests in the preceding minute, but they all look benign to me (a few image thumbs, css, js, gifs; looks as if someone did a force reload of their browser). Of course the images had to be loaded to produce the thumbs, but I am sure that they total nowhere close to 27 Mb (they were all already pretty much thumb size to start with, except our "image of the day", which might have been a couple of Mb). Is the PIL process of generating the thumbs that memory hungry? I hope not. I have also noticed a few instances where there are no requests for a few minutes but still some growth of the Zope process (not by much though). I assume that these are due to Zope internal housekeeping? I do have an automated process that loads some content, but it runs at a fixed time in the night and I have not seen large memory jumps while that runs. If someone feels like having a look, for fun ;-), that memory-use/access-log file I have been looking at is a 6 Mb HTML file (I removed some info from the access entries so they are easier to scan through, and show the memory increases/decreases as a bar graph so it's easy to scroll down the file and see big changes). The log spans one Zope restart (I see a couple big and then more little memory jumps soon after Zope has started, which makes sense to me). Other tidbits. I have seen one post stating that large file imports may increase memory use by 2 to 3 times the size of the file. I have looked at some POSTs of large images and see about that. Are these increases cumulative, or should they plateau at some point? Also it wasn't clear in that post if that was supposed to be a WebDav issue only (we are not using WebDav so far). When I flush the cache, the DateTime refcount goes down dramatically, but the top counts (ValidationChain, EmptyValidator, AttributeStorage, StringField) do not go down significantly. Alex and I are volunteers doing this work for a non-profit, so any ideas or "yep, that's normal, live with it" would be greatly appreciated! (If the latter answer, I would still like to better understand where that memory is being used; I still don't understand how it could be the object cache, which is often cited as the main cause of memory creep). Cheers, Jean
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 25 Feb 2007, at 00:05, Jean Lagarde wrote:
I fully understand that Zope has to be restarted regularly
This may be needed in a situation where you have a memory-hungry application (like Plone), but otherwise there is no general need to restart Zope regularly. My websites run months without restart, and they don't balloon in memory footprint.
My object cache size was initially at the default of 5000 objects per threads (4 threads default). I reduced this to 2000 just to see, but as I expected, the overall memory use behavior was the same.
That setting is a little bit misleading. These values represent targets, not hard limits. If a request pulls in a much larger number of objects into memory then that chunk of memory is now held by Python and won't be released, small target value or not.
Overall, my custom memory-use/access-log shows long periods with no increase in memory at all, but there are some big jumps at times (during a one minute tick) by 5, 10, even 25 Mb! As pretty much expected, Zope never gives much memory back to the OS; that part I understand, however I would like to get it to a state where it mostly plateaus and then only slowly creeps up.
This is likely to be impossible, especially with large amounts of content. I don't know your application, but I have two sites I know pretty well where content is public, and web crawlers/spiders become the enemy because they cause large chunks of content to be loaded that is not within the normal "window" of content used by normal site users. That's just one specific example where a quick jump in memory footprint can be observed.
Alex and I are volunteers doing this work for a non-profit, so any ideas or "yep, that's normal, live with it" would be greatly appreciated! (If the latter answer, I would still like to better understand where that memory is being used; I still don't understand how it could be the object cache, which is often cited as the main cause of memory creep).
You would need to study all the inefficiencies inherent in the content types you use, or all the nasty stuff that gets pulled in for viewing the site furniture shown for every page etc. Very few product authors out there put memory efficiency onto their priority list. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFF4W4ZRAx5nvEhZLIRAsFvAKC189MpGOfP+fI5w0ckR7W9ceMIZgCgovW/ 2cMz670YuvmUi7dWQ3GufUY= =kdBX -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Alex and I are volunteers doing this work for a non-profit, so any ideas or "yep, that's normal, live with it" would be greatly appreciated! (If the latter answer, I would still like to better understand where that memory is being used; I still don't understand how it could be the object cache, which is often cited as the main cause of memory creep).
You would need to study all the inefficiencies inherent in the content types you use, or all the nasty stuff that gets pulled in for viewing the site furniture shown for every page etc. Very few product authors out there put memory efficiency onto their priority list.
P.S.: Memory is cheap. Cheap enough even for a non-profit. Go get some, or at least compare the cost of buying meory with the cost of countless hours you already have and may continue to spend on this. I'm not saying it's not a worthwhile task to try and get to the bottom of code inefficiencies, but keep the practical side in mind. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFF4W+WRAx5nvEhZLIRAq5tAKCCB16LWniHD5lZuAO8cLmn7oeMZACbBTAn aH0XuUuNyOY63pvT7j47+cA= =pN7N -----END PGP SIGNATURE-----
On 2/25/07, Jens Vagelpohl <jens@dataflake.org> wrote:
On 25 Feb 2007, at 00:05, Jean Lagarde wrote:
I fully understand that Zope has to be restarted regularly
This may be needed in a situation where you have a memory-hungry application (like Plone), but otherwise there is no general need to restart Zope regularly. My websites run months without restart, and they don't balloon in memory footprint.
Thanks Roché and Jens. Yes, of course I meant this in the context of a public Plone application.
That setting is a little bit misleading. These values represent targets, not hard limits. If a request pulls in a much larger number of objects into memory then that chunk of memory is now held by Python and won't be released, small target value or not.
I also realize that. Although here the part that is not clear to me: Python holds that memory and won't release it to the OS, but can't it reuse it internally for future object loads? If it just keeps piling up objects in memory then even talking of a target is meaningless.
You would need to study all the inefficiencies inherent in the content types you use, or all the nasty stuff that gets pulled in for viewing the site furniture shown for every page etc. Very few product authors out there put memory efficiency onto their priority list.
P.S.: Memory is cheap. Cheap enough even for a non-profit. Go get some, or at least compare the cost of buying meory with the cost of countless hours you already have and may continue to spend on this. I'm not saying it's not a worthwhile task to try and get to the bottom of code inefficiencies, but keep the practical side in mind.
Yep, that's what I keep on telling my co-administrators. As I said we actually have a lot more physical memory available, but I need their buy-in to increase the process limits on FreeBSD (I'm not succeeding by the way -- got an email this morning suggesting that Zope/Plone was a bad idea instead). And yes, I do allow crawlers and they are poking at all kinds of things. However, I was still hoping that someone would address the specific examples I provided to tell me if they are in the ballpark of what is commonly observed. I understand all the generalities, but I want to find out if something is broken in my particular implementation (possibly because of a bad practice on my part, etc.). I realize that looking at requests only does not give the whole picture (although I think it should be the largest source of Zope activity by far, am I wrong?), and that the descriptions I have provided are far from sufficient to do a complete determination, but let's just take the first one: Memory size was at 417640 (Kb) from 03:01:40 to 03:04:41 (for three minutes). During these three minutes, there were requests for our front page and apparently everything that accompanies it in Plone, i.e. the Plone scripts.jss, the style sheets, bunch of icon gifs, image thumbs the logo.jpg, etc. No memory increase during that time. At 03:05:41 I logged a memory increase to 422704. Between 03:04:41 and 03:05:41, there was a single request at 03:05:02 for an image_view_fullscreen. For those unfamiliar with Plone, that is one of the barest page served: Just an image and a link to go back to Plone (none of the Plone portlets, etc.). The access log shows that the number of bytes returned was 1137. There was no GET for the actual image. There were no memory increase whatsoever for at least the next 30 minutes, and there were a number of other requests during that time. Now I'm looking for hypotheses to explain the 5Mb memory increase. Here are some: 1) It's impossible. I must have made a mistake when I created that merged log (i.e. I must be looking at bad data). 2) The 5 Mb necessarily had to be used by the object cache to load one or more objects. Note that the image was 2 Mb at most, and the object itself can't be much larger, so even if it was loaded in the cache then, my question is what else was loaded? Or, does Zope/Python grab more memory for the cache in chunks? (I know that when appending to a Python list for example, the list will grow in chunks, but that's only space for pointers and not the actual objects pointed to.) 3) Part of the 5 Mb was probably used loading into the object cache, but a significant part could have been used elsewhere (but the increase was still due to the request). So my question then is: Where else would previously unallocated megabytes of memory be needed to serve a page? I am using the default CacheFu behavior for images, so I know it wasn't also RAM cached. 4) Could have been a coincidence, unrelated to the request. My question then is: What else could Zope be doing in the background, other than serve requests, that consumes megabytes of new memory. (We don't use Webdav, or ftp; I'm quite positive in any case that no imports were being made. I do use external editor, but that would show up in the HTTP access log). Cheers, Jean
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 25 Feb 2007, at 18:58, Jean Lagarde wrote:
Memory size was at 417640 (Kb) from 03:01:40 to 03:04:41 (for three minutes). During these three minutes, there were requests for our front page and apparently everything that accompanies it in Plone, i.e. the Plone scripts.jss, the style sheets, bunch of icon gifs, image thumbs the logo.jpg, etc. No memory increase during that time. At 03:05:41 I logged a memory increase to 422704. Between 03:04:41 and 03:05:41, there was a single request at 03:05:02 for an image_view_fullscreen. For those unfamiliar with Plone, that is one of the barest page served: Just an image and a link to go back to Plone (none of the Plone portlets, etc.). The access log shows that the number of bytes returned was 1137. There was no GET for the actual image. There were no memory increase whatsoever for at least the next 30 minutes, and there were a number of other requests during that time. Now I'm looking for hypotheses to explain the 5Mb memory increase. Here are some:
Comparing the number of bytes served during a request and how much memory is consumed is a bit naive. You have *no* idea what all needs to be touched internally to serve that one object. The weightier the framework you run on to of Zope, the higher the chance that you're touching a whole lot of more or less related objects to serve the one you're interested in. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFF4dH5RAx5nvEhZLIRAnbpAKCvKjO9tPVK4FX/Uu9pMkkb/OFfnQCgqbHk IqCHidTP/Oa9Ypdqi3EAWSw= =acGh -----END PGP SIGNATURE-----
On 2/25/07, Jens Vagelpohl <jens@dataflake.org> wrote:
Comparing the number of bytes served during a request and how much memory is consumed is a bit naive. You have *no* idea what all needs to be touched internally to serve that one object. The weightier the framework you run on to of Zope, the higher the chance that you're touching a whole lot of more or less related objects to serve the one you're interested in.
1) Some degree of naivete when looking at a problem is sometimes useful 2) I stated the amount served (which has some value as a lower bound) but also granted that the actual image object might have been loaded to serve the page, and offered hypotheses (2) and (3) which could fit your "explanation", so you're making me more naive than I was. Which begs the question if you read an understood what I wrote. 3) As for all the baggage that might be tied to an object in the ZODB, I will admit a lot of naivete there, but Zope's behavior should not be black magic. In the end, behavior has to be deterministic. So to say that one can have *no* idea of all that needs to be touched internally is a bit cynical; if I had sufficient time on my hands I could find out *exactly* what is being touched and where that memory is going. To throw ones hands in the air and say it's unknowable means that there's no way to know if something is broken. I will agree though that it may be very difficult to find out. 4) If by "You have *no* idea" you mean I, specifically, have no idea, but you do, then I partly agree (I know I don't, but I can't tell if you do from your answers so far). That's why I asked the questions. By the way it's not my intent to be hard on you Jens, on the contrary, I thank you very much for having taken the time to look at my questions. Your point is also well taken about just throwing memory at it and restarting before it runs out; I acknowledged that myself in my initial post. I am still the obstinate curious type who like to know though. Cheers, Jean
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 25 Feb 2007, at 20:07, Jean Lagarde wrote:
3) As for all the baggage that might be tied to an object in the ZODB, I will admit a lot of naivete there, but Zope's behavior should not be black magic. In the end, behavior has to be deterministic.
This is a wrong conclusion. This is not Zope behavior. The inefficiencies are in the applications/frameworks on top of Zope.
So to say that one can have *no* idea of all that needs to be touched internally is a bit cynical; if I had sufficient time on my hands I could find out *exactly* what is being touched and where that memory is going. To throw ones hands in the air and say it's unknowable means that there's no way to know if something is broken. I will agree though that it may be very difficult to find out.
I don't think it's cynical, I think it's the only sane thing to do for me. I cannot afford to spend days digging into Plone to find all the problems, I just live with a fact that Plone-based applications tend to be memory hogs, that's all. If you on the other hand have days to spend, good for you.
4) If by "You have *no* idea" you mean I, specifically, have no idea, but you do, then I partly agree (I know I don't, but I can't tell if you do from your answers so far). That's why I asked the questions.
I don't have any specific idea where the memory goes, either. I choose the slimmest framework possible to base my sites on, so I hardly ever have to worry about this type of problem. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFF4eI6RAx5nvEhZLIRAh0WAKC41TGWGVEH2/vU0/o8E81uLohL+QCdG268 /7pDNta40mEQ8Ug+w7G1AaE= =Ns5I -----END PGP SIGNATURE-----
Fair enough about me blaming Zope when Plone is the likely issue (I actually meant the system as a whole, but that's my bad for not thinking and writing clearly). As for the effort required to get to the bottom of these issues, then we are in agreement; I won't take the time to do so either (at least not right now -- too many other prorities). I was hoping that someone had made that effort already and had reached some general conclusions they would share (other than the ones already offered). Thanks again, Jean On 2/25/07, Jens Vagelpohl <jens@dataflake.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 25 Feb 2007, at 20:07, Jean Lagarde wrote:
3) As for all the baggage that might be tied to an object in the ZODB, I will admit a lot of naivete there, but Zope's behavior should not be black magic. In the end, behavior has to be deterministic.
This is a wrong conclusion. This is not Zope behavior. The inefficiencies are in the applications/frameworks on top of Zope.
So to say that one can have *no* idea of all that needs to be touched internally is a bit cynical; if I had sufficient time on my hands I could find out *exactly* what is being touched and where that memory is going. To throw ones hands in the air and say it's unknowable means that there's no way to know if something is broken. I will agree though that it may be very difficult to find out.
I don't think it's cynical, I think it's the only sane thing to do for me. I cannot afford to spend days digging into Plone to find all the problems, I just live with a fact that Plone-based applications tend to be memory hogs, that's all. If you on the other hand have days to spend, good for you.
4) If by "You have *no* idea" you mean I, specifically, have no idea, but you do, then I partly agree (I know I don't, but I can't tell if you do from your answers so far). That's why I asked the questions.
I don't have any specific idea where the memory goes, either. I choose the slimmest framework possible to base my sites on, so I hardly ever have to worry about this type of problem.
jens
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin)
iD8DBQFF4eI6RAx5nvEhZLIRAh0WAKC41TGWGVEH2/vU0/o8E81uLohL+QCdG268 /7pDNta40mEQ8Ug+w7G1AaE= =Ns5I -----END PGP SIGNATURE----- _______________________________________________ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
I've not seen whole thread so sorry if I missed something.
there were requests for our front page and apparently everything that accompanies it in Plone, i.e. the Plone scripts.jss, the style sheets, bunch of icon gifs, image thumbs the logo.jpg, etc. Maybe this is your specific configuration for tests, or something that you are aware of (if so then sorry for telling you something obvious) but in general things like css, js, images should be cached and not served (always) directly by Zope/Plone instance. Properly set up caching is a incredible performance boost. If you have no caching then serving all this stuff from zope may also cause bigger RAM usage as all these objects are read into memory by Zope when they're 'touched'.
-- Maciej Wisniowski
I've not seen whole thread so sorry if I missed something.
there were requests for our front page and apparently everything that accompanies it in Plone, i.e. the Plone scripts.jss, the style sheets, bunch of icon gifs, image thumbs the logo.jpg, etc. Maybe this is your specific configuration for tests, or something that you are aware of (if so then sorry for telling you something obvious) but in general things like css, js, images should be cached and not served (always) directly by Zope/Plone instance. Properly set up caching is a incredible performance boost. If you have no caching then serving all this stuff from zope may also cause bigger RAM usage as all these objects are read into memory by Zope when they're 'touched'.
Wow, this is key, clearly. What are your suggestions for the best (i.e. least resource-intensive) form of caching? Alex
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 25 Feb 2007, at 19:44, Maciej Wisniowski wrote:
I've not seen whole thread so sorry if I missed something.
there were requests for our front page and apparently everything that accompanies it in Plone, i.e. the Plone scripts.jss, the style sheets, bunch of icon gifs, image thumbs the logo.jpg, etc. Maybe this is your specific configuration for tests, or something that you are aware of (if so then sorry for telling you something obvious) but in general things like css, js, images should be cached and not served (always) directly by Zope/Plone instance. Properly set up caching is a incredible performance boost. If you have no caching then serving all this stuff from zope may also cause bigger RAM usage as all these objects are read into memory by Zope when they're 'touched'.
Serving from cache still means they need to be touched at least once to get them into the cache, and (in case of normal caching behavior) they will be touched again once the cached record expires. Caching will not make any difference to memory behavior I'd say. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFF4eKhRAx5nvEhZLIRAhiNAJ9um4o4dGWZcS3vTCi0uHKSOKzY9QCfXLK/ 5Zz9P9siPoh+wAWC5ZOFJb0= =X7PR -----END PGP SIGNATURE-----
On 2/25/07, Jens Vagelpohl <jens@dataflake.org> wrote:
Serving from cache still means they need to be touched at least once to get them into the cache, and (in case of normal caching behavior) they will be touched again once the cached record expires. Caching will not make any difference to memory behavior I'd say.
Yes of course I forgot to mention that the caches also expire in my reply. My experience so far is indeed that the caches save on the CPU usage much more than on the memory. -- Jean
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 25 Feb 2007, at 20:48, Jean Lagarde wrote:
On 2/25/07, Jens Vagelpohl <jens@dataflake.org> wrote:
Serving from cache still means they need to be touched at least once to get them into the cache, and (in case of normal caching behavior) they will be touched again once the cached record expires. Caching will not make any difference to memory behavior I'd say.
Yes of course I forgot to mention that the caches also expire in my reply. My experience so far is indeed that the caches save on the CPU usage much more than on the memory.
Yes, that's true, CPU usage on the Zope side will definitely improve. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFF4gUmRAx5nvEhZLIRAkkeAJ9+ZRCHSRZrmkocnpb5AMbhhPJ/FACeOGNE IrXNzyXUPDORP/MjxOa3Td0= =Qo0B -----END PGP SIGNATURE-----
Serving from cache still means they need to be touched at least once to get them into the cache, and (in case of normal caching behavior) they will be touched again once the cached record expires. Yes, right. I tried to write this with 'always': "not served (always) directly by Zope/Plone instance" :)
Caching will not make any difference to memory behavior I'd say. I'm not expert here, so I might be wrong but AFAIK Zope instance loads 'touched' objects from ZODB into cache. Each ZODB connection has its own cache, so having object touched more than once may cause it is loaded to more than one cache.
The same will possibly be with multiple ZEO clients. External cache will save them from being touched (sometimes even once). ZODB connections cache is cleaned sometimes, so it should flush objects that are old and possibly save some memory then. I don't know whether amount of memory used because of above is huge enough to cause described problem. It's just a though. Feel free to correct me if I'm wrong. -- Maciej Wisniowski
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 25 Feb 2007, at 21:19, Maciej Wisniowski wrote:
Caching will not make any difference to memory behavior I'd say. I'm not expert here, so I might be wrong but AFAIK Zope instance loads 'touched' objects from ZODB into cache. Each ZODB connection has its own cache, so having object touched more than once may cause it is loaded to more than one cache.
The same will possibly be with multiple ZEO clients. External cache will save them from being touched (sometimes even once).
ZODB connections cache is cleaned sometimes, so it should flush objects that are old and possibly save some memory then.
I don't know whether amount of memory used because of above is huge enough to cause described problem. It's just a though.
This is a bit like clutching at straws. You're doctoring the symptoms, hoping that some things may not be loaded into memory. It's not a real solution. Memory-hungry applications will remain memory- hungry, cached or not. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFF4gWoRAx5nvEhZLIRAoGbAJ9U0DMyhO6JA3A8HCAcTkrbpjzq/QCfTxGk wrad3smxlEaFKUXNNacUzio= =Ef9a -----END PGP SIGNATURE-----
This is a bit like clutching at straws. You're doctoring the symptoms, hoping that some things may not be loaded into memory. It's not a real solution. Memory-hungry applications will remain memory-hungry, cached or not. I'm just trying to do something, just like Jean does. You're convincing us that we have to live with Plone's memory appetite and that only reasonable optimization is to buy more memory? Sometimes this is not a solution, but maybe your'e right. I don't know this side of Plone (yet :) ).
One more thing. There is a setting in zope.conf that says about cache size. It is used with ZEO setups. Maybe it is worth looking at how big value it is set to. -- Maciej Wisniowski
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 26 Feb 2007, at 00:41, Maciej Wisniowski wrote:
One more thing. There is a setting in zope.conf that says about cache size. It is used with ZEO setups. Maybe it is worth looking at how big value it is set to.
The ZEO cache is purely disk-based. Its size will not influence the memory footprint. jens -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (Darwin) iD8DBQFF4pBtRAx5nvEhZLIRAtyFAJ92N50VOnLB/qaQ+71I1KcLQXStCgCfTG6T G8IcC+B7dbYSKhzvUtmooRM= =oAIp -----END PGP SIGNATURE-----
Thanks Maciej, Yes, I am using CacheFu (a comprehensive caching product for Plone). I have Squid installed and ready to go (had it in the loop for a little while, but we were having all kinds of server instabilities at some point and I turned that one off to simplify things; we are more stable now so will turn it back on soon). Now I will again admit to some overall experience, with even HTTP, so correct me if I'm wrong: Without squid, I can either cache objects in a Zope memory cache or in browser caches using headers (that's part of what CacheFu helps to configure). However, neither of those prevents Zope from entirely stopping to get requests for cached content. For memory cached content, it will just be served faster and without requiring to access the ZODB, but it will look the same in the access log, and for browser cached content, everything will still have to be loaded every time a browser is opened anew or when a force reload is requested. Once Squid is up, then I do expect to see a lot fewer requests to Zope. Am I missing something major here? Cheers, Jean P.S. Just so you guys don't think that I am a complete newby, I did realize for example that I needed to add some templates by hand to a CacheFu rule, to what was being cached by default, to account for the "Composite Page" Plone product I am using (the templates for the "slots" were not being cached and caused significant performance issues). On 2/25/07, Maciej Wisniowski <maciej.wisniowski@coig.katowice.pl> wrote:
I've not seen whole thread so sorry if I missed something.
there were requests for our front page and apparently everything that accompanies it in Plone, i.e. the Plone scripts.jss, the style sheets, bunch of icon gifs, image thumbs the logo.jpg, etc. Maybe this is your specific configuration for tests, or something that you are aware of (if so then sorry for telling you something obvious) but in general things like css, js, images should be cached and not served (always) directly by Zope/Plone instance. Properly set up caching is a incredible performance boost. If you have no caching then serving all this stuff from zope may also cause bigger RAM usage as all these objects are read into memory by Zope when they're 'touched'.
-- Maciej Wisniowski
Now I will again admit to some overall experience, with even HTTP, so correct me if I'm wrong: Without squid, I can either cache objects in a Zope memory cache or in browser caches using headers (that's part of what CacheFu helps to configure). However, neither of those prevents Zope from entirely stopping to get requests for cached content. For memory cached content, it will just be served faster and without requiring to access the ZODB, but it will look the same in the access log, and for browser cached content, everything will still have to be loaded every time a browser is opened anew or when a force reload is requested.
Once Squid is up, then I do expect to see a lot fewer requests to Zope.
Am I missing something major here? For me, you're absolutely right. I didn't mean to stop Zope from getting any requests. I just wonder if repeadetly touching a big amount of objects (css, js, images) may cause consuming a big amount of memory.
P.S. Just so you guys don't think that I am a complete newby, I did realize for example that I needed to add some templates by hand to a CacheFu rule, to what was being cached by default, to account for the "Composite Page" Plone product I am using (the templates for the "slots" were not being cached and caused significant performance issues). I said sorry if I'm telling something obvious :)
-- Maciej Wisniowski
participants (4)
-
alex＠schnarff.com -
Jean Lagarde -
Jens Vagelpohl -
Maciej Wisniowski