I am starting to hate Zope/Plone development and its bl**dy lack of refresh.
Back in the olden days Zope developments was tiresome. You did a change in your source file, and you needed to restart Zope to see the change. Then a 'refresh' option was made, and products would get automatically refreshed when a change was made. Sweet times. But doing development in Plone has become increasingly tiring since Zope 2.8 and up. Refresh no longer works, and the old "debug, correct, restart" cycle is back. Only this time a software stack the size a skyscrapers has to be loaded. Making it even slower than in the old Zope days. I am developing on a 1.2 GHz Athlon machine with 1 GB of ram. A slow machine by modern standards, I know. But it has been fast enough for everything so far. Just not for modern Plone development! A restart of Plone 2.5 - (Zope 2.8.6-final, python 2.3.5, win32) takes 46 seconds on this machine. For every little change. And it is driving me crazy. If I install zeo I can cut it down to 40 seconds :-s I am not sure what is causing the problems. I am mostly a developer of end user tools. Not a core Zope/Plone developer. And I am not really interested in becoming one either. There must be a niche for developers like me that focus on end users. I love Python because development/testing is so fast. Recompiling code in Java etc. to see a little chance, is a bad idea to me. But The Zope/Plone has developed into exactly this. I know that it technically is two different processes. But for me the end result is the same. I could try running a zeo client from a usb drive to get a faster restart, but it takes 20 minute to copy all the files to it. Does anybody have a clue as to where the problem is? And why aren't there any more people complaining about it? Is it because I develop on Windows and am the only one doing so? What do I have to do to get the sweet sweet refresh working again? Frustrated regards Max -- hilsen/regards Max M, Denmark http://www.mxm.dk/ IT's Mad Science Phone: +45 66 11 84 94 Mobile: +45 29 93 42 96
Max M wrote:
Refresh no longer works, and the old "debug, correct, restart" cycle is back.
Only this time a software stack the size a skyscrapers has to be loaded. Making it even slower than in the old Zope days.
There are a few answers to this question - Get a better machine. :) Probably not your favourite one - Load less. Deleting PloneLanguageTool, for example, speeds up startup since you don't need to load all kinds of translations. - Write tests! The latter is the reason why I don't miss this quite so much. I sometimes miss it, especially when I'm exploring things that I don't know how to write tests for and I need to see it in the browser in order to visualise it, but in general: - Tests are quicker to execute than restarting Zope - You can write many tests and observe many failures, fixing many of them before re-running the tests - Tests are important anyway - you should have them - Your tests probably don't need to load all of Zope all the time. - With the recent layers support that Whit has been working on, you can have even more control over how much gets loaded, when. Refresh of skin templates, of course, works fine when you're in debug mode. I find that skin tweaks is the main thing I need to do TTW, and I find that I tend to do this after I've written the tests that prove that my code works. That saves me a bunch of restarts. But - I'd still like to see a reliable refresh come back. Even if it only works in 80% of the cases, and *even* if it sometimes creates funny SNAFUs that require a restart and lead to mystery errors, it *was* very convenient. Martin -- View this message in context: http://www.nabble.com/I-am-starting-to-hate-Zope-Plone-development-and-its-b... Sent from the Zope - Dev forum at Nabble.com.
On Mon, 2006-04-09 at 12:55 +0200, Max M wrote:
Does anybody have a clue as to where the problem is? And why aren't there any more people complaining about it? Is it because I develop on Windows and am the only one doing so?
What do I have to do to get the sweet sweet refresh working again?
This isn't really a Plone issue, but since it affects so many Plone developers I'll give my 2 cents. I will begin by saying refresh has never been totally reliable for me... even in Zope 2.7. Often times I would refresh and get frustrated by the fact that my code wasn't reloaded and end up restarting anyhow. But the true killer right now is the Zope 3 CA. Basically, if you use zope 3 components, they cannot be refreshed currently. Plone 2.5 uses a ton of Zope 3 views. Now this doesn't excuse why the code *you* write (if it's not using Zope 3 CA) isn't able to be refreshed. For that, I have no answer. So, I've learned to change my habits. I'm sure you've heard this a million times, but there's good reason... it just works. And that is to write unit tests to build your functionality rather than loading it up in Zope. This cuts down on restart times *bigtime*. This is not specific to core plone developers. Everyone who writes python code should be writing unit tests. My first test of the quality of a third-party plone product is to inspect it's test coverage. Anyhow, hope that helps. Regards, Rocky -- Rocky Burt ServerZen Software -- http://www.serverzen.com News About The Server (blog) -- http://www.serverzen.net
--On 4. September 2006 09:43:07 -0230 Rocky Burt <rocky@serverzen.com> wrote:
So, I've learned to change my habits. I'm sure you've heard this a million times, but there's good reason... it just works. And that is to write unit tests to build your functionality rather than loading it up in Zope.
There is not much difference. Restarting a Plone instance with a bag full of bricks is nearly the same as running Plone unittests with the same amount of bricks on your back. -aj -- ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany Web: www.zopyx.com - Email: info@zopyx.com - Phone +49 - 7071 - 793376 E-Publishing, Python, Zope & Plone development, Consulting
On Mon, 2006-04-09 at 14:25 +0200, Andreas Jung wrote:
--On 4. September 2006 09:43:07 -0230 Rocky Burt <rocky@serverzen.com> wrote:
So, I've learned to change my habits. I'm sure you've heard this a million times, but there's good reason... it just works. And that is to write unit tests to build your functionality rather than loading it up in Zope.
There is not much difference. Restarting a Plone instance with a bag full of bricks is nearly the same as running Plone unittests with the same amount of bricks on your back.
Unit tests don't need to load in Plone. More times than not they don't even need to load up ZTC. It's the integration tests that end up loading ZTC and PTC. I write those last :) - Rocky -- Rocky Burt ServerZen Software -- http://www.serverzen.com News About The Server (blog) -- http://www.serverzen.net
--On 4. September 2006 09:59:29 -0230 Rocky Burt <rocky@serverzen.com> wrote:
On Mon, 2006-04-09 at 14:25 +0200, Andreas Jung wrote:
--On 4. September 2006 09:43:07 -0230 Rocky Burt <rocky@serverzen.com> wrote:
So, I've learned to change my habits. I'm sure you've heard this a million times, but there's good reason... it just works. And that is to write unit tests to build your functionality rather than loading it up in Zope.
There is not much difference. Restarting a Plone instance with a bag full of bricks is nearly the same as running Plone unittests with the same amount of bricks on your back.
Unit tests don't need to load in Plone. More times than not they don't even need to load up ZTC. It's the integration tests that end up loading ZTC and PTC. I write those last :)
Usually I don't write any unittests but only integration tests for Plone using PTC. That's what I am talking about. Usually I don't write unittests for AT/Plone boilerplate code...you test it one time and it works. More important are integration tests and they really take more or less the same amount of startup time. -aj
Max M wrote at 2006-9-4 12:55 +0200:
... But doing development in Plone has become increasingly tiring since Zope 2.8 and up.
Refresh no longer works, and the old "debug, correct, restart" cycle is back.
I know that refresh still works in 2.8.1 (I am using it extensively). And I do not understand why it should not work in higher Zope versions. The base mechanism is quite robust -- almost only defeatable with global registries (as those which might be filled with "Five"). There has been a long thread recently (on "zope@zope.org", I think). This thread seems to indicate that old objects remain in the ZODB caches -- although this is very difficult to believe. -- Dieter
Max M wrote:
I am not sure what is causing the problems. I am mostly a developer of end user tools. Not a core Zope/Plone developer. And I am not really interested in becoming one either. There must be a niche for developers like me that focus on end users.
Well ok. I have researched a bit more. All the speedup suggestions gave at maximum 5-10 seconds. Not nearly enough. (Didn't try to buy a new computer though. Will do that next week ;-) I also tried to run Zope as a zeo client from a usb flash disk. As expected the initial compilation to byte code was sloooow. I then expected it to run fast after that. It didn't! *clever* configuration clearly wasn't clever enough for me. So I made a site with a standard Zope tool: class simple_tool(UniqueObject, PropertyManager, SimpleItem.SimpleItem, ActionProviderBase): And I added a simple Plone skin. Now refresh worked like a charm. Both when changing the code in the tool and the page templates in the skin. So the problem only exists for me in Archetype based products. Anytime I do a change in any AT content type I need to restart the site. Can anybody confirm this behavior? -- hilsen/regards Max M, Denmark http://www.mxm.dk/ IT's Mad Science Phone: +45 66 11 84 94 Mobile: +45 29 93 42 96
On Tue, Sep 05, 2006 at 03:39:14PM +0200, Max M wrote: | So the problem only exists for me in Archetype based products. Anytime I | do a change in any AT content type I need to restart the site. | | Can anybody confirm this behavior? That's a known-(at least to me)-issue. When you refresh an Archetypes-based product you need to refresh Archetypes as well. You can do this by marking the product as dependent on Archetypes and then refresh Archetypes (which will refresh it's dependencies, refreshing your product). If you don't know what I'm talking about go research more about the refresh functionality in Zope 2. -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214
Sidnei da Silva wrote:
On Tue, Sep 05, 2006 at 03:39:14PM +0200, Max M wrote: | So the problem only exists for me in Archetype based products. Anytime I | do a change in any AT content type I need to restart the site. | | Can anybody confirm this behavior?
That's a known-(at least to me)-issue.
When you refresh an Archetypes-based product you need to refresh Archetypes as well. You can do this by marking the product as dependent on Archetypes and then refresh Archetypes (which will refresh it's dependencies, refreshing your product).
I have tried that. It doesn't work. -- hilsen/regards Max M, Denmark http://www.mxm.dk/ IT's Mad Science Phone: +45 66 11 84 94 Mobile: +45 29 93 42 96
On Tue, Sep 05, 2006 at 04:34:14PM +0200, Max M wrote: | I have tried that. It doesn't work. Maybe you didn't try hard enough. This has always worked for me. -- Sidnei da Silva Enfold Systems http://enfoldsystems.com Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214
Sidnei da Silva wrote at 2006-9-5 10:46 -0300:
... That's a known-(at least to me)-issue.
When you refresh an Archetypes-based product you need to refresh Archetypes as well. You can do this by marking the product as dependent on Archetypes and then refresh Archetypes (which will refresh it's dependencies, refreshing your product).
If you don't know what I'm talking about go research more about the refresh functionality in Zope 2.
We are refreshing AT contents types without a need to refresh AT itself (in Zope 2.8.1, AT 1.3.x). -- Dieter
Sidnei da Silva wrote: Regarding: "I'm starting to get bored of people complaining about Zope" Being a full time active Zope developer (as in developing with zope) for at least 7 year, having contributed code, libraries, documentation and products, I believe I have earned the right to complain once every second year so! I know that if I have problems developing in Zope, so does many more people. And yes Zope 3 is the future, and will make everything so easy for developers. But in the meantime we don't have to scare people of with a difficult development process. -- hilsen/regards Max M, Denmark http://www.mxm.dk/ IT's Mad Science Phone: +45 66 11 84 94 Mobile: +45 29 93 42 96
Max M wrote:
Sidnei da Silva wrote:
Regarding: "I'm starting to get bored of people complaining about Zope"
Being a full time active Zope developer (as in developing with zope) for at least 7 year, having contributed code, libraries, documentation and products, I believe I have earned the right to complain once every second year so!
Constructive complaints are good, agreed. I complain more than once a year, even!
I know that if I have problems developing in Zope, so does many more people.
And yes Zope 3 is the future, and will make everything so easy for developers. But in the meantime we don't have to scare people of with a difficult development process.
And Zope 3 isn't perfect either, so you can catch us complain about that too. Complaints are good if they lead to an improvement. That could be an improvement in code but also an improvement in practices. Regards, Martijn
On Wed, 06 Sep 2006 01:39:10 -0700, Max M <maxm@mxm.dk> wrote:
Being a full time active Zope developer (as in developing with zope) for at least 7 year, having contributed code, libraries, documentation and products, I believe I have earned the right to complain once every second year so!
Complaining is why Plone happened in the first place. ;) Keep it up! -- _____________________________________________________________________ Alexander Limi · Chief Architect · Plone Solutions · Norway Consulting · Training · Development · http://www.plonesolutions.com _____________________________________________________________________ Plone Co-Founder · http://plone.org · Connecting Content Plone Foundation · http://plone.org/foundation · Protecting Plone
One problem with fixing the refresh is that the more you fix, the longer the refresh will take. To fix Five to be stable in refresh, we may very well have to kill all things registered by ZCML, and re-register all of it. It's highly unlikely that we get a real stable refresh by just refreshing one products ZCML. What happens if there are overrides, and so on. So, in the end, you end up with refreshing everything, which would take just as long as stopping and starting the ZEO client. We can probably make it possible to make the refresh somewhat more stable and support a wider range of cases. But you still have to make very careful configuration of the refresh in most of this cases, and if you don't do it just right, it won't work. So I'm not sure it's worth the effort. Doing tests is a good idea especially if you take a bit of care. The calendar products I'm working on have that care taken automatically, as it's split into three products, one pure python, one Zope layer with the persistance and UI and one CPS integration. Most tests are on the python product, or the Zope product. The python tests take 0.8 seconds to run, the Zope tests 4 seconds. With CPS it gets worse, because the tests need to create a CPS site. It's the same with Plone. In those cases tests will not be faster than refreshing. Running all of them takes a bit less than a minute I think, and running just one isn't that much faster. But the amount of tests needed there is very low, as most is done on the CalCore and CalZope layer. This way development is pretty fast, and doesn't need much refreshing. Then comes a part where you need to get it to actually work, and not just pass the tests. Here you need to refresh a bit. And then most of the work is UI-fixing, which doesn't need refresh again, because it's mostly HTML. I find this mode of development faster than when I had working refresh, beause the existance of the tests stop me from shooting my foot. ;)
participants (9)
-
Alexander Limi -
Andreas Jung -
Dieter Maurer -
Lennart Regebro -
Martijn Faassen -
Martin Aspeli -
Max M -
Rocky Burt -
Sidnei da Silva