Is there a best practice for making changes to a Zope Product? Specifically I've written a product based off of ZCatalog. In the newest version I have added a few things to the _properties and a new index. I've found that instances of my product that have not made changes to the default properties pick up the new _properties. However if someone had previously added a property or changed the default values the new _properties do not show up. I was thinking it may be best to have an upgrade method that can bring any instance up to date by adding properties, etc. Is there a standard way for doing this? Any experiences on one way being better than another? FYI, We've got about a 40 gig ZODB and searching all of it to find objects to upgrade is about impossible. Thanks, Brian Brinegar Engineering Computer Network Purdue University
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Brian Brinegar wrote: | Is there a best practice for making changes to a Zope Product? | Specifically I've written a product based off of ZCatalog. In the | newest version I have added a few things to the _properties and a new | index. | | I've found that instances of my product that have not made changes to | the default properties pick up the new _properties. However if | someone had previously added a property or changed the default values | the new _properties do not show up. | | I was thinking it may be best to have an upgrade method that can | bring any instance up to date by adding properties, etc. Is there a | standard way for doing this? Any experiences on one way being better | than another? | | FYI, We've got about a 40 gig ZODB and searching all of it to find | objects to upgrade is about impossible. You can set default values on the object's class which will be picked up. E.g., ~ # old version ~ class Foo(SimpleItem): ~ """ Foo objects. ~ """ ~ bar = 0 ~ baz = '' ~ _properties = ({'id': 'bar', 'type': 'int', 'mode': 'w'}, ~ {'id': 'baz', 'type': 'string', 'mode': 'w'}, ~ ) ~ # updated version ~ class Foo(SimpleItem): ~ """ Foo objects. ~ """ ~ bar = 0 ~ baz = '' ~ qux = () ~ _properties = ({'id': 'bar', 'type': 'int', 'mode': 'w'}, ~ {'id': 'baz', 'type': 'string', 'mode': 'w'}, ~ {'id': 'qux', 'type': 'tokens', 'mode': 'w'}, ~ ) Tres. - -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCQwDHGqWXf00rNCgRAqtuAJ9eu45rhwA6YdhybbjlR2lZTtIo9gCfbEIU ady8TAI2fNCM4ApuH6+YeJI= =7Jo1 -----END PGP SIGNATURE-----
You can set default values on the object's class which will be picked up. E.g.,
~ # old version ~ class Foo(SimpleItem): ~ """ Foo objects. ~ """ ~ bar = 0 ~ baz = '' ~ _properties = ({'id': 'bar', 'type': 'int', 'mode': 'w'}, ~ {'id': 'baz', 'type': 'string', 'mode': 'w'}, ~ )
~ # updated version ~ class Foo(SimpleItem): ~ """ Foo objects. ~ """ ~ bar = 0 ~ baz = '' ~ qux = () ~ _properties = ({'id': 'bar', 'type': 'int', 'mode': 'w'}, ~ {'id': 'baz', 'type': 'string', 'mode': 'w'}, ~ {'id': 'qux', 'type': 'tokens', 'mode': 'w'}, ~ )
Tres, This isn't the case. Some instances of my product (those that haven't had properties added to them or changed) will pick up the changes, but not all instances. So for example, if I initiated an object of the old version, then added additional properties through the ZMI, then changed the product code to the updated version and restarted zope the new property will not show up in that instance. Also, this does not address adding a new index which is set normally set during the __init__ method. -Brian
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Brian Brinegar wrote: | This isn't the case. Some instances of my product (those that haven't | had properties added to them or changed) will pick up the changes, but | not all instances. If you are adding properties manually through the ZMI, then they will no longer use the '_properties' map defined by the product -- they *will*, however, have the actual attribute defined via the class (this is how Python's attribute lookup works). | So for example, if I initiated an object of the old version, then | added additional properties through the ZMI, then changed the product | code to the updated version and restarted zope the new property will | not show up in that instance. You can't reuse the default PropertyManager in this fashion: either all instances use the '_properties' map from their class (in which case, they can be upgraded cleanly), or they are "one-offs". If you want to "mix and match", you need to do more work, e.g., overriding the PropertyManager API methods to force inclusion of the version from the class. | Also, this does not address adding a new index which is set normally | set during the __init__ method. I don't know what 'index' is, but the only backwardly-compatible way to add such an attribute is to define a "default" at class scope (e.g., 'index = None'), and then check for that default value whenever you need to use it, and instantiate the "real" value at that point. I usually use an accessor method for such purposes. Tres. - -- =============================================================== Tres Seaver tseaver@zope.com Zope Corporation "Zope Dealers" http://www.zope.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCQxCdGqWXf00rNCgRAlI3AJ0Yn0a4lMsFSV1dAwITagGBL/s+6wCfd252 1wYEVPbpYqqpZxHK8E1vD/o= =FFRQ -----END PGP SIGNATURE-----
If you are adding properties manually through the ZMI, then they will no longer use the '_properties' map defined by the product -- they *will*, however, have the actual attribute defined via the class (this is how Python's attribute lookup works).
I understand this, it sounds like my best option is to compare the instances _properties map with that defined by the class adding things when necessary. It would be nice if there was a standard method that Zope called upon accessing an object that could be responsible for making these changes.
| Also, this does not address adding a new index which is set normally | set during the __init__ method.
I don't know what 'index' is, but the only backwardly-compatible way to add such an attribute is to define a "default" at class scope (e.g., 'index = None'), and then check for that default value whenever you need to use it, and instantiate the "real" value at that point. I usually use an accessor method for such purposes.
I had mentioned earlier that this was based off of ZCatalog, I'm refering to a ZCatalog Index. Initially the __init__ method of my product would add several indexes to the catalog on creation, however in the new version I want to add an additional index. I've seen several solutions to this problem, one is to have an upgrade() method that every method of your object calls first. Another is to run a script that finds all instances and does the upgrade code. There doesn't seem to be a standard/clean way of doing this. -Brian
Brian Brinegar wrote at 2005-3-24 11:47 -0500:
... I was thinking it may be best to have an upgrade method that can bring any instance up to date by adding properties, etc. Is there a standard way for doing this? Any experiences on one way being better than another?
A long time ago, Chris Withers has posted an update/migration script. Maybe you find it (search!).
FYI, We've got about a 40 gig ZODB and searching all of it to find objects to upgrade is about impossible.
Wow! Hopefully, you have an index that allows you to locate the affected objects... As an alternative to an explicit migration script (that finds and modifies all affected objects), there is the "__setstate__" kludge. "__setstate__" is used to initialize an object with state read from the ZODB. Thus, you can fix up the object there. This is a kludge, as changes you make in "__setstate__" are only made persistent when the object is later modified. I think that this can be controlled in a future ZODB version. You might get such a ZODB version with Zope 2.8 (or implement this feature yourself. It is not difficult). -- Dieter
FYI, We've got about a 40 gig ZODB and searching all of it to find objects to upgrade is about impossible.
Wow!
Is it unusual to have a ZODB of this size? Is there something else people would recommend? We had to switch to repozo for backups, but haven't had any problems in 5 years and at least that many versions of Zope. -Brian
On Mar 24, 2005, at 21:15, Brian Brinegar wrote:
FYI, We've got about a 40 gig ZODB and searching all of it to find objects to upgrade is about impossible.
Wow!
Is it unusual to have a ZODB of this size? Is there something else people would recommend? We had to switch to repozo for backups, but haven't had any problems in 5 years and at least that many versions of Zope.
This is one of the largest sizes I have heard about, yet it's nothing that should present any problems for FileStorage. jens
Unless the database mostly consists of records that will never again be touched, I would suggest going through the whole database and upgrade object. I have found that when upgrading objects is necessary, it's a good thing to set an __object_version__ attribute or something on the object, to quickly check if it has been upgraded or not. This is especially important if you don't do batch upgrading, as you don't want to have twenty lines of code doing the upgrade each time you access the object. The __setstate__ kludge is typically not a good one, since the changes aren't persisted. It's better to create an _upgrade() method on the object, that first checks the __object_version__ and upgrades if necessary, and then simply call this method first in all accessing methods. If you usually access the attributes directly before any methods are called, you are pretty much in deep shit. ;) Then you really have to do a "batch" upgrade. -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/
Hi. I am working on a financial product and it appears to me that the /manage login for Zope could be a potential problem if you are running zope since your server is easily guessed and one can go to this url and try passwords. Can someone suggest an alternative to this or some modification to Zope that might make this less obvious. I best I can think of would be to do a rewrite on the /manage url but I still need manager access to zmi through the web. I plan on forcing ssl through apache when making a connection on whatever URL is used to login. Any ideas? Regards, David
On Fri, 25 Mar 2005 08:30:05 -0400, David Pratt <fairwinds@eastlink.ca> wrote:
Hi. I am working on a financial product and it appears to me that the /manage login for Zope could be a potential problem if you are running zope since your server is easily guessed and one can go to this url and try passwords. Can someone suggest an alternative to this or some modification to Zope that might make this less obvious. I best I can think of would be to do a rewrite on the /manage url but I still need manager access to zmi through the web. I plan on forcing ssl through apache when making a connection on whatever URL is used to login. Any ideas?
You can set up apache so it only allows access to "manage*" from certain adresses, like your internal net and stuff. I don't have the examples at close hand,sorry. -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/
Yes, this is a good approach, however am concerned about management from locations that may not have a static IP (if IP changes, then you are hooped). I am also looking for a way that this might not be tied to where someone might be located. I don't know if there is a solution that could involve a rewrite rule to manage and having a specific URL (other than manage) for logging in that is only known to the manager. I guess the other thing I ought to be considering is another rule to prevent username passwords from being passed in URL as well. I am sure someone has probably done this as well. Regards, David On Friday, March 25, 2005, at 08:48 AM, Lennart Regebro wrote:
On Fri, 25 Mar 2005 08:30:05 -0400, David Pratt <fairwinds@eastlink.ca> wrote:
Hi. I am working on a financial product and it appears to me that the /manage login for Zope could be a potential problem if you are running zope since your server is easily guessed and one can go to this url and try passwords. Can someone suggest an alternative to this or some modification to Zope that might make this less obvious. I best I can think of would be to do a rewrite on the /manage url but I still need manager access to zmi through the web. I plan on forcing ssl through apache when making a connection on whatever URL is used to login. Any ideas?
You can set up apache so it only allows access to "manage*" from certain adresses, like your internal net and stuff. I don't have the examples at close hand,sorry. -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/
On Fri, 25 Mar 2005 09:19:53 -0400, David Pratt <fairwinds@eastlink.ca> wrote:
Yes, this is a good approach, however am concerned about management from locations that may not have a static IP (if IP changes, then you are hooped). I am also looking for a way that this might not be tied to where someone might be located. I don't know if there is a solution that could involve a rewrite rule to manage and having a specific URL (other than manage) for logging in that is only known to the manager.
No, but I think you can allow manage* only via SSL for example. This makes it very unobvious that it uses Zope, and increases security quite a bit. -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/
David Pratt wrote:
Yes, this is a good approach, however am concerned about management from locations that may not have a static IP (if IP changes, then you are hooped). I am also looking for a way that this might not be tied to where someone might be located. I don't know if there is a solution that could involve a rewrite rule to manage and having a specific URL (other than manage) for logging in that is only known to the manager.
I guess the other thing I ought to be considering is another rule to prevent username passwords from being passed in URL as well. I am sure someone has probably done this as well.
It sounds like you should just be doing everything over https. How much of a problem would this be? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk
Lennart Regebro wrote at 2005-3-25 13:48 +0100:
... You can set up apache so it only allows access to "manage*" from certain adresses, like your internal net and stuff. I don't have the examples at close hand,sorry.
Note that any knowledgable person can easily work around such a restriction implemented in Apache. You can construct requests causing arbitrary traversal in Zope without Apache seeing anything about this... -- Dieter
David Pratt wrote at 2005-3-25 08:30 -0400:
I am working on a financial product and it appears to me that the /manage login for Zope could be a potential problem if you are running zope since your server is easily guessed and one can go to this url and try passwords. Can someone suggest an alternative to this or some modification to Zope that might make this less obvious.
You can use a "Post Authentication Hook" to perform additional tests. There are two competing proposals for such a hook (one in the collector and one on my Zope page). I doubt that any one has been integrated in the actual Zope code. But it should not be difficult to apply a patch. -- Dieter
Brian Brinegar wrote at 2005-3-24 15:15 -0500:
FYI, We've got about a 40 gig ZODB and searching all of it to find objects to upgrade is about impossible.
Wow!
Is it unusual to have a ZODB of this size? Is there something else people would recommend? We had to switch to repozo for backups, but haven't had any problems in 5 years and at least that many versions of Zope.
40 GB is already quite large. Maybe not for normal operation -- but for a migration script (if locating the affected objects it not supported by an appropriate index). -- Dieter
participants (7)
-
Brian Brinegar -
Chris Withers -
David Pratt -
Dieter Maurer -
Jens Vagelpohl -
Lennart Regebro -
Tres Seaver