April 2004 - Zope-Dev

RE: Zope + Ape + Subversion (was: RE: [Zope-dev] Using a truely r evis ion based storage for Zope ?)
by Jean-Francois.Doyon＠CCRS.NRCan.gc.ca 14 Apr '04

14 Apr '04

Hello, Hmmm, well it's as stable as Ape and Subversion are respectively :) I wouldn't call it stable no, it's something I did over the long week-end we just had, and that's about it :) Ape is at 0.8 and therefore becoming quite mature, I'd have to let others speak as to it stability however ... Subversion is also probably quite stable (It just reached 1.0), though I don't know how heavily tested it's been in a long running process (Might it have some memeory leaks ?) ... Basically, I wouldn't recommend for usage yet. I'm not even distributing it just yet, mostly simply because right now it's code is still intermingled with Ape's ... As soon as I pull it out into a seperate product I'll be sure ot post it on zope.org and notify the list, as I'm sure I'll be looking for feedback/testers. I would estimate that moment to be weeks away though ... Thanks, J.F. -----Original Message----- From: Arthur Chan Chi Chuen To: Kapil Thangavelu; Jean-Francois.Doyon(a)CCRS.NRCan.gc.ca Cc: zope-dev-bounces(a)zope.org; shane(a)zope.com; zope-dev(a)zope.org Sent: 13/04/2004 2:13 PM Subject: Re: Zope + Ape + Subversion (was: RE: [Zope-dev] Using a truely revis ion based storage for Zope ?) Hi all, I've read your discussion about version control, it seems a cool thing you guys making good progress. Btw, can I ask is the Ape using Subversion in Zope stable? how able CMF stuff? I wanna make/find a document management system which can provide some kinda version control in Plone. Thanks Arthur On Tue, 13 Apr 2004 04:06:26 -0400, Kapil Thangavelu wrote > On Mon, 2004-04-12 at 18:03, Jean-Francois.Doyon(a)CCRS.NRCan.gc.ca wrote: > > G'Day, > > > > Well, step one is done ... I now have Zope + Ape using Subversion as it's > > "filesystem" !! > > > > cool! > > > This is step one because, as Shawn suggested (Thanks for the pointer, that's > > what I needed!), this simply means that Zope uses SVN purely as a > > filesystem. > > > > > Because of subversion's nature, I want to look at 2 things beyond this that > > traditional filesystems don't support: > > > > - Use zope's username for SVN logging. > > using AccessControl.getSecurityManager().getUser().getId() and > setting it up as a revision prop ( or directly when creating the > repo transaction) should do it. > > > - History/Undo support: View past revisions of an object, and revert to such > > a past revision. > > perhaps the code for this would be helpful, (remove image for code > link) http://zope.org/Members/k_vertigo/Products/CMFSubversionBrowser/FileRevi sions. png > > > - Zope Version support: SVN is fully transactional and atomic, this should > > allow for support of Zope versions (I think ?) > > > > zope version support isn't really all that worthwhile, esp in a cmf > context. in general zope's version support (or zodb more > particularly) is a database level feature masquerading as an > application one. since objects modified in a version are in essence > locked from participating in other transactions, actions like > modifying content in a version in a cmf site amounts to locking the > catalog from changes outside of the version, which amounts to > shutting down write activities to a cmf site. otoh, integration with > zope's transaction manager would be a good thing, although its > problematic to integrate between svn and zope txn models, more on > that in a moment. > > > In the longer term, there's great opportunity for: > > > > - "Built-in" conflict management and resolution: No more need for a > > "SafetyBelt" type approach. Right now I haven't looked at this at all. I > > plan to implement smart merging where possible (It might work already > > actually, I just need to test it). True conflicts (Where a merge can't be > > accomplished withouth user interaction) would raise some sort of conflict > > error. > > > > i don't know that conflict management is really useful in this context. > svn like zope relies on optimistic concurrency control, and currently > doesn't support dav locking (which zope does). ie, it will just > throw an error if the content has been changed since the transaction > began. the 'normal' concurrency control of svn is better but > dependent on using the working copy (wc) layer, which is additional > programming and storage overhead. so at the layer of the svn client > this is already done and working and good, but integrating this > functionality into zope is a bit harder without wc overhard. > > this also makes the transaction integration becomes harder because both > zope and svn are using what amounts to optimistic concurrency control > which makes it impossible afaics, to get real txn integration, ie in > zope's two phase commit strategy, the last chance for a participant > to safely abort is tpc_vote, but there is no real way of knowing if > the svn txn will suceed or not until its tried. if it is tried at > this stage and succeeds then there is the possibility of a latter > txn participant failing the tpc_vote and the txn being aborted, and > if waits till tpc_finish (last part of two phase commit) and the svn > txn fails it can hose the composite txn integrity. > > once svn supports dav locks, doing txn integration via resource locking > as part of tpc_vote (or earlier) would be possible, till then.. i > dunno, i can't see a way around this for real txn integration. > > i'm also curious how you dealt with svn transactions as part of the ape > integration work to date. > > > - Editing Zope content objects through interaction with the svn repository. > > I can checkout the repository, edit some objects, and chek them back in, > > never interacting with Zope directly ... I've already tried this ! Works > > great for text based content types such as PageTemplates or DTML Documents > > and so on ... I even did it with a JPG, though because the properties hold > > width and height, you get some weird looking pictures :) The concept is > > valid though. There may someday be a way to leverage this functionality > > better with a purpose built client of some sort. > > to me this is one of the fundamental benefits of using svn, giving users > the ability to use interfaces like tortoisesvn (win shell extensions) > or mac finder extensions to interact directly with content. > > > > > - Leveraging SVN's property management. Content in SVN has properties, much > > like Zope does. I haven't looked at it yet, but I've noticed ".properties" > > file appearing ... I'm guessing those are the Zope properties, which would > > be better handled by subversion's property mechanism. And properties are > > versioned too ! > > definitely! > > > > > In the realm of the wishful thinking, there's even more: > > > > Right now, HEAD (Latest/youngest revision) is always used and worked with. > > The really powerful feature I want to eventually get to is publsihing > > something of a given revision, while editing another. One potential > > paradigm for distiguishing between the two modes of operation could be to > > use anonymous vs. authenticated. This is not useful to everyone, but can be > > in certain circumstances, most notably where authenticated = > > authors/developpers and anonymous = normal users. This however requires ZMI > > interfaces, and in my case CMF ones as well ... This would be global though > > ... Eventually it'd be nice to have per object control of this stuff. Andy > > McKay says it can't be done, anybody care to contradict him ? :P I image I'd > > have to monkey pathc something DEEP in the Zope code base, but I find the > > mix-in class that's the commonn denominator ... why not ? > > > > to me this seems a separate point then svn usage at all, its really > about content view paradigms for revision workflow. the two predominant > ones are using staging to deploy different sites representing different > parts of the publication cycle/wf, and the other which afaik, is only > widely used by silva is view multiplexing where the view of a > content is responsible for retrieving the proper version to render a > display for, although afaicr silva support for such is initimately > tied to its workflow and versioning paradigm. > > > Anyways, I'm just rambling by now ... Comments, thoughts and constructive > > criticism welcome ! > > i've done some thinking and work with integration of svn as content > in zope as well, though using a different software stack than ape. some > notes/ideas culled from a dev diary i kept while working on it that > might be useful. > > - using svn hook scripts for event notification back into zope (content > reindex), although it should be careful about modifying the content > otherwise some client which just committed might be put out of date. > > - transform layers (using portal transforms) with a private content type > registry... although integrating at the ape layer this might not be > as useful. > > cheers, > > -kapil > > > > > Thanks, > > J.F. > > > > -----Original Message----- > > From: zope-dev-bounces(a)zope.org [mailto:zope-dev-bounces@zope.org]On > > Behalf Of Shane Hathaway > > Sent: April 8, 2004 11:20 AM > > To: Jean-Francois.Doyon(a)ccrs.nrcan.gc.ca > > Cc: zope-dev(a)zope.org > > Subject: Re: [Zope-dev] Using a truely revision based storage for Zope ? > > > > > > Jean-Francois.Doyon(a)CCRS.NRCan.gc.ca wrote: > > > I've started looking at the ZODB and APE packages to try and get some > > > understanding of how the whole storage interaction works, but it'll take > > me > > > some time to figure it all out ... So I thought I'd get feedback on the > > idea > > > first ... > > > > Sounds great! If I were you, I would start by replacing > > Ape/lib/apelib/fs/fileops.py with something that uses the Subversion > > module. It's a very simple module and you should be able to get pretty > > far that way. > > > > Shane > > _______________________________________________ > Zope-Dev maillist - Zope-Dev(a)zope.org > http://mail.zope.org/mailman/listinfo/zope-dev > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope ) -- Open WebMail Project (http://openwebmail.org)

3 2

PlacelessTranslationService syntax
by Milos Prudek 14 Apr '04

14 Apr '04

What's the correct ZPT syntax for PlacelessTranslationService? I tried this: <b i18n:translate="" i18n:domain="ibc" i18n:target="string:cz">Good morning</b> This gets rendered into: <b>Good morning</b> No translation, although I do have an entry for 'Good morning' in my .po file under an 'ibc' domain. Is my syntax wrong? -- Milos Prudek

2 3

ZCatalog fast as admin, dog slow as anonymous and other users
by Jason Spisak 14 Apr '04

14 Apr '04

Zopistas, My ZCatalog is fast as admin, but dog slow as anonymous and other users. Anyone had this same experience? Details: marjors: Zope 2.6.2 CMF 1.3 Plone 1.0.5 about 50,000 cataloged objects (dual xenon server, plenty of ram, RAID) User folder has 15k users in it, but admin is one of those users. i've factored it way down and a simple search with just 1 Event loaded, like: return context.portal_catalog(Type='Event') is instantaneous for the 'admin' user, but takes about 10 seconds to return 1 SINGLE object. (only 1 Event loaded) imagine the time it takes to search and display 1300 events (which is what's normally in there). ouch. i've rebuilt the indexes, and still no dice. Any help would be greatly appreciated. All my best, Jason Spisak

4 17

RE: Zope + Ape + Subversion (was: RE: [Zope-dev] Using a truely r evis ion based storage for Zope ?)
by Jean-Francois.Doyon＠CCRS.NRCan.gc.ca 14 Apr '04

14 Apr '04

Brad, About the branch thing ... That's basically the idea! The bigger problem here is how to "manage" this both internally and from a user/administrator perspective. This paradigm only really makes sense in the CMF world anyways, and I want to focus on basic Zope before moving up to the extra layer CMF represents. I'm also worried getting something like this to work with CMF woudl require an ungodly amount of monkey patching :P The main things with CMF would be: 1) "published" type states vs. other states: how does one tell/define the difference ? How does this information get passed to/from the publishing system or the storage layer ? Possible solution: Have the subversion/zope interface interact with DCWorkflow. When an object is changed, consult DCWorkflow settings and determine what state the object is in. If the state is deemed "published", copy the object into the publishing branch. 2) If we got this far, then I need to know when to use the branch vs. the trunk. From a user prespective this isn't too big a deal, a ZMI interface, cookies, etc ... That's all ok ... The problem is that by the time the request comes to the storage side of things, that information is most likely gone (Don't know though, I'm still learning). Then there's the whole caching thing that would probably complicate things. 2) Message logging: CMF "comments" when doing a workflow action could go straight into SVN's logging, again how this might be accomplished isn't clear as of yet. This is all for the future though, I'm nowhere near this yet!! Also, I'm thinking the cooler features may just have to wait for Zope 3 ... Implementing them in Zope might be too convoluted. I should probably start a Wiki on this topic ! J.F. -----Original Message----- From: zope-dev-bounces(a)zope.org To: zope-dev(a)zope.org Sent: 13/04/2004 8:40 AM Subject: Re: Zope + Ape + Subversion (was: RE: [Zope-dev] Using a truely revis ion based storage for Zope ?) On 12 Apr 2004 at 18:03, Jean-Francois.Doyon(a)CCRS.NRCan.gc.ca wrote: > > Anyways, I'm just rambling by now ... Comments, thoughts and constructive > criticism welcome ! This sounds wonderful! Regarding versions. Perhap you can have a control panel setting that selects the branch to publish "by default". The ZMI could offer another mechanism that uses cookies to select the desired "branch or version" That is, a branch name or "PREV, HEAD" etc. But, I use jEdit and FTP for most of my Zope work, so would really like to have a way to access version/branches via FTP. Maybe that can be done by having pseudo subdirectories named after revisions or branches. So /myFolder/.Prev/index_html would do what you expect (note you can't save to old versions anyway ..) -- Brad Clements, bkc(a)murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax http://www.wecanstopspam.org/ AOL-IM: BKClements _______________________________________________ Zope-Dev maillist - Zope-Dev(a)zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )

2 1

Zope Developers
by Paul Edward Brinich 13 Apr '04

13 Apr '04

Hello, I was wondering if someone on this list could point me in the direction of an appropriate place to post Zope job openings. I am looking for an audience well-versed in Zope. Thanks for any guidance! -Paul Brinich

2 1

[Zope-CMF] Tests (Was: some small proposals)
by Lennart Regebro 13 Apr '04

13 Apr '04

I would like to see the following functionality when it comes to unit testing: 1. An easy consistent way to run ONE unit test, no matter if it's made for Zope, or CMF, and no matter it it's done with straight unittest or ZopeTestCase. 2. An easy and consistent way to run all unit test in a directory. No matter if its... and so on. 3. An easy way to run all unit tests in Products. No matter etc, etc... 4. An easy way to run all unit tests in both Products and lib/python. No matter blablabla... 5. This should be reasonably easy to do from both Windows and Unix. To me, it seems like testrunner should be able to do this, if it weren't for the CMF tests for some reason unknown to me breaking this pattern. I would like that fixed. I'm not 100% sure what the problems are and my feeble attempt of suggesting fixes earlier haven't been any good. I think it would be great if thsi could be dicsussed, and useful solutions suggested.

1 0

Re: [ZODB-Dev] Re: [Zope3-dev] Proposal: Move to subversion for source code control of the Zope and ZODB projects
by Sidnei da Silva 12 Apr '04

12 Apr '04

On Mon, Apr 12, 2004 at 08:11:44AM -0400, Tres Seaver wrote: | Lennart Regebro wrote: | >From: "Jim Fulton" <jim(a)zope.com> | > | >>Initially, I propose to move just the repository heads. Maintenamce | >>branches (e.g. Zope 2.6 and Zope 2.7) will remain in CVS. | > | >What is the rationale behind not moving it all? | | Risk mitigation; we have significant support requirements for deployed | systems built around CVS for both 2.6- and 2.7-based sites, whereas the | head and Zope3 have no such inertia. We don't know enough about the | migration process to have any confidence that existing history / | branches, etc., won't be somehow messed up, which makes it prudent to | plan the migration around the already-unstable bits. Just FYI, Kapil has been doing a good job at improving the script that migrates from CVS to SVN, and we are ready to flip the switch to move Plone, Collective and possibly Archetypes to SVN. -- Sidnei da Silva <sidnei(a)awkly.org> http://awkly.org - dreamcatching :: making your dreams come true http://plone.org/about/team#dreamcatcher Real computer scientists like having a computer on their desk, else how could they read their mail?

2 1

ZConfig 2.1 released
by Fred Drake 12 Apr '04

12 Apr '04

I've posted a distribution for ZConfig 2.1 on the ZConfig page: http://zope.org/Members/fdrake/zconfig/ This fixes a few bugs and improves the ability to set default values in schemas. It also adds some helpful schema building blocks, including a general mapping type and support for Python's logging package. This version of ZConfig requires Python 2.3. >From the news file: - Removed compatibility with Python 2.1 and 2.2. - The default objects use for section values now provide a method getSectionAttributes(); this returns a list of all the attributes of the section object which store configuration-defined data (including information derived from the schema). - Default information can now be included in a schema for <key name="+"> and <multikey name="+"> by using <default key="...">. - More documentation has been added to discuss schema extension. - Support for a Unicode-free Python has been fixed. - Derived section types now inherit the datatype of the base type if no datatype is identified explicitly. - Derived section types can now override the keytype instead of always inheriting from their base type. - <import package='...'/> makes use of the current prefix if the package name begins witha dot. - Added two standard datatypes: dotted-name and dotted-suffix. - Added two standard schema components: ZConfig.components.basic and ZConfig.components.logger. -Fred -- Fred L. Drake, Jr. <fred at zope.com> PythonLabs at Zope Corporation

1 0

circular referenced persistent objects
by zope＠netchan.cotse.net 12 Apr '04

12 Apr '04

I have the following setup (unrelated lines are omitted): class Deliverer(Folder): def manage_afterAdd(self, item, container): if item is self: self.__ac_local_roles__ = dr_localroles(self) class dr_localroles(Persistent): def __init__(self, dr): self.dr = dr dr_localroles implements a few methods of the dictionary interface, so I can always check the effective settings via the Security/local roles zmi page. In order to computing roles, dr_localroles requires its parent object instance (Deliverer), that's why I store a reference to it as an attribute. Question: - Is there a way for dr_localroles to refer its parent without holding a reference to it or using weak reference? - Is there an equivalent of sys.getrefcount for ZODB persistent objects? Also a note for the archives: I wanted to make sure that if I delete a Deliverer object, it will indeed be removed from ZODB. For this I got the oid of the object, deleted it and than tried to get it back via the oid. The trick is, that you'll only get a POSKeyError if you pack the database and flush the cache. This latter one was not obvious for me. Reagards, Sandor

3 4

Thread in ZODB
by Andre Meyer 11 Apr '04

11 Apr '04

Hi Zopers Here is a nice challenge, I hope: I have a multi-threaded Python application that I want to link to Zope. The idea is to provide a Web interface for viewing and editing the state of multiple threads running in Python. On thread should be associated with Zope and act as the "door" between the ZServer and the other threads. Unfortunately, I do not manage to create a reference to a thread in the Zope product's main class because the ZODB refuses to add it. This is the error message when trying to instantiate the product: Site Error An error was encountered while publishing this resource. Error Type: UnpickleableError Error Value: Cannot pickle <type 'thread.lock'> objects And this is the code where it goes wrong: class Zpyse(SimpleFolder): meta_type = "Zpyse" manage_options = ( {'label':'Edit', 'action':'manage_main'}, {'label':'View', 'action':'index_html'} ) index_html = PageTemplateFile('zpt/index_html', globals()) def manage_editZpyse(self, title, REQUEST=None): "Method to edit Zpyse instances." self.title = title if REQUEST is not None: return self.index_html(self, REQUEST) def __init__(self, id, title): # Sender/Receiver Test print 'init Zpyse' ams = spyse.getAMS() * self.za = ams.createThread('ZopeThread', 'ZopeThread', globals()) qa = ams.createThread('Receiver', 'ReceiverThread', globals()) sa = ams.createThread('Sender', 'SenderThread', globals()) * only this goes wrong, the other threads are created (without self.). Is there anybody with a good idea about how to handle this? thanks a lot in advance Andre -- Dr. Andre P. Meyer http://home.hccnet.nl/a.meyer/ TNO FEL Command & Control and Simulation, http://www.fel.tno.nl/div2/ Delft Cooperation on Intelligent Systems, http://www.decis.nl/

3 3