December 2007 - Zope-Dev

Zope Tests: 5 OK
by Zope Tests Summarizer 17 Dec '07

17 Dec '07

Summary of messages to the zope-tests list. Period Sun Dec 16 12:00:00 2007 UTC to Mon Dec 17 12:00:00 2007 UTC. There were 5 messages: 5 from Zope Unit Tests. Tests passed OK --------------- Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Sun Dec 16 20:52:12 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008806.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Sun Dec 16 20:53:42 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008807.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Sun Dec 16 20:55:14 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008808.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Sun Dec 16 20:56:44 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008809.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Sun Dec 16 20:58:14 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008810.html

1 0

Zope Tests: 5 OK
by Zope Tests Summarizer 16 Dec '07

16 Dec '07

Summary of messages to the zope-tests list. Period Sat Dec 15 12:00:00 2007 UTC to Sun Dec 16 12:00:00 2007 UTC. There were 5 messages: 5 from Zope Unit Tests. Tests passed OK --------------- Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Sat Dec 15 20:52:39 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008801.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Sat Dec 15 20:54:09 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008802.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Sat Dec 15 20:55:39 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008803.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Sat Dec 15 20:57:10 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008804.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Sat Dec 15 20:58:40 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008805.html

1 0

Zope Tests: 5 OK
by Zope Tests Summarizer 15 Dec '07

15 Dec '07

Summary of messages to the zope-tests list. Period Fri Dec 14 12:00:00 2007 UTC to Sat Dec 15 12:00:00 2007 UTC. There were 5 messages: 5 from Zope Unit Tests. Tests passed OK --------------- Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Fri Dec 14 21:07:10 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008796.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Fri Dec 14 21:08:40 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008797.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Fri Dec 14 21:10:11 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008798.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Fri Dec 14 21:11:41 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008799.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Fri Dec 14 21:13:11 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008800.html

1 0

new FAQ entry: please review before I add it
by Christophe Combelles 15 Dec '07

15 Dec '07

how do I automatically create some needed object at application startup? Do it by subscribing to IDatabaseOpenedWithRootEvent (from zope.app.appsetup) Example code: @adapter(IDatabaseOpenedWithRootEvent) def CreateMyContainer(event): conn = event.database.open() root = conn.root()['Application'] if 'mycontainer' not in root: root['mycontainer'] = MyContainer() transaction.commit() conn.close() Then register this subscriber in your configure.zcml: <subscriber handler="myapp.CreateMyContainer" />

5 5

Configuring PAU - Should it be this hard?
by kevin gill 14 Dec '07

14 Dec '07

I spent approximately a weeks work (spread over a month) to get the PAU configured with my PostgreSQL database. When I look at what I am doing now, it seems very complicated. Please have a look at the following and let me know if I am doing it the hard way. This may also be helpful for anyone trying to achieve the same result. Here is my notes: Notes on Setting up an external authentication system for Zope3 =============================================================== I use an external database to store my data. 1. My Security Model ===================== I have three types of users (principal), members, casting directors and administrators. The login information (login name, password and roles) are stored in a relational database. I am migrating a Zope 2 system exUserFolder based solution. The Zope3 concept of a role is different from the Zope2 concept. In Zope3, roles are bundles of permissions only. The user (principal) side of the functionality is provided by groups. I configure my security rules in the code using permissions, and I bundle these permissions for logical users using role ZCML statements. In the Zope ZMI I configure a PAU with three groups. I map my users (principals) to the groups from the database. In the Zope ZMI I use the grant tab in the [top] folder to map the roles to permissions. Thus my relationships are: principals -> grouped mapped to groups in database groups -> mapped to roles in grant at root (ZMI) roles -> mapped to permissions in ZCML 2. Configuring PAU =================== I wrote three plugins for the PAU to get this configuration to work. 1. I wrote a PrincipalFolder. The PrincipalFolder provides authentication for the principals. It also provides lists of principals for Rotterdam UI. 2. I wrote a Group. The Group maps a single group to the members of that group. 3. I wrote a GroupFolder. The default GroupFolder caches the data in the Group. To update the default GroupFolder you have to trigger events. The external database updates do not trigger events so this model has to be replaced. I have to do the following configurations: 1. In the manage site -> default folder create a PAU. 2. Configure the credentials plugin. For now I use : 'Zope Realm Basic-Auth (a utility)' and 'No Challenge if Authenticated (a utility)' 3. I select the PAU Plugins tab. I create my principal Folder I go back to the PAU Configure Tab and add my Principal Folder to the Authenicator Plugins 5. I select the PAU Plugins tab I create my group folder I select the new group folder I add three new group objects. The object name maps to the 'role' on my database 6. I go back to the PAU Configure Tab I add the new group folder to my Authenticator Plugins 7. I select the registration tab. I register the PAU 8. I go the the [top] I select the grant tab I use this to grant the configured roles to the new groups 9. I edit the page template and add in the following to print out the logged in person information: Developer Info: Context: Logged in user: Groups: 10. I login with the user from the database. The groups should be displayed in the developer info block. Issues ====== This solution requires a second database call to get the group after the Principal has been configured. I may have to cache when I tune the system. I have a prefix on the user. The user ids map to prefix + the database user id. The principal prefix is hard coded in the group folder to generate compatible zope principal ids.

1 0

Runaway processes
by Stephan Richter 14 Dec '07

14 Dec '07

Hi everyone, I have a problem and I am hoping that it has been solved already by someone or that I will get some input on at least.I apologize for the lengthy E-mail in advance, but I wanted to provide a detailed discussion as a starting point. Zope is designed to have very short-lived transactions. If transactions are long-living all sorts of problems arise, most notably: 1. We occupy one thread for a long time. 2. The chance of conflict errors increases. Problem 1 can be addressed by increasing the number of allowed threads or to simply add more Zope servers. But his has clearly its limits and is really just a work-around. Another way to solve the problem is to identify long-running operations and calling them asynchronous. Many of us have implemented solutions for this, one of which is lovely.remotetask. Problem number 2 can only be addressed by identifying the long-running tasks beforehand and move them into an async call, again via lovely.remotetask for example. But what happens, if a something unexpected happens and we have an unanticipated long-runnning process? The worst case being something runs forever. Then whenever this problem occurs, one thread will be locked forever, and we can have a total system lockdown in no-time. So how can this be solved? Effectively, from within Zope we cannot do anything, because (a) Zope makes no assumption about running in a thread, and (b) the application is stuck and won't have a hook to get unstuck. So we have to solve the problem from outside. Currently, Zope is commonly run from an application thread. At least both WSGI servers that we commonly use, twisted and zserver, are implemented this way. This means that by some criterion, probably some timeout, the thread should be killed. But hold on! In Python threads cannot be killed. :-( I have done some research and found issue 221115 [1], which discusses the shortcoming of not being able to kill a thread. The discussion ended in making a feature request in PEP 42 [2] which has not been implemented as far as I can tell. So I googled some more to find possible implementations. Here are two distinctively different solutions (others I have found are either obviously trivial and will not work, or are derivatives of these two): 1. A Python-only solution using sys.settrace [3]. Besides making everything very slow, sys.settrace() is only called when a new byte code instruction is executed. So in case a low-level call hangs up the process, then the trace intercept will never be called. 2. Use an exception to intercept execution on the C-level [4]. This looked very promising, until I read the following comment on the page: The exception will be raised only when executing python bytecode. If your thread calls a native/built-in blocking function, the exception will be raised only when execution returns to the python code. So my conclusion is that Python threads cannot be unconditionally killed. BTW, if a low-level call is blocking, then all Python threads are blocked. From the Python `thread` library documentation[5]: Not all built-in functions that may block waiting for I/O allow other threads to run. (The most popular ones (time.sleep(), file.read(), select.select()) work as expected.) In all fairness, though, those are very rare occurrences. Most libraries are non-blocking and the above solutions would be just fine. But in my case, I really need to find a way to kill a Zope execution environment when a C call hangs. So what other choices do we have? On Unix-like systems, we can use `os.fork()`. The advantage of this approach is that I can use OS system calls to kill the process. However, ZODB database storages cannot be shared between processes. Nikolay Kim has done some preliminary experiments and found that `db.open()` locks the system (for both, `FileStorage` and `ZeoClientStorage`). I have not verified these results or tried to figure out why it is hanging, but I can see the problem for `FileStorage`. Are there any known side-effects on what happens, if I fork after the connection has been made? Since I am using the original process merely as a control, I guess I should be fine. Of course, the interesting question is: what happens to the ZODB connection, not to mention to the DB, if it is in the middle of writing? I guess the safest solution would be to fork within the constraint of the transaction. Any comments will be very much appreciated. Once we decide on the forking approach, we have to solve the issue for Windows of course too. My googling did not turn out immediately successful, but I think if we use Windows' native threads they will provide us with the necessary API, since I can exit it at any time. .. [1]: http://bugs.python.org/issue221115 .. [2]: http://www.python.org/dev/peps/pep-0042/ .. [3]: http://www.velocityreviews.com/forums/t330554-kill-a-thread-in-python.html .. [4]: http://sebulba.wikispaces.com/recipe+thread2 .. [5]: http://docs.python.org/lib/module-thread.html Regards, Stephan -- Stephan Richter CBU Physics & Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training

6 8

zopeproject and external apidocs
by Christophe Combelles 14 Dec '07

14 Dec '07

Hi, When creating a buildout with zopeproject, the site.zcml puts the package registration before the apidoc registration: <include package="${package}" />  <include file="apidoc.zcml" /> This means that the bookchapters of some added packages (such as z3c.*) will never show up in the apidoc, because the <meta:provides feature="apidoc" /> will be located **after** all the zcml:condition="have apidoc" I suggest to move the <include file="apidoc.zcml" /> on the top of the site.zcml by default. Christophe

2 1

Zope Tests: 5 OK
by Zope Tests Summarizer 14 Dec '07

14 Dec '07

Summary of messages to the zope-tests list. Period Thu Dec 13 12:00:00 2007 UTC to Fri Dec 14 12:00:00 2007 UTC. There were 5 messages: 5 from Zope Unit Tests. Tests passed OK --------------- Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Thu Dec 13 20:53:01 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008791.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Thu Dec 13 20:54:32 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008792.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Thu Dec 13 20:56:02 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008793.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Thu Dec 13 20:57:32 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008794.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Thu Dec 13 20:59:02 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008795.html

1 0

Zope Tests: 5 OK
by Zope Tests Summarizer 13 Dec '07

13 Dec '07

Summary of messages to the zope-tests list. Period Wed Dec 12 12:00:00 2007 UTC to Thu Dec 13 12:00:00 2007 UTC. There were 5 messages: 5 from Zope Unit Tests. Tests passed OK --------------- Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Wed Dec 12 20:55:33 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008786.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Wed Dec 12 20:57:03 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008787.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Wed Dec 12 20:58:34 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008788.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Wed Dec 12 21:00:04 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008789.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Wed Dec 12 21:01:34 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008790.html

1 0

Zope Tests: 5 OK
by Zope Tests Summarizer 12 Dec '07

12 Dec '07

Summary of messages to the zope-tests list. Period Tue Dec 11 12:00:00 2007 UTC to Wed Dec 12 12:00:00 2007 UTC. There were 5 messages: 5 from Zope Unit Tests. Tests passed OK --------------- Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Tue Dec 11 20:53:07 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008781.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Tue Dec 11 20:54:37 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008782.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Dec 11 20:56:08 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008783.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Dec 11 20:57:38 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008784.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Tue Dec 11 20:59:08 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-December/008785.html

1 0