- Zope - Zope lists

long running methods
by Jürgen Herrmann 23 May '10

23 May '10

hi there! as our zope2 zodb grows we experience more and more conflicts with long running transactions. so i'm going to rewrite these methods to commit transactions in batches - here my attempt to tackle this problem. the idea is to get a list of objects to be modified in this transaction, break it down to batches and commit after each batch was processed. to avoid zope's retry-on-conflict machinery and still have a chance to retry per batch transaction commits, the iteration over one batch is guarded by a try:except ConflictError: and a retry logic. i wrote some more or less pseudo code to make this clear. def myLongRunningMethod(self): BATCH_SIZE = 10 MAX_CONFLICTS = 3 work_items = [some, work, to , do, ...] # list of persistent objects to be modified in this method idx = 0 while idx < len(work_items): conflicts = 0 try: my_batch = work_items[idx, idx+BATCH_SIZE] for work_item in my_batch: do_some_work(work_item) transaction.commit() except ConflicError: conflicts += 1 if conflicts > MAX_CONFLICTS: raise else: idx += BATCH_SIZE does this sound like a reasonable approach? jürgen -- >> XLhost.de - eXperts in Linux hosting ® << XLhost.de GmbH Jürgen Herrmann, Geschäftsführer Boelckestrasse 21, 93051 Regensburg, Germany Geschäftsführer: Volker Geith, Jürgen Herrmann Registriert unter: HRB9918 Umsatzsteuer-Identifikationsnummer: DE245931218 Fon: +49 (0)800 XLHOSTDE [0800 95467833] Fax: +49 (0)800 95467830 WEB: http://www.XLhost.de IRC: #XLhost(a)irc.quakenet.org

6 9

javascript experts?
by Garry Saddington 22 May '10

22 May '10

I am using some dynamic javascript to build form input controls and I have got everything working perfectly in all browsers except IE. The javascript uses square bracket notation to allow the phones.number:records to build multiple database input records for Zope. if (eval("document.forms.playlist['phones.number:records'][" + idx + "].value") > 0) However,IE interprets it as: "document.forms.playlist.phones.number:records" and of course therefore does not find the element referenced. Can anyone suggest a workaround for IE in this respect. Thanks Garry

2 2

PAU + loginpagename = FAIL
by Dr. Volker Jaenisch 20 May '10

20 May '10

Hi zope list! Tried to use the PAU feature of the challenge mechanism of the sessionbasedcredential plugin in bluebream. The main registration of PAU : @adapter(INewMLSApplicationEvent) def createPAU( event ): sm = event.object.getSiteManager() pau = PluggableAuthentication() sm['authentication'] = pau sm.registerUtility(pau, IAuthentication) users = PrincipalFolder() sm['authentication']['Users'] = users sm.registerUtility(users, IAuthenticatorPlugin, name="Users") pau.credentialsPlugins = ( "No Challenge if Authenticated", "Session Credentials" ) # introduce my own login page for name, plugin in pau.getCredentialsPlugins(): #iterate over all credential plugin tupels if hasattr(plugin,'loginpagename'): plugin.loginpagename = '@@login' This code should bring the functionality that an unauthorized user will jump to the login page (@@login). I addressed without a former login the following URL http://127.0.0.1:8080/test11/@@test where test11 is my custom site-manager-instance and @@test a empty test-view just containing a pdb in __call__. class testPage(BrowserPagelet): """test pagelet""" def __call__(self): import pdb; pdb.set_trace() def update(self): import pdb; pdb.set_trace() I end up with URL: http://127.0.0.1:8080/test11/%40%40test Module paste.evalexception.middleware:*306* in |respond| <http://127.0.0.1:8080/test11/@@test#> |<< <http://127.0.0.1:8080/test11/@@test#> *try**:* __traceback_supplement__ *=* errormiddleware*.*Supplement*,* self*,* environ app_iter *=* self*.*application*(*environ*,* detect_start_response*)* *try**:* return_iter *=* list*(*app_iter*)*||>> <http://127.0.0.1:8080/test11/@@test#>app_iter *=* self*.*application*(*environ*,* detect_start_response*)*| Module paste.translogger:*68* in |__call__| <http://127.0.0.1:8080/test11/@@test#> |<< <http://127.0.0.1:8080/test11/@@test#> self.write_log(environ, method, req_uri, start, status, bytes) return start_response(status, headers) return self.application(environ, replacement_start_response) def write_log(self, environ, method, req_uri, start, status, bytes):||>> <http://127.0.0.1:8080/test11/@@test#>*return* self*.*application*(*environ*,* replacement_start_response*)*| Module zope.app.wsgi:*59* in |__call__| <http://127.0.0.1:8080/test11/@@test#> |<< <http://127.0.0.1:8080/test11/@@test#> handle_errors *=* environ*.*get*(*'wsgi.handleErrors'*,* self*.*handleErrors*)* request *=* publish*(*request*,* handle_errors*=*handle_errors*)* response *=* request*.*response # Get logging info from principal for log use||>> <http://127.0.0.1:8080/test11/@@test#>request *=* publish*(*request*,* handle_errors*=*handle_errors*)*| Module zope.publisher.publish:*131* in |publish| <http://127.0.0.1:8080/test11/@@test#> |<< <http://127.0.0.1:8080/test11/@@test#> obj *=* publication*.*getApplication*(*request*)* obj *=* request*.*traverse*(*obj*)* publication*.*afterTraversal*(*request*,* obj*)*||>> <http://127.0.0.1:8080/test11/@@test#>obj *=* request*.*traverse*(*obj*)*| Module zope.publisher.browser:*556* in |traverse| <http://127.0.0.1:8080/test11/@@test#> |<< <http://127.0.0.1:8080/test11/@@test#> nsteps *=* *0* ob*,* add_steps *=* publication*.*getDefaultTraversal*(*self*,* ob*)* *while* add_steps*:* nsteps *+=* len*(*add_steps*)*||>> <http://127.0.0.1:8080/test11/@@test#>ob*,* add_steps *=* publication*.*getDefaultTraversal*(*self*,* ob*)*| Module zope.app.publication.browser:*36* in |getDefaultTraversal| <http://127.0.0.1:8080/test11/@@test#> |<< <http://127.0.0.1:8080/test11/@@test#> *if* IBrowserPublisher*.*providedBy*(*ob*)**:* # ob is already proxied, so the result of calling a method will be *return* ob*.*browserDefault*(*request*)* *else**:* adapter *=* queryMultiAdapter*(**(*ob*,* request*)**,* IBrowserPublisher*)*||>> <http://127.0.0.1:8080/test11/@@test#>*return* ob*.*browserDefault*(*request*)*| *Unauthorized: (<z3c.pagelet.zcml.testPage object at 0x9e4838c>, 'browserDefault', 'zope.ManageContent') *This trace is correct in so far that the view has the following permission settings:  <z3c:pagelet for="*" name="test" class=".test.testPage" permission="zope.ManageContent" layer="propertyshelf.mls.skin.interfaces.IDefaultLayer" /> * * 1) The plugin "Session Credentials" is registered and works. I know this because the "extractCredentials"-Routine of the plugin is called and it is my instance of this plugin because it has its loginpagename set to "@@login" which is not the default value. 2) But the "challenge"-routine of the plugin which should bring up the login page is never called. 3) I traced that back into PAU itself. The "unauthorized"-routine in zope.pluggableauth-1.0.1-py2.6.egg/zope/pluggableauth/authentication.py which will in turn call the plugins challenge-routine is also not called. 4) But the routine "unauthenticatedPrincipal" is called which is not implemented. 5) I had the slight suspicion that this has something to do with the exception/event-handling. So I tried to run the bluebream instance not in debug-mode (which catches some ecxeptions) but in deploy-mode utilizing ./bin/paster serve deploy.ini But this ends in the following stacktrace 2010-05-20T00:08:57 ERROR SiteError Error while reporting an error to the Error Reporting utility Traceback (most recent call last): File "/home/volker/workspace/vbb/propertyshelf.mls/eggs/zope.app.publication-3.10.2-py2.6.egg/zope/app/publication/zopepublication.py", line 263, in _logErrorWithErrorReportingUtility errUtility = zope.component.getUtility(IErrorReportingUtility) File "/home/volker/workspace/vbb/propertyshelf.mls/eggs/zope.component-3.9.3-py2.6.egg/zope/component/_api.py", line 171, in getUtility raise ComponentLookupError(interface, name) ComponentLookupError: (<InterfaceClass zope.error.interfaces.IErrorReportingUtility>, '') <zope.publisher.interfaces.EndRequestEvent object at 0x94185ac> Any help or idea appreciated Volker -- ==================================================== inqbus it-consulting +49 ( 341 ) 5643800 Dr. Volker Jaenisch http://www.inqbus.de Herloßsohnstr. 12 0 4 1 5 5 Leipzig N O T - F Ä L L E +49 ( 170 ) 3113748 ====================================================

1 0

Problem with non-latin character sets in Page Templates
by Christopher N. Deckard 17 May '10

17 May '10

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello, Due to ICANN's decision to allow for non-Latin characters in domain names, I thought I'd give our Zope installation a test to see how it would handle it. We've had problems with people copy and pasting from MS Word, and its use of strange characters. I'm sure at some point the faculty here at Purdue will have a need to link to web sites using non-Latin characters. This is my example HTML: <html> <head> <title>URL Test</title> </head> <body> <a href="http://موقع.وزارة-الأتصالات.مصر">http://موقع.وزارة-الأتصالات.مصر</a>. </body> </html> Not only is it a different character set, but it is a right to left character set. This code works fine while editing a Page Template. However, when viewing it, all of those characters in the href are converted to question marks. If the same HTML is pasted into a DTML Method or a File object, Zope will convert the characters to ASCII characters. When viewed it will be viewed correctly. Any reason that Page Templates may fail to render this properly? - -Chris - -- - -------------------------------------------------------------------- Christopher N. Deckard | Lead Web Systems Developer cnd(a)ecn.purdue.edu | Engineering Computer Network http://eng.purdue.edu/ECN/ | Purdue University - ---- zlib.decompress('x\234K\316Kq((-J)M\325KM)\005\000)"\005w') --- -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.12 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkvxoQcACgkQ+zGkEDwn8UZcGACgufURwanURBIBcZUSLYYRkEvV vKwAoNnGMlmOhE/GciZuTO3ldptuBr4Z =mypE -----END PGP SIGNATURE-----

2 1

Is shared ZEO cache possible?
by Roopesh P Raj 17 May '10

17 May '10

Hi, In the zope application that I am working on, there are nearly 40 ZEO clients. Restarting 40 ZEO clients is a time consuming process which takes hours - especially the creation of .zec files in each of the ZEO clients takes a lot of time. My doubt is whether it is possible to use one .zec file shared among all the ZEO clients? Thanks, Roopesh

3 2

Zope Control Panel extension
by Encolpe Degoute 17 May '10

17 May '10

Hello, I found a collateral damage of the suppression of ZClasses in Zope 2.12: eggs that try to add a new item in the Zope Control Panel can do it but they disappear after the initialization phase. Three eggs are concerned: - collective.monkeypatcherpanel (pure Zope2) - Products.PlacelessTranslationService (mainly Plone) - WingDBG (Zope 2 ans Plone) They all use the same code : http://svn.plone.org/svn/collective/collective.monkeypatcherpanel/trunk/col… Is there a method to add item in the Zope Control Panel without set enable-product-installation to 'on' ? Regards, Encolpe DEGOUTE -- Si ca ne marche pas du premier coup, appelez-le version 1.0.

1 0

Dependent ZCA utility registration ?
by Peter Sabaini 12 May '10

12 May '10

I have trouble registering utilities which depend on the presence of other utilities; eg. I have a package A where I want to register utility B which depends on registration of utility C. I tried to register B by putting registration in the __init__.py of A, which fails because at import time C is not yet registered. Registration by subscribing to the utility registration event handler for C also failed, because by the time the handler gets called, the utility C is not yet installed. My current workaround involves registration upon the IDatabaseOpen event, which is doubly ugly because Zope 2.10 doesnt send that event out of the box -- I send it in a monkey-patched Zope2.App.startup module. What is the canonical way to do this? Thanks, peter.

1 0

installation zope on windows 7
by henk laloli 11 May '10

11 May '10

I have tried to install the latest zope 2.12.5 on windows 7 (64 bit). Ran the zope egg. Made a zope instance with mkzopeinstance. One needs to disable the user accounts settings (run command uac and set slider to null) otherwise the service will not install. Still after this Zope will not start: there is no PID file present. After starting the Zope process immediately dies. So what could be the cause? Henk This is the error message: The instance's SvcRun() method failed Traceback (most recent call last): File "C:\Python26\lib\site-packages\win32\lib\win32serviceutil.py", line 806, in SvcRun self.SvcDoRun() File "C:\Python26\lib\site-packages\zope2-2.12.5-py2.6-win-amd64.egg\nt_svcutils\service.py", line 115, in SvcDoRun self.stop(pid) File "C:\Python26\lib\site-packages\zope2-2.12.5-py2.6-win-amd64.egg\nt_svcutils\service.py", line 277, in stop open(self.getReg('pid_filename',keyname='PythonClass')).read() IOError: (2, 'No such file or directory', 'C:\\Users\\henk\\zope\\var\\Z2.pid') There is also a warning: Zope-2034764167 (Zope instance at C:\Users\henk\zope): Process terminated with exit code 1. Traceback (most recent call last): File "C:\Python26\Scripts\runzope-script.py", line 8, in <module> load_entry_point('Zope2==2.12.5', 'console_scripts', 'runzope')() File "C:\Python26\lib\site-packages\zope2-2.12.5-py2.6-win-amd64.egg\Zope2\Startup\run.py", line 21, in run starter.prepare() File "C:\Python26\lib\site-packages\zope2-2.12.5-py2.6-win-amd64.egg\Zope2\Startup\__init__.py", line 75, in prepare self.setupSecurityOptions() File "C:\Python26\lib\site-packages\zope2-2.12.5-py2.6-win-amd64.egg\Zope2\Startup\__init__.py", line 144, in setupSecurityOptions import AccessControl File "C:\Python26\lib\site-packages\zope2-2.12.5-py2.6-win-amd64.egg\AccessControl\__init__.py", line 16, in <module> from AccessControl.Implementation import setImplementation File "C:\Python26\lib\site-packages\zope2-2.12.5-py2.6-win-amd64.egg\AccessControl\Implementation.py", line 98, in <module> setImplementation("C") File "C:\Python26\lib\site-packages\zope2-2.12.5-py2.6-win-amd64.egg\AccessControl\Implementation.py", line 51, in setImplementation from AccessControl import ImplC as impl File "C:\Python26\lib\site-packages\zope2-2.12.5-py2.6-win-amd64.egg\AccessControl\ImplC.py", line 17, in <module> from AccessControl.cAccessControl import rolesForPermissionOn ImportError: DLL load failed: is not a valid Win32 application

1 0

cron like service
by Garry Saddington 05 May '10

05 May '10

Can anyone suggest the easiest way to automatically have a python script fire at set times of the day/month/year. Regards Garry

5 5

oddity in MailHost queue processor thread
by Maric Michaud 04 May '10

04 May '10

Hi all, I was surprised by the behavior of asynchronous mailing in zope 2.11.3 MailHost product. It creates a thread by instance of MailHost by the mean of using self.absolute_url(1) as key for the queue_threads dictionnary. It seems obviously wrong, IMO, if several instances share the same queue directory, mesages actually get sent more than once. Following is the monkey patch which works fine for me, but the idea is straightforward, it's to use the realpath of the smtp_queue_directory as the identifier of the thread, ensuring that MailHosts that share the same queue directory will also share the same thread for processing mails. regards, # un monkey patch du mailhost de zope2.11, ce dernier utilise une thread # par instance de mailhost pour la getsion de la queue de messages, ce n'est # pas correct, il faut qu'il garantisse qu'il n'y a qu'une thread par # répertoire réel de queue. from Products.MailHost import MailHost as mh if not hasattr(mh.MailBase, '_cfenet_patched_queueThreadAlive'): mh.MailBase._cfenet_patched_queueThreadAlive = mh.MailBase.queueThreadAlive def queueThreadAlive(self): """return True/False is queue thread is working""" from Products.MailHost import MailHost as mh from os.path import realpath # Il est important d'avoir un mailer unique par répertoire th = mh.queue_threads.get(realpath(self.smtp_queue_directory)) return th and th.isAlive() mh.MailBase.queueThreadAlive = queueThreadAlive if not hasattr(mh.MailBase, '_cfenet_patched__stopQueueProcessorThread'): mh.MailBase._cfenet_patched__stopQueueProcessorThread = \ mh.MailBase._stopQueueProcessorThread @mh.synchronized(mh.MailBase.lock) def _stopQueueProcessorThread(self): """ Stop thread for processing the mail queue """ from Products.MailHost import MailHost as mh from os.path import realpath path = realpath(self.smtp_queue_directory) if mh.queue_threads.has_key(path): th = mh.queue_threads[path] th.stop() while th.isAlive(): # wait until thread is really dead mh.time.sleep(0.3) del mh.queue_threads[path] mh.LOG.info('Thread for %s stopped' % path) mh.MailBase._stopQueueProcessorThread = _stopQueueProcessorThread if not hasattr(mh.MailBase, '_cfenet_patched__startQueueProcessorThread'): mh.MailBase._cfenet_patched__startQueueProcessorThread = \ mh.MailBase._startQueueProcessorThread @mh.synchronized(mh.MailBase.lock) def _startQueueProcessorThread(self): """ Start thread for processing the mail queue """ from Products.MailHost import MailHost as mh from os.path import realpath path = realpath(self.smtp_queue_directory) if not mh.queue_threads.has_key(path): th = mh.QueueProcessorThread() th.setMailer(self._makeMailer()) th.setQueuePath(self.smtp_queue_directory) th.start() mh.queue_threads[path] = th mh.LOG.info('Thread for %s started' % path) mh.MailBase._startQueueProcessorThread = _startQueueProcessorThread -- _____________ Maric Michaud

2 3