- Zope - Zope lists

oddity in MailHost queue processor thread
by Maric Michaud 04 May '10

04 May '10

Hi all, I was surprised by the behavior of asynchronous mailing in zope 2.11.3 MailHost product. It creates a thread by instance of MailHost by the mean of using self.absolute_url(1) as key for the queue_threads dictionnary. It seems obviously wrong, IMO, if several instances share the same queue directory, mesages actually get sent more than once. Following is the monkey patch which works fine for me, but the idea is straightforward, it's to use the realpath of the smtp_queue_directory as the identifier of the thread, ensuring that MailHosts that share the same queue directory will also share the same thread for processing mails. regards, # un monkey patch du mailhost de zope2.11, ce dernier utilise une thread # par instance de mailhost pour la getsion de la queue de messages, ce n'est # pas correct, il faut qu'il garantisse qu'il n'y a qu'une thread par # répertoire réel de queue. from Products.MailHost import MailHost as mh if not hasattr(mh.MailBase, '_cfenet_patched_queueThreadAlive'): mh.MailBase._cfenet_patched_queueThreadAlive = mh.MailBase.queueThreadAlive def queueThreadAlive(self): """return True/False is queue thread is working""" from Products.MailHost import MailHost as mh from os.path import realpath # Il est important d'avoir un mailer unique par répertoire th = mh.queue_threads.get(realpath(self.smtp_queue_directory)) return th and th.isAlive() mh.MailBase.queueThreadAlive = queueThreadAlive if not hasattr(mh.MailBase, '_cfenet_patched__stopQueueProcessorThread'): mh.MailBase._cfenet_patched__stopQueueProcessorThread = \ mh.MailBase._stopQueueProcessorThread @mh.synchronized(mh.MailBase.lock) def _stopQueueProcessorThread(self): """ Stop thread for processing the mail queue """ from Products.MailHost import MailHost as mh from os.path import realpath path = realpath(self.smtp_queue_directory) if mh.queue_threads.has_key(path): th = mh.queue_threads[path] th.stop() while th.isAlive(): # wait until thread is really dead mh.time.sleep(0.3) del mh.queue_threads[path] mh.LOG.info('Thread for %s stopped' % path) mh.MailBase._stopQueueProcessorThread = _stopQueueProcessorThread if not hasattr(mh.MailBase, '_cfenet_patched__startQueueProcessorThread'): mh.MailBase._cfenet_patched__startQueueProcessorThread = \ mh.MailBase._startQueueProcessorThread @mh.synchronized(mh.MailBase.lock) def _startQueueProcessorThread(self): """ Start thread for processing the mail queue """ from Products.MailHost import MailHost as mh from os.path import realpath path = realpath(self.smtp_queue_directory) if not mh.queue_threads.has_key(path): th = mh.QueueProcessorThread() th.setMailer(self._makeMailer()) th.setQueuePath(self.smtp_queue_directory) th.start() mh.queue_threads[path] = th mh.LOG.info('Thread for %s started' % path) mh.MailBase._startQueueProcessorThread = _startQueueProcessorThread -- _____________ Maric Michaud

2 3

Data.fs data record exceeds transaction record
by Michele Marcionelli 02 May '10

02 May '10

Hello 2 hours ago I wanted to go to sleep... and "unfortunately" I noticed that our websites were offline since about 5 hours; I coudn't start zope anymore because the Data.fs seems to be corrupted. The quick solution has been to copy a 2 days old version of the Data.fs back... but I still hope to recover the old Data.fs. But wenn I try to recover it with the fsrecover.py tool I get this error message: 2010-05-02 00:38:39 WARNING OFS.Application Duplicate Product name After loading Product 'Five' from '/scratch/ZOPE-BOT/current/Products', I skipped the one in '/scratch/local/app/zope/2.8.9.1-2.3.6/lib/python/ Products'. 2010-05-02 00:38:39 WARNING Init Class Products.LDAPUserFolder.LDAPUserFolder.LDAPUserFolder has a security declaration for nonexistent method 'getAttributeOfAllObjects' 2010-05-02 00:38:39 WARNING Init Class Products.LDAPUserFolder.LDAPUserFolder.LDAPUserFolder has a security declaration for nonexistent method 'getAttributeOfAllObjects' 2010-05-02 00:38:39 WARNING Silva Silva Documents require upgrading. There are Silva Documents which have not been upgraded to Silva 0.9.3. Upgrade via service_extensions. 2010-05-02 00:38:40 WARNING ZODB.FileStorage Ignoring index for / scratch/ZOPE-BOT/current/var/Data.fs 2010-05-02 00:38:51 CRITICAL ZODB.FileStorage /scratch/ZOPE-BOT/ current/var/Data.fs data record exceeds transaction record at 949568131 Traceback (most recent call last): File "<string>", line 1, in ? File "/scratch/local/app/zope/2.8.9.1-2.3.6/lib/python/Zope2/ __init__.py", line 51, in app startup() File "/scratch/local/app/zope/2.8.9.1-2.3.6/lib/python/Zope2/ __init__.py", line 47, in startup _startup() File "/scratch/local/app/zope/2.8.9.1-2.3.6/lib/python/Zope2/App/ startup.py", line 57, in startup DB = configuration.dbtab.getDatabase('/', is_root=1) File "/scratch/local/app/zope/2.8.9.1-2.3.6/lib/python/DBTab/ DBTab.py", line 96, in getDatabase db = self._createDatabase(name, is_root) File "/scratch/local/app/zope/2.8.9.1-2.3.6/lib/python/DBTab/ DBTab.py", line 113, in _createDatabase db = factory.open() File "/scratch/local/app/zope/2.8.9.1-2.3.6/lib/python/Zope2/ Startup/datatypes.py", line 163, in open DB = self.createDB() File "/scratch/local/app/zope/2.8.9.1-2.3.6/lib/python/Zope2/ Startup/datatypes.py", line 160, in createDB return ZODBDatabase.open(self) File "/scratch/local/app/zope/2.8.9.1-2.3.6/lib/python/ZODB/ config.py", line 97, in open storage = section.storage.open() File "/scratch/local/app/zope/2.8.9.1-2.3.6/lib/python/ZODB/ config.py", line 133, in open quota=self.config.quota) File "/scratch/local/app/zope/2.8.9.1-2.3.6/lib/python/ZODB/ FileStorage/FileStorage.py", line 169, in __init__ read_only=read_only, File "/scratch/local/app/zope/2.8.9.1-2.3.6/lib/python/ZODB/ FileStorage/FileStorage.py", line 1731, in read_index name, pos) File "/scratch/local/app/zope/2.8.9.1-2.3.6/lib/python/ZODB/ FileStorage/FileStorage.py", line 52, in panic raise CorruptedTransactionError(message) ZODB.FileStorage.FileStorage.CorruptedTransactionError: %s data record exceeds transaction record at %s Thanks, Michele -- ETH Zürich Michele Marcionelli IT Support Gruppe D-MATH HG G 32.1 Rämistrasse 101 CH–8092 Zürich michele.marcionelli(a)math.ethz.ch http://www.math.ethz.ch/~michele +41 44 632 6193 Telefon +41 44 632 1354 Fax

2 1

Zope 2.12 and ZPT
by Peter Bengtsson 01 May '10

01 May '10

I'm trying to get a product running in Zope 2.12 which I installed in a virtualenv with easy_install. It starts fine and all but when I try to render a piece of ZPT I get this weird error: Traceback (innermost last): Module ZPublisher.Publish, line 127, in publish Module ZPublisher.mapply, line 77, in mapply Module ZPublisher.Publish, line 47, in call_object Module Products.IssueTrackerProduct.IssueTracker, line 12629, in AddIssue Module Shared.DC.Scripts.Bindings, line 324, in __call__ Module Shared.DC.Scripts.Bindings, line 361, in _bindAndExec Module Products.PageTemplates.PageTemplateFile, line 130, in _exec Module Products.PageTemplates.PageTemplate, line 80, in pt_render Module zope.pagetemplate.pagetemplate, line 115, in pt_render Module zope.tal.talinterpreter, line 271, in __call__ Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 888, in do_useMacro Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 533, in do_optTag_tal Module zope.tal.talinterpreter, line 518, in do_optTag Module zope.tal.talinterpreter, line 513, in no_tag Module zope.tal.talinterpreter, line 343, in interpret Module zope.tal.talinterpreter, line 742, in do_insertStructure_tal Module Products.PageTemplates.Expressions, line 220, in evaluateStructure Module zope.tales.tales, line 696, in evaluate Module Products.PageTemplates.ZRPythonExpr, line 49, in __call__ - __traceback_info__: here.getRoot().title_or_id().replace(' ',' ') Module PythonExpr, line 1, in <expression> UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 0: ordinal not in range(128) After a lot of trial-and-error I managed to figure out that the culprit was this piece of ZPT code: tal:content="structure python:here.getRoot().title_or_id().replace(' ',' ')" Stupid and easy to fix; maybe but what if there are other problems like this? What's the cause of this? Do I have to set something special up in zope.conf just for Page Templates to not convert 'nbsp;' into '\xa0\xc2'? -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com fun crosstips.org

3 7

problems with emergency user
by Nuno Carvalho 30 Apr '10

30 Apr '10

Hi all, I'm using Zope (version 2.10.11) through Plone. I lost my admin password to access ZMI. I created a file named "access" using the zpassword.py script, as described here: http://docs.zope.org/zope2/zope2book/Security.html?highlight=password#creat… I tried to place this file in several directories, such as: /usr/local/Plone/Zope-2.10.11-final-py2.4 /usr/local/Plone/zeocluster /usr/local/Plone/zeocluster/var /usr/local/Plone/zeocluster/var/filestorage (this last one is where Data.fs is) among others. I'm not able to login through http://localhost:8080/manage I need some help! Can anyone give me a hint on how can I solve this? Thanks in advance! Nuno Carvalho INESC-ID/IST http://www.gsd.inesc-id.pt/~nonius/

2 1

Older Zope w/newer CA packages
by Jim Pharis 28 Apr '10

28 Apr '10

We are currently on Zope 2.9.6 and we are working on upgrading but it isn't going to happen anytime soon. I'm trying to use a package that is depending on a few newer component architecture packages; zope.component, zope.interface... Even though zope.component and zope.interface hadn't been egged in 2.9.6, Zope Products seem to import from zope.component in Python lib instead of zope/component in the Zope. Is there any clean way to isolate the newer packages? If its possible this might actually help with a gradual upgrade as well. Any recommendations or techniques would be most useful. TIA, - Jim

2 1

problem with commits in SQLAlchemyDA
by Maric Michaud 27 Apr '10

27 Apr '10

Hi all, I wanted to use SQALchemyDA with a standalone Zope 2.11 (we only depends on CMFCore). i tried to get it up using easy_install, which is pretty straightforward, once removed the incompatible zope.component package needed in the dependencies. It ends up with the following installed packages (skipping zope 3 components) : zope.sqlalchemy-0.4-py2.4.egg z3c.sqlalchemy-1.3.10.1-py2.4.egg SQLAlchemy-0.5.5-py2.4.egg Products.SQLAlchemyDA-0.4.1-py2.4.egg At first glance, this works well, but the DA actually can't manage to commit any sql requests (I try only for insert in Mysql and postgres but I'm sure it's always true). This is quite an unexpectable behavior for a DA. I think the problem is related to this thread I found on tg-trunk newsgroup : http://www.mail-archive.com/turbogears-trunk@googlegroups.com/msg07302.html There is a misleading comment in the docstring of zope.sqlalchemy.datamanager.join_transaction speaking of a DirtyAfterFlush SessionExtension, which I never seen except in plone code. In the last I finally manage to make it work by monkey patching SQLAlchemyDA itself this way : try : from Products.SQLAlchemyDA import da from z3c.sqlalchemy import getSAWrapper, createSAWrapper if '0.4.1' in da.__file__ and not hasattr(da.SAWrapper, '_patched__wrapper_property') : da.SAWrapper._patched__wrapper_property = da.SAWrapper._wrapper def _always_invalidated_wrapper(self): """The property '_wrapper' patched by CFENet to correct what seems to be a bug in SQLAlchemyDA which prevents any commit.""" from zope.sqlalchemy.datamanager import STATUS_INVALIDATED if self.dsn: try: return getSAWrapper(self.util_id) except ValueError: return createSAWrapper( self.dsn, forZope=True, transactional=self.transactional, engine_options={'convert_unicode' : self.convert_unicode, 'encoding' : self.encoding}, extension_options={'initial_state': # the whole # point is here STATUS_INVALIDATED}, name=self.util_id) return None da.SAWrapper._wrapper = property(_always_invalidated_wrapper) except ImportError : pass Anyone has any insights about this problem, any comments on my solution, did it have been reported and corrected, or am I simply missing something ? -- _____________ Maric Michaud

5 21

identity values from sql server , mxodbc da
by Josh Burvill 24 Apr '10

24 Apr '10

Hi I am using zope 2.8.11 I am using mxodbc da to connect to sql server and I was wondering what is the best way to get the new value of the primary key (an identity field) after inserting. It looks like there are 3 slightly different functions I could use: """"""""""" SCOPE_IDENTITY, IDENT_CURRENT, and @@IDENTITY are similar functions because they return values that are inserted into identity columns. IDENT_CURRENT is not limited by scope and session; it is limited to a specified table. IDENT_CURRENT returns the value generated for a specific table in any session and any scope. For more information, see IDENT_CURRENT (Transact-SQL) <http://msdn.microsoft.com/en-us/library/ms175098.aspx>. SCOPE_IDENTITY and @@IDENTITY return the last identity values that are generated in any table in the current session. However, SCOPE_IDENTITY returns values inserted only within the current scope; @@IDENTITY is not limited to a specific scope. """""" (from http://msdn.microsoft.com/en-us/library/ms190315.aspx) It seems like scope_identity might be the best one to use, but is it possible that two zope transactions might occur within the one sql server "scope" and "session" and therefore get the wrong identity value depending on timing etc. Thanks in advance

2 1

Re: [Zope] javascript problem
by Jaroslav Lukesh 23 Apr '10

23 Apr '10

----- Original Message ----- From: "Jaroslav Lukesh" <lukesh(a)seznam.cz> > I think that you have bad code - see your error message. So try somethong > like: > document.forms.playlist.phones.number[1] > instead of > document.forms.playlist.phones.number:records.1.value Oh, sorry, javascript, not python :o) JL

2 1

javascript problem
by Garry Saddington 23 Apr '10

23 Apr '10

I'm using the following javascript to gather phones:number values for form submission. for (var idx = 1; idx < max; idx++) { if (eval("document.forms.playlist['phones.number:records'][" + idx + "].value") > 0) Has anyone got any idea why it works fine in every browser (Safari,FF,Chrome,Webkit) but not in ie8 where I get this error: Message: 'document.forms.playlist.phones.number:records.1.value' is null or not an object Any help appreciated, but I am not surprised that ie errors, it usually does. Garry

2 3

ANN: eGenix mxODBC Zope Database Adapter 2.0.1
by eGenix Team: M.-A. Lemburg 23 Apr '10

23 Apr '10

________________________________________________________________________ ANNOUNCEMENT mxODBC Zope Database Adapter Version 2.0.1 for Zope and the Plone CMS Available for Zope 2.12 and later on Windows, Linux, Mac OS X, FreeBSD and other platforms This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-mxODBC-Zope-DA-2.0.1-GA.html ________________________________________________________________________ INTRODUCTION The eGenix mxODBC Zope Database Adapter allows you to easily connect your Zope or Plone installation to just about any database backend on the market today, giving you the reliability of the commercially supported eGenix product mxODBC and the flexibility of the ODBC standard as middle-tier architecture. The mxODBC Zope Database Adapter is highly portable, just like Zope itself and provides a high performance interface to all your ODBC data sources, using a single well-supported interface on Windows, Linux, Mac OS X, FreeBSD and other platforms. This makes it ideal for deployment in ZEO Clusters and Zope hosting environments where stability and high performance are a top priority, establishing an excellent basis and scalable solution for your Plone CMS. ________________________________________________________________________ NEWS We are pleased to announce a new version 2.0.1 of our mxODBC Zope DA product. With the patch level 2.0.1 release we have backported the mxODBC Zope DA product to also run on Zope 2.10, 2.11 as well as Python 2.4. This was done to meet popular demand, since Plone 3 still uses these Zope and Python versions. The mxODBC Zope DA is now fully compatible with all recent Plone versions, including the upcoming Plone 4.0 release. Please see the mxODBC Zope DA documentation for details on how to install the product for use in Plone. Compared to the previous 1.0 version of the mxODBC Zope DA, the 2.0 version includes these enhancements: * Includes mxODBC 3.1 with updated support for many current ODBC drivers, giving you more portability and features for a wider range of database backends. * Mac OS X 10.6 (Snow Leopard) support. * Python 2.4, 2.5 and 2.6 support. * Zope 2.10, 2.11 and 2.12 support. * Zero maintenance support to automatically reconnect the Zope connection after a network or database problem. * More flexible Unicode support with options to work with pure Unicode, plain strings or mixed setups - even for databases that don't support Unicode * Automatic and transparent text encoding and decoding * More flexible date/time support including options to work with Python datetime objects, mxDateTime, strings or tuples * New decimal support to have the Zope DA return decimal column values using Python's decimal objects. * Fully eggified to simplify easy_install and zc.buildout based installation ________________________________________________________________________ UPGRADING Licenses purchased for version 2.0.0 of the mxODBC Zope DA will continue to work with the 2.0.1 patch level release. If you have already bought mxODBC Zope DA 1.0.x licenses, we offer you a time limited upgrade option: * If you have purchased the licenses between 2009-06-01 and 2009-12-31 you can get a 20% discount on the full price of version 2.0. * If you have purchased the licenses after 2010-01-01 you can get a 40% discount on the full price of version 2.0. This offer is time limited until 2010-09-30. Please write to sales(a)egenix.com for details on how to get the needed discount coupons. ________________________________________________________________________ MORE INFORMATION For more information on the mxODBC Zope Database Adapter, licensing and download instructions, please visit our web-site: http://www.egenix.com/products/zope/mxODBCZopeDA/ You can buy mxODBC Zope DA licenses online from the eGenix.com shop at: http://shop.egenix.com/ ________________________________________________________________________ Thank you, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 23 2010) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/

1 0