- Zope - Zope lists

PAS & scripted auth
by Sebastian Tänzer 02 Jun '15

02 Jun '15

Hello Tres (maintainer of PAS) and Zope community, I’m trying to get PAS replace CookieCrumpler to do authentication to Zope2 from a python script/Product or external method. Is there a how-to or tutorial somewhere how to get started on this? What I want is to authenticate a user through a web login form as Cookie Crumbler does (but I’d like to to it from my own script that does other things,too). I’ve - installed PAS - replaced the default acl_users in my folder - added a ZODB user folder - added a CookieAuthHelper Now, how do I authenticate the user to Zope if I have a username and password (i.e. from request.get(‚username‘) etc.) Best regards Sebastian

2 1

Zope loses data irretrievably upon restart
by Tom Cameron 27 May '15

27 May '15

We’ve been running Zope servers for about 15 years. In the last few years we have had a number of Zope servers instantly lose data upon restart. This has happened to us about 5 times over the past 2 years on 4 different servers. Upon restarting Zope we notice a rollback of data to sometimes months old and a Data.fs with a much older timestamp than expected. At this point the data was irretrievable. We find that it is not in any backups or anywhere on the file system. We have finally worked out what is happening but not exactly why. Below is a detailed explanation of what we are seeing. We are keen to know if anybody else is having similar issues or if perhaps we have setup something incorrectly and need to change how we pack the database. Also this may serve as a warning to others. *Explanation* The issue is that Zope is losing the file descriptor, the Data.fs still remains in the folder but Zope keeps writing to a (deleted) file descriptor which can be seen with lsof. Every server affected by this was running ZEO and had zeopack in the default configuration. We believe for some reason when zeopack tries to pack the ZODB, that when the Data.fs is moved and renamed to Data.fs.old and eventually deleted, Zope doesn’t recognize the new file. This is what we believe happens: 1. Zope is connected to a file Data.fs. 2. zeopack packs this file into a new file Data.fs.tmp 3. zeopack deletes any file named Data.fs.old 4. zeopack then renames Data.fs to Data.fs.old 5. zeopack then renames Data.fs.tmp to Data.fs The problem is that Zope is connected directly to the Data.fs file via its handler and even though it is renamed it continues to write to Data.fs.old. Later on, we do a subsequent zeopack and the Data.fs.old file is deleted in the process, but Zope keeps writing to this deleted file. We now have a situation where Zope is connected to a deleted file and is writing all its data to it. If Zope is ever stopped, the connection is lost and so effectively is the file. It is true that expert forensics could retrieve the file but in the first few instances that this happened we were totally baffled as to where the data had gone. In one case we had not restarted Zope for 4 months and thus we lost 4 months of data. *Environment* We are running standard Plone servers on Debian 5.0.10 in this case, with following: - Plone 4.3.4.1 (4309) - CMF 2.2.8 - Zope 2.13.22 - Python 2.7.8 (default, Jan 27 2015, 14:19:28) [GCC 4.3.2] - PIL 2.3.0 (Pillow) - ZEO: plone.recipe.zeoserver 1.2.6 Effective-user is set correctly for this instance “landcarevic” and properly mounts the Data.fs under normal circumstances. *Detailed Symptoms* This is a real case we discovered and managed to retrieve the lost file. Zope will continue to write to a Data.fs that has been technically deleted. We are able to notice this because the Data.fs.tmp will have a much more recent timestamp than the Data.fs in the var/filestorage directory. -rw-r--r-- 1 landcarevic psacln 6735952811 2015-04-12 00:11 Data.fs -rw-r--r-- 1 landcarevic psacln 18594749 2015-04-12 00:11 Data.fs.index -rw-r--r-- 1 landcarevic psacln 6 2015-03-30 11:20 Data.fs.lock -rw-r--r-- 1 landcarevic psaserv 14600344 2015-05-26 10:27 Data.fs.tmp An “lsof | grep Data.fs” can show if the file descriptor is pointing to a deleted file on the fs: python2.7 9307 landcarevic 7uW REG 0,31 6 4986845 /var/www/vhosts/ landcarevic.net.au/plone43/var/filestorage/Data.fs.lock python2.7 9307 landcarevic 11u REG 0,31 14600344 4986235 /var/www/vhosts/ landcarevic.net.au/plone43/var/filestorage/Data.fs.tmp python2.7 9307 landcarevic 12u REG 0,31 7058741434 4986349 (deleted)/var/www/vhosts/ landcarevic.net.au/plone43/var/filestorage/Data.fs.old The process id for the instance is 9307 and it’s still holding open a Data.fs.old that has been deleted. Most likely due to a ZEO pack before from the cron in this case. Because it packs the Data.fs and then moves the old one to Data.fs.old which eventually gets deleted. But Zope never properly recognises the new Data.fs for some reason. *Recovery* As long as the Zope server doesn’t get shut down or restarted, it will hold this open and continue to write to the disk. It can be recovered by copying from /proc where /proc/{process ID}/fd/{file descriptor ID} can be found from running lsof as previously: # cp /proc/9307/fd/12 /home/alyssa/Data.fs-current # ls -al /home/alyssa/ ... -rw-r--r-- 1 root root 7058741434 2015-05-26 11:11 Data.fs-current Which has the correct timestamp and file size. -- *Tom Cameron*

2 1

Iterating over objects in ZODB
by Rich Harley 21 May '15

21 May '15

When iterating over dtml objects, the contents of those scripts can be returned: e.g. for item in context.objectValues(): if item.meta_type=='DTML Method’: print item But when iterating over Python Scripts, only: <PythonScript at handleday> is returned. How can I access the actual contents of the Python Script? Thanks Rich

4 3

ZCatalog queries performance
by Sebastian Tänzer 19 May '15

19 May '15

Hello, I’m querying a ZCatalog with a lot of indexes and some meta data for event objects to get related events by tags. These need to be displayed with a title image (which itself is a sub object of the event object) and a pre-defined short text. My python query script gets the results, I then loop over them and get the object with unrestrictedTraverse(x.getPath(), None). I understood getObject is not really performant, but getting the objects with unrestrictedTraverse is not a lot faster. What would be the best way to query for these objects and then display the results with the image? Should I add the url of the image as meta data (title_image etc.)? What is the best way to display search results? Is getting the whole object via unrestrictedTraverse() a good idea? What are the alternatives? Best, Sebastian

2 4

Re: [Zope] Zope Digest, Vol 127, Issue 3
by Rich Harley 17 May '15

17 May '15

Thanks both. .body() is the simple answer, .read() also works. Rich ---- > > Message: 1 > Date: Wed, 13 May 2015 11:36:03 +0000 > From: Rich Harley <raharley0(a)gmail.com> > To: zope(a)zope.org > Subject: [Zope] Iterating over objects in ZODB > Message-ID: <86A3FE9B-5060-4D9A-B5BA-7F96AACD7361(a)gmail.com> > Content-Type: text/plain; charset=windows-1252 > > When iterating over dtml objects, the contents of those scripts can be returned: > > e.g. > > for item in context.objectValues(): > if item.meta_type=='DTML Method?: > print item > > > But when iterating over Python Scripts, only: > > <PythonScript at handleday> > > is returned. How can I access the actual contents of the Python Script? > > Thanks > Rich > > ------------------------------ > > Message: 2 > Date: Wed, 13 May 2015 13:57:30 +0200 > From: Niels Dettenbach <nd(a)syndicat.com> > To: zope(a)zope.org > Subject: Re: [Zope] Iterating over objects in ZODB > Message-ID: <2201822.Nhz8cIhMP1@gongo> > Content-Type: text/plain; charset="us-ascii" > > Am Mittwoch, 13. Mai 2015, 11:36:03 schrieb Rich Harley: >> is returned. How can I access the actual contents of the Python Script? > The python script object seems a bit more complex then a DTML-Method object - > means you have to access the script content of the python script object > directly (not shure if it's "params" and "body" - have to look into Zope help > (PythonScript.py) or Zope sources for the exact "field"). > > regarduing the object docs it shoudl work with .read() directly (if it IS a > PythonScript object) or "document_src": > > def read(): > """ > > Return the body of the Python Script, with a special comment > block prepended. This block contains meta-data in the form of > comment lines as expected by the 'write' method. > > """ > def document_src(REQUEST=None, RESPONSE=None): > """ > > Return the text of the 'read' method, with content type > 'text/plain' set on the RESPONSE. > > """ > > > hth a bit and works, > > > Niels. > -- > --- > Niels Dettenbach > Syndicat IT & Internet > http://www.syndicat.com > PGP: https://syndicat.com/pub_key.asc > --- > > > > >

1 0

ZEO manageUndo not working - DisconnectedError
by Sebastian Tänzer 06 May '15

06 May '15

Hello once more, I’m running a fresh Zope2 ZEO setup (pip install --index-url=http://download.zope.org/Zope2/index/2.13.22/ Zope2 and pip install zope.mkzeoinstance) inside a virtualenv setup. As said: No products, fresh setup, happens with any object. Going to any undo tab produces this error: Error Type: DisconnectedError Client log: 2015-05-06 01:16:46 INFO ZEO.ClientStorage zeostorage Disconnected from storage: "('localhost', 9999)" 2015-05-06 01:16:46 ERROR Zope.SiteErrorLog 1430867806.070.275579780745 http://192.168.10.38:8080/temp_folder/manage_UndoForm Traceback (innermost last): Module ZPublisher.Publish, line 138, in publish Module ZPublisher.mapply, line 77, in mapply Module ZPublisher.Publish, line 48, in call_object Module Shared.DC.Scripts.Bindings, line 322, in __call__ Module Shared.DC.Scripts.Bindings, line 359, in _bindAndExec Module App.special_dtml, line 185, in _exec Module App.Undo, line 112, in undoable_transactions Module ZODB.DB, line 901, in undoInfo Module ZEO.ClientStorage, line 1283, in undoInfo Module ZEO.ServerStub, line 285, in undoInfo Module ZEO.zrpc.connection, line 776, in call Module ZEO.zrpc.connection, line 796, in wait DisconnectedError 2015-05-06 01:16:46 WARNING ZEO.zrpc (19205) CW: error connecting to ('::1', 9999): ECONNREFUSED 2015-05-06 01:16:46 INFO ZEO.ClientStorage zeostorage Testing connection <ManagedClientConnection ('127.0.0.1', 9999)> 2015-05-06 01:16:46 INFO ZEO.zrpc.Connection('C') (127.0.0.1:9999) received handshake ‚Z3101' ZEO server log: 2015-05-06T01:15:13 INFO ZEO.zrpc.Connection('S') (127.0.0.1:47063) received handshake 'Z3101' ------ 2015-05-06T01:16:46 ERROR ZEO.zrpc (19082) can't decode message: '(K\x9dI00\nU\x08undoInfo(K\x00K\x14}U\x0bdescriptioncco...' ------ 2015-05-06T01:16:46 ERROR ZEO.zrpc.Connection('S') (127.0.0.1:47063) Error caught in asyncore Traceback (most recent call last): File "/opt/python/2.7.9/lib/python2.7/asyncore.py", line 83, in read obj.handle_read_event() File "/opt/python/2.7.9/lib/python2.7/asyncore.py", line 449, in handle_read_event self.handle_read() File "/opt/python/zope2-latest/lib/python2.7/site-packages/ZEO/zrpc/smac.py", line 240, in handle_read self.message_input(msg) File "/opt/python/zope2-latest/lib/python2.7/site-packages/ZEO/zrpc/connection.py", line 417, in message_input msgid, async, name, args = self.decode(message) File "/opt/python/zope2-latest/lib/python2.7/site-packages/ZEO/zrpc/marshal.py", line 80, in server_decode return unpickler.load() # msgid, flags, name, args File "/opt/python/zope2-latest/lib/python2.7/site-packages/ZEO/zrpc/marshal.py", line 120, in server_find_global raise ZRPCError("import error %s: %s" % (module, msg)) ZRPCError: import error copy_reg: ------ 2015-05-06T01:16:46 INFO ZEO.StorageServer (127.0.0.1:47063) disconnected The path difference in /opt/python/2.7.9/lib/python2.7/asyncore.py vs /opt/python/zope2-latest/lib/python2.7/site-packages make me wonder if this is a bug or a problem with my setup? Client + ZEO server were created from the same virtualenv. Python 2.7.9 build from source. Any ideas? Best, Sebastian

3 6

manage_renameObjects with remote ZEO
by Sebastian Tänzer 05 May '15

05 May '15

Hello Zope community, does anyone know how to rename a OFS.Folder object from a remote python script (i.e. command line)? I’m trying this via ZEO.ClientStorage, but it fails with "The object <em>renametest1</em> does not support this operation.“ ,triggered by OFS/CopySupport.py", line 537, in _verifyObjectPaste. Full test case code is at http://pastebin.com/j9HY3PVJ At first I thought this is because of a missing REQUEST and/or authentication problem, as _verifyObjectPaste does some security checks or else fails with this message. Still, using makerequest and newSecurityManager doesn’t work. Any help would be greatly appreciated :-) Best Sebastian - tänzermedien GmbH - Agentur für Online-Medien Kekuléstr. 39 | 53115 Bonn, Germany E-Mail: st(a)taenzer.me | Web: http://www.taenzer.me Tel.: 0228-304134-0 | Fax.: 0228-304134-99

2 2

Using Template in script (python)
by Miguel Beltran R. 26 Apr '15

26 Apr '15

Hi list, I'm trying to use the make a string using a Template and fill it with the result of a query, but gime this error You are not allowed to access 'safe_substitute' in this context Zope = 2.12 from string import Template rst=someSQL()[0] txt=Template(""" <span id="datos_vehiculo" data-placa="${placa}" data-serie="${serie}" data-marca="${marca}" data-submarca="${submarca}" data-modelo="${modelo}"> </span> """) newtxt=txt.safe_substitute(rst) do exist some way to do it? -- ________________________________________ Lo bueno de vivir un dia mas es saber que nos queda un dia menos de vida

2 2

ANN: eGenix mxODBC Plone/Zope Database Adapter 2.2.1
by eGenix Team: M.-A. Lemburg 20 Apr '15

20 Apr '15

________________________________________________________________________ ANNOUNCING mxODBC Plone/Zope Database Adapter Version 2.2.1 for the Plone CMS and Zope server platform Available for Plone 4.0-4.3 and Plone 5.0, Zope 2.12 and 2.13, on Windows, Linux, Mac OS X, FreeBSD and other platforms This announcement is also available on our web-site for online reading: http://www.egenix.com/company/news/eGenix-mxODBC-Zope-DA-2.2.1-GA.html ________________________________________________________________________ INTRODUCTION The eGenix mxODBC Zope DA allows you to easily connect your Zope or Plone CMS installation to just about any database backend on the market today, giving you the reliability of the commercially supported eGenix product mxODBC and the flexibility of the ODBC standard as middle-tier architecture. The mxODBC Zope Database Adapter is highly portable, just like Zope itself and provides a high performance interface to all your ODBC data sources, using a single well-supported interface on Windows, Linux, Mac OS X, FreeBSD and other platforms. This makes it ideal for deployment in ZEO Clusters and Zope hosting environments where stability and high performance are a top priority, establishing an excellent basis and scalable solution for your Plone CMS. Product page: http://www.egenix.com/products/zope/mxODBCZopeDA/ ________________________________________________________________________ NEWS The 2.2.1 release of our mxODBC Zope/Plone Database Adapter product is a patch level release of the popular ODBC database interface for Plone and Zope. It includes these enhancements and fixes: Feature Updates --------------- * Fully compatible with *Zope External Methods* and Zope's built-in transaction mechanism: use Connection Objects in Python modules set up as Zope External Methods to all the features of the mxODBC Zope DA from Python and even tap into the low-level features of mxODBC within Zope's transaction mechanism. * Full support of *Stored Procedures*: when using Zope External Methods, you have full access to SQL stored procedures, which allows direct integration with external database APIs. Driver Compatibility Enhancements --------------------------------- * ODBC driver compatibility updated. Upgraded to the latest mxODBC 3.3.2 release, adding compatibility enhancements for MS SQL Server, SAP Sybase ASE and stored procedure output parameters. See the mxODBC 3.3.0 and 3.3.2 release announcements for full details. http://www.egenix.com/company/news/eGenix-mxODBC-3.3.2-GA.html Fixes ----- * Documented use of allow_hosts in buildout.cfg The complete list of changes is available on the mxODBC Zope DA changelog page. http://www.egenix.com/products/zope/mxODBCZopeDA/changelog.html mxODBC Zope DA 2.2.0 was released on 2014-12-11. Please see the mxODBC Zope DA 2.2.0 release announcement for all the new features we have added. http://www.egenix.com/company/news/eGenix-mxODBC-Zope-DA-2.2.0-GA.html For the full list of features, please see the mxODBC Zope DA feature list: http://www.egenix.com/products/zope/mxODBCZopeDA/#Features The complete list of changes is available on the mxODBC Zope DA changelog page. ________________________________________________________________________ UPGRADING Users are encouraged to upgrade to this latest mxODBC Plone/Zope Database Adapter release to benefit from the new features and updated ODBC driver support. We have taken special care not to introduce backwards incompatible changes, making the upgrade experience as smooth as possible. For major and minor upgrade purchases, we will give out 20% discount coupons going from mxODBC Zope DA 1.x to 2.2 and 50% coupons for upgrades from mxODBC 2.x to 2.2. After upgrade, use of the original license from which you upgraded is no longer permitted. Patch level upgrades (e.g. 2.2.0 to 2.2.1) are always free of charge. Please contact the eGenix.com Sales Team with your existing license serials for details for an upgrade discount coupon. If you want to try the new release before purchase, you can request 30-day evaluation licenses by visiting our web-site or writing to sales(a)egenix.com, stating your name (or the name of the company) and the number of eval licenses that you need. http://www.egenix.com/products/python/mxODBCZopeDA/#Evaluation ________________________________________________________________________ DOWNLOADS Please visit the eGenix mxODBC Zope DA product page for downloads, instructions on installation and documentation of the packages: http://www.egenix.com/company/products/zope/mxODBCZopeDA/ If you want to try the package, please jump straight to the download instructions: http://www.egenix.com/products/zope/mxODBCZopeDA/#Download Fully functional evaluation licenses for the mxODBC Zope DA are available free of charge: http://www.egenix.com/products/zope/mxODBCZopeDA/#Evaluation ________________________________________________________________________ SUPPORT Commercial support for this product is available directly from eGenix.com. Please see the support section of our website for details: http://www.egenix.com/services/support/ ________________________________________________________________________ MORE INFORMATION For more information on eGenix mxODBC Zope DA, licensing and download instructions, please write to sales(a)egenix.com. About eGenix (http://www.egenix.com/) eGenix is a software project, consulting and product company focusing on expert project services and professional quality products for companies, Python users and developers. Enjoy, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Apr 20 2015) >>> Python Projects, Coaching and Consulting ... http://www.egenix.com/ >>> mxODBC Plone/Zope Database Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::::: Try our mxODBC.Connect Python Database Interface for free ! :::::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/

1 0

Sessions & DTML Execution
by Justin Dunsworth 08 Apr '15

08 Apr '15

Hello, I am running into an issue that I can't quite figure out. I'm trying to leverage storing data within sessions then setting a variable in the header of my page to the session variable (for ease of calling) where I can then use it throughout the rest of my document. Here's the way it goes: <dtml-if "REQUEST.SESSION.get('testVarDict')"> <dtml-call "REQUEST.set('testVarDict',REQUEST.SESSION.get('vx9member'))"> <dtml-else> <dtml-call "RESPONSE.redirect(BASE1+'/login')"> </dtml-if> <dtml-var testVarDict> Here I am wanting to redirect to login should testVarDict not be set. Instead it seems like Zope is still wanting to execute the rest of the page prior to redirecting so this will throw a NameError for testVarDict should the session data not be set. I then found out that I can change the dtml-call to a dtml-return for the redirect and it will stop executing the dtml and redirect. This is all fine but if I have another page I'm going to call that uses the header like so: <dtml-var headerDoc> <dtml-var testVarDict> <dtml-var footerDoc> This then will throw the same NameError regardless of the dtml-return redirect in the header. I can get around this by placing the same checks in the top of every.single.page that I want to utilize testVarDict variable but that's too cumbersome. I then tried to make a script that would check the same thing and then redirect but it doesn't seem to work at all - for fun here's the python script: if context.REQUEST.SESSION.get('testVarDict'): context.REQUEST.set('vx9member',context.REQUEST.SESSION.get('testVarDict')) else: return context.REQUEST.RESPONSE.redirect('/login') I've also tried just the context.REQUEST.RESPONSE.redirect without the return but it doesn't seem to work and I suspect it's because python return is not the same as dtml-return. If anyone could assist or show me a different way then I would greatly appreciate it. Thanks, J

1 0