- Zope - Zope lists

ZODB data loss - help required
by Tom Cameron 05 Dec '13

05 Dec '13

Ive just had a very disturbing and odd experience with data loss in the ZODB. Ive been running Zope servers for 15 years and never seen this before until two previous incidents a few months ago. The sequence of events for those incidents were vague so I put it down to user error, but now that its happened a third time on a different site Im getting worried. Plone 4.2.0.1 (4206) CMF 2.2.6 Zope 2.13.15 Python 2.7.3 (default, Feb 6 2013, 01:00:51) [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] PIL 2.0.0 (Pillow) The site was been working perfectly for 4 months with many updates and perfectly fine performance. Im now faced with the horrible job of explaining this to the client and I dont quite understand it myself. *Sequence of events* The Plone site was launched to production in September and has been running ever since. It has been restarted several times since then with minor product updates. Today I needed to do a yum update and then I restarted Zope using an init script that stops Zope & ZEO and restarts them. When Zope came up the Plone site was now showing data from August 5. Somehow everything between August 5 and December 5 has been lost. A scan of the transactions via the Undo Log or using data.fs tools shows transactions up to August 5 then nothing until December 5 >From the logs I can see that the server itself was rebooted on August 5 and there was also a copy of the Data.fs made on that date too. We have not packed the database since that date as it was not large or growing fast. All backups also show the August 5 data - which means the file system copies of the Data.fs were simply copying the old version of the database. Ive cloned the file system and kept the original untouched although in the first hour or so after the restart I did several file copies and moves as I was unaware of the gravity of the situation. Ive tried extundelete and debugfs and neither of them detect any deleted Data.fs file that can be recovered. *Questions* My main aim is to work out what happened. My best guess is that Zope was somehow connecting to a stale or outdated file pointer and updating that file all along while the Data.fs was pointing to an August 5 copy. But how could this situation eventuate and persist for so long? The odd thing is that we had 2 very similar incidents on 2 different Zope servers a few months ago but both resulted in almost no data loss as the timeframes were shorter and I dismissed them as some odd user error. We have recently moved most of our Zope servers to Linode - could it be their file system? or could it be the new way we setup the buildouts and init scripts? Any clues at all would be welcomed. -- *Tom Cameron* Technical Director *Mooball IT*

4 3

Re: [Zope] Python 3 support in any Zope Toolkit version or fork
by Sebastien Douche 04 Dec '13

04 Dec '13

On Tue, Nov 12, 2013 at 10:52 PM, Stephan Richter <stephan.richter(a)gmail.com> wrote: > Well, we replaced Paste by writing our own startup script, which was trivial. > Note that PasteScript is ported. See zope.paste. I tried zope.paste. Works great but how configure loggers (previously in zope.conf)? -- Sebastien Douche <sdouche(a)gmail.com> Twitter: @sdouche / G+: +sdouche

1 0

Product with external relationships coordinated with the transaction manager
by Jesus Cea 18 Nov '13

18 Nov '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, there. I am considering adopting "ExtFile/ExtImage" product and bring it to zope 2.13. The sourcecode is easy enough, although I suspect tons of "weirdness" is derived by legacy. A complete rewrite is an option, I should not maintain code I don't understand. Zope documentation is really incomplete and very old so I wonder about this: 1. What interfaces (I only care about Zope 2.13, no legacy) should I look at to participate in transactions?. I need to know if a transaction I was involved in was aborted (to rollback external changes too) or committed (to activate cascading actions previously "queued" during the transaction progress). Old ExtFile produc uses TM (transaction manager) from ZSQLmethods, but it is not in core Zope, I need to install an extra package, and I guess there is a more "official" and supported way. 2. How to participate in copy&paste?. I see stuff like "event" listening, but I can not find any documentation about them, beside just reading the source code. Any pointer and suggestion are welcomed :-). Thanks! - -- Jesús Cea Avión _/_/ _/_/_/ _/_/_/ jcea(a)jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ Twitter: @jcea _/_/ _/_/ _/_/_/_/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQCVAwUBUoqaHZlgi5GaxT1NAQI20QQAkAhLd1wck8szGLTyd18yF8IQQWrg+12u RRdYHXhtr6fbtsXIV+80x/7NpGsYTczhAJDbH1aIFlmhCbz9Sd6TdY0T2y+71Yt0 XFavL5UihcJiS4RB1x3wTImPSHyw3ZhkVewOpS4pUOqrS3HUipv4nexvUmLaMwQf A0zkqh++3AA= =LW1m -----END PGP SIGNATURE-----

1 0

Refresh Products without restart
by Gerhard Schmidt 18 Nov '13

18 Nov '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I'm starting to port some of our Products form Zope2.11 to Zope2.13. Why is the Refresh Tab gone form the Product Object. A quick look in the Source revealed that the refresh tab is still there only the manage_refresh method is gone. Is there another way to refresh single Products without restarting the whole server. Regards Estartu -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (FreeBSD) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQCVAwUBUonD+wzx22nOTJQRAQKNfgQAjgCWRnfgu5N4N3RovNucOMOHVrfGeKLM tPTDlxZWPnIBIpUN82XBOKM28p5Sg8rCDgPspfNbCbG8MqNFmeaH+KwwIohlS/X4 35nHo+XEi5vrJnBNO9jX5gXp8EwQ575HUD8KYKsNCyXgSLS0K1YKx084C0JjXCPJ sZcZcfx5AHk= =Kfj9 -----END PGP SIGNATURE-----

1 0

ExtFile/ExtImage in Zope2 2.13 and Transaction Manager
by Jesus Cea 16 Nov '13

16 Nov '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I did some tests a year ago about upgrading to 2.13.19. I use ExtFile/ExtImage extensively and I had to patch it a bit. I hit the problem that ExtFile tries to import "from Shared.DC.ZRDB.TM import TM". Messing around a bit, I solved the issue installing "Products.ZSQLMethods" product (via <https://mail.zope.org/pipermail/zope/2012-January/176324.html>), but this is not working with 2.13.21. I am getting tons of upgrade warnings about zope packages. The documentation for "Products/ExtFile/TM.py" says this: """ TMRegistry and ProxyTM Use case - -------- Persistent objects (ExtFiles) need to participate in ZODB transactions. ExtFiles perform all operations using temporary files which are saved on commit or deleted on abort. Constraints - ----------- - - TransactionManagers (TM) must not be persistent themselves, i.e. must not have a _p_jar attribute. - - We have to make sure the ExtFile's _abort is called *before* the ZODB destroys the attributes of the persistent object. Solution - -------- - - ProxyTM is a subclass of TM.TM that keeps a (hard) reference to the (wrapped) persistent object it manages. Calls to _begin, _finish, and _abort are forwarded to the target object. - - TMRegistry is a module-level container for ProxyTMs. It creates and holds ProxyTMs keyed by (target_id, thread_id). - - ExtFiles implement _finish and _abort and register with the machinery by calling TM.register(self). - - On commit (or abort) the ProxyTM notifies its target object and removes itself from the registry. Hacks - ----- - - We manipulate the transaction's _resources attribute directly. This is to guarantee the ProxyTM is processed before other resources. There may be a way to achieve this using official APIs only, but I can't seem to find one. """ How can I solve this?. Is there any product similar to ExtFile/ExtImage? I can program Python if somebody guides me around the zope2 internal details... - -- Jesús Cea Avión _/_/ _/_/_/ _/_/_/ jcea(a)jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ Twitter: @jcea _/_/ _/_/ _/_/_/_/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQCVAwUBUobdLJlgi5GaxT1NAQJCbgP/cjU/H48QqZar3/quLDGRdTEBt+5uhIve zf5SLjWnDDqcDQ0UgEhDm7BbAFC4wLDFL8TV0sP7+jCIrvqT/1P9IXUXJ13o9flV dW2ni1DILG2zD4G3pES9OQKcMyaoXeZT3/JYwB/KD+TXEnDkek1zCxwRAAdRWveh PeTL6HCbrWU= =P7r7 -----END PGP SIGNATURE-----

2 1

Zope 2.13.21 misses "bootstrap.py"
by Jesus Cea 16 Nov '13

16 Nov '13

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I am not familiar with zc.buildout, but I have quite bad experiences installing zope 2.13 using virtualenv (basically, zope needs to use specific pinned versions of products, and that would be fine if they were notified to "pip", but they are not). So I tried to install 2.13.21 using "zc.buildout", following instructions at <http://docs.zope.org/zope2/releases/2.13/INSTALL-buildout.html>, but it failed just at the very beginning, since there is no "bootstrap.py" script anywere. I have a previous 2.13.19 around, and it has a "bootstrap.py" and a "buildout.cfg". But 2.13.20 and 2.13.21 miss that file. So, how is current Zope installed?. Could you update documentation?. Thanks!. - -- Jesús Cea Avión _/_/ _/_/_/ _/_/_/ jcea(a)jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ Twitter: @jcea _/_/ _/_/ _/_/_/_/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQCVAwUBUobW9Zlgi5GaxT1NAQKn4gQAhCEhqWK+/ws0TtSlk41kQHA9uGd07EAU oQ79CDHq4O2YeUFM8gPhPV1XNYjuxBsxm+y8Uh0JfMOJ6D3Im6xDQVTvzZ+rch8B GpYAGG3tEXQVWoik6iMG66WD4OTDEaV4fCgM22Q4W5bb1U6ylLW0xCt/2RlJdCCB tMOXnkJ9V9o= =n4LG -----END PGP SIGNATURE-----

2 2

Python 3 support in any Zope Toolkit version or fork
by Dotan Cohen 12 Nov '13

12 Nov '13

Hi all, I have a Python3 web application that I am considering porting to Zope Toolkit. Currently I am handling all the web connections in low-level Python. However, Python3 is a hard requirement. Are there any Zope Toolkit versions or forks which are Python3 compatible? What are the major stumbling blocks to porting, other than syntax? I've googled the subject a bit but I've found no details on the subject presented by anybody associated with the project. Thank you! -- Dotan Cohen http://gibberish.co.il http://what-is-what.com

3 4

Re: [Zope] Python 3 support in any Zope Toolkit version or fork
by Dotan Cohen 12 Nov '13

12 Nov '13

On Tue, Nov 12, 2013 at 4:09 PM, Stephan Richter <stephan.richter(a)gmail.com> wrote: > ZTK 2.0 aims at Python 3 compatibility. All relevant packages have been > ported, but many are alpha releases, because they awaited a final ZODB 4.0 > release, which brought Python 3 support. > > Since ZODB 4.0 is now released, it is mostly a matter of releasing all > packages as final. > > I have ported an entire app to Python 3 before, so I know the porting is > pretty complete. For a full list of ported packages go here: > > http://zope3.pov.lt/py3/ > > Regards, > Stephan Thank you Stephan. I intend to play around with Zope Toolkit 2 and I'll report any issues that I come across. -- Dotan Cohen http://gibberish.co.il http://what-is-what.com

1 0

manipulate ZSQL results in Zope code
by Jaroslav Lukesh 30 Sep '13

30 Sep '13

Hi all, I have switched from years old database adapter to SQLRelay, yep, it is rugged stable on unstable connections, but this adapter get all number values as strings (older version where numbers are numbers does not work with cp1250 - problem solved in 0.51 this year). I need numbers as numbers, of course, because application have thousands of queries. I was try to change source in SQLrelay, but no success probably due academy-like code. So as I know that Zope have readable code... I was make changes in /lib/python/Shared/DC/ZRDB in files RDB, Results, Aqueduct (*.py) Again, all my changes are unsuccessfull.... seems like I was change some differ place, but no, if I make syntax error, zope wont start and changes in column names was OK too. Would you like to help me, where is the right place to manipulate result values from ZSQL methods, please? Many thanks, J. Lukesh

1 0

wsgi environment variable for WebOb
by Tamer Higazi 18 Aug '13

18 Aug '13

Hi people! I am running a small zope3 (bluebream) application, I want to get the the WSGI.environ variable as dict for the WebOb package in runtime. Can somebody tell me how to get it or in which package which method or function I need to call to get the WSGI environment variable as dict ?! I would thank you Tamer

1 0