[Fwd: XML export/import is cool! (Was Re: [Zope-dev] Deprecating XML export/import?)]
Hi! I would like to resurrect this thread with my own recent experience. My problem was to spread a set of zodb objects over several zope installations. Every installation required some slight modifications to be applied to original objects (such as usernames and passwords). So I decided to export the original data in an xml dump and write a small application which modifies the dump as necessary before import. But to my distress I couldn't import neither a modified dump nor even the original dump. The process failed with UnicodeDecodeError exception. After some investigation I realized what was the problem (at least in my case). The xml parser extracts all texts as unicode strings. But among them are base64-encoded strings which are decoded into non-unicode strings containing binary data. Of course this data can't be decoded by any codec. The code in ppml.py sometimes concatenates the raw and unicode strings and this raises UnicodeDecodeError. I worked this around by converting the unicode strings into non-unicode ones. Please look at the patch attached. Zope version 2.9.4. -- Best regards, Alexei On 25. March 2006 21:40:48 +0100 Yoshinori Okuji <yo at nexedi.com> wrote:
On Saturday 25 March 2006 15:56, Andreas Jung wrote:
Zope 2.7 throws a BadPickleGet, 12 exception, Zope 2.8 throws BadPickleGet, 13 and Zope 2.9 raises the described UnicodeDecodeError. I don't expect that the import functionality works for even more complex objects. So I consider the whole functionality as totally broken. The generated XML might be useful to perform any processing outside Zope but using it for re-importing it into another Zope systems definitely does _not_ work. So if the functionality should remain in Zope then it should be fixed for Zope 2.10 lately.
Here is a quick patch for this problem (against 2.9.1). There were two different problems:
- the id attributes were not generated, because the conditional was reverse.
- unlike xmllib, expat always returns Unicode data, so simply concatenating binary values generates Unicode objects with non-ascii characters.
It has not worked for a long time, and that's because not many people use it. But those who do usually uses it a lot, so it would be bad if it goes away. I would propose that somone that uses this heavily (like the Nexedi people), joins the Zope Foundation as a commiter, gets commiter rights and holds his or her hands over this functionality. :) And if they could write some unittests that would be even better. :) -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.nuxeo.org/
On 10/27/06, Lennart Regebro <regebro@gmail.com> wrote:
It has not worked for a long time, and that's because not many people use it. But those who do usually uses it a lot, so it would be bad if it goes away.
I would propose that somone that uses this heavily (like the Nexedi people), joins the Zope Foundation as a commiter, gets commiter rights and holds his or her hands over this functionality. :)
He, thats odd, just yesterday at a local zope meeting, a friend mentioned that he knows a company that does the same thing quite regulary. Maybe they can be forces joined in getting it right in the zope core again! And if they could write some unittests that would be even better. :)
-- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.nuxeo.org/ _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
On Friday 27 October 2006 16:46, Lennart Regebro wrote:
I would propose that somone that uses this heavily (like the Nexedi people), joins the Zope Foundation as a commiter, gets commiter rights and holds his or her hands over this functionality. :)
We are definitely willing to help the maintenance of this feature, but I am not sure what should be done. IIRC, Jim noted that it would be better to backport zope3's, while our patches are for zope2's. I don't have much time for now, so I don't think I can do that. If it is acceptable to simply fix the current code, we can provide patches.
And if they could write some unittests that would be even better. :)
Sure, it is a good thing to write tests. YO -- Yoshinori Okuji, Nexedi CTO Nexedi: Consulting and Development of Free / Open Source Software http://www.nexedi.com ERP5: Full Featured High End Open Source ERP http://www.erp5.com ERP5 Wiki: Developer Zone for ERP5 Community http://www.erp5.org
On 10/29/06, Yoshinori Okuji <yo@nexedi.com> wrote:
IIRC, Jim noted that it would be better to backport zope3's, while our patches are for zope2's.
See http://mail.zope.org/pipermail/zope-dev/2006-May/027503.html for a short thread on successfully using xmlpickle. With that recipe and patch I was able to import and export a Plone site at the time. Michael
On 10/28/06, Yoshinori Okuji <yo@nexedi.com> wrote:
for now, so I don't think I can do that. If it is acceptable to simply fix the current code, we can provide patches.
Of course it's acceptable. Although it would as previously mentioned be better if you could check them in yourselves instead.
And if they could write some unittests that would be even better. :)
Sure, it is a good thing to write tests.
If you just provide patches, tests are necessary. Nobody likes to check in others code without clear tests to show what it does. -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.nuxeo.org/
participants (5)
-
Alexei Ustyuzhaninov -
Lennart Regebro -
Michael Dunstan -
Patrick Gerken -
Yoshinori Okuji