December 2002 - Zope

cPickle.UnpicklingError on invalid load key
by Brad Clements 09 Dec '02

09 Dec '02

I see Chris had an issue with this before, with ZCatalog, but I'm not using ZCatalog. My system was fine last week. Over the weekend I rebooted the machine (RH 8.0) and now, one of my top level folders (and all it's children) give this error when I try to "view" instances of this ZClass based object under this folder. Details: Zope 2.5.1 with custom_zodb loading from ZEO (couldn't find the ZEO version, but ZEO.StorageServer is 1.28.2.5 I have a "Rendered STX" Zclass based on ZObject, ZDTMLMethod, _ZClass_for_Renderable Instances of this class work fine everywhere in the tree, except for those under /Research in /Research and children of /Research, they do not work. If I copy the instances of this class into another folder (ie. /test) they suddenly start working again. I made a backup of Data.fs, then did a pack, then restarted Zope 2.5.1, but the problem remains. gzip'd Data.fs is 25 megabytes 2 levels below /Research there is a MountedStorage instance "PythonCE". --- Attempts to fix -- I created a new top level folder /nResearch and copied the children of /Research into /nResearch Now, everything under /nResearch seems to work.. Except for the MountedStorage. I wasn't sure if the copy process would copy the Mount, or copy child objects of the mount. so in /nResearch/ ... I deleted the PythonCE folder and recreated the Mount. However, now when I try to "view" children of the mounted PythonCE under /nResearch, I get an UnpicklingError. I'll post the original exception traceback below. At this point, I am going to move to try to move to Zope 2.6 and ZEO 2, though I suspect my ZODB is corrupted. -- Aha Seems that /Images/PanellImages is corrupted and can't be loaded in ZMI. So this appears to be the actual corruption location. And now /Research/.. PythonCE (the mount) is broken and I get > The object at > http://marimba.murkworks.com:8080/Main/Research/Python/PythonCE/PythonC > EWiki/FrontPage has an empty or missing docstring. Objects must have a > docstring to be published. Argh.. going to find the backup tape.. -- Original traceback Site Error An error was encountered while publishing this resource. Error Type: TALESError Error Value: cPickle.UnpicklingError on invalid load key, ''. in '', at line 155, column 21 -------------------------------------------------------------------------------- Troubleshooting Suggestions The URL may be incorrect. The parameters passed to this resource may be incorrect. A resource that this resource relies on may be encountering an error. For more detailed information about the error, please refer to the HTML source for this page. If the error persists please contact the site maintainer. Thank you for your patience. Traceback (innermost last): File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/ZPublisher/Publish.py, line 150, in publish_module File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/ZPublisher/Publish.py, line 114, in publish File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/Zope/__init__.py, line 159, in zpublisher_exception_hook (Object: Overview) File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/ZPublisher/Publish.py, line 98, in publish File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/ZPublisher/mapply.py, line 88, in mapply (Object: index_html) File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/ZPublisher/Publish.py, line 39, in call_object (Object: index_html) File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/OFS/DTMLMethod.py, line 127, in __call__ (Object: index_html) File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/DocumentTemplate/DT_String.py, line 473, in __call__ (Object: index_html) File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/DocumentTemplate/DT_Return.py, line 31, in render (Object: render) File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/Shared/DC/Scripts/Bindings.py, line 266, in __render_with_namespace__ (Object: render) File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/Shared/DC/Scripts/Bindings.py, line 283, in _bindAndExec (Object: render) File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/Products/PythonScripts/PythonScript.py, line 302, in _exec (Object: render) (Info: ({'script': <PythonScript instance at 891dfd0>, 'context': <RenderedSTXDocument instance at 8b464a8>, 'container': <RenderedSTXDocument instance at 8b464a8>, 'traverse_subpath': []}, (), {}, None)) File Script (Python), line 3, in render File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/Shared/DC/Scripts/Bindings.py, line 252, in __call__ (Object: DisplaySTD) File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/Shared/DC/Scripts/Bindings.py, line 283, in _bindAndExec (Object: DisplaySTD) File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/Products/PageTemplates/ZopePageTemplate.py, line 210, in _exec (Object: DisplaySTD) File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/Products/PageTemplates/PageTemplate.py, line 86, in pt_render (Object: DisplaySTD) (Info: {'container': <Folder instance at 961f3b8>, 'here': <RenderedSTXDocument instance at 8b464a8>, 'modules': <Products.PageTemplates.ZRPythonExpr._SecureModuleImporter instance at 0x884501c>, 'nothing': None, 'options': {'args': ()}, 'request': <h3>form</h3><table></table><h3>cookies</h3><table><tr valign="top" align="left"><th>_ZopeId</th><td>'75921408A0lkHpV-PNU'</td></tr><tr valign="top" align="left"><th>email</th><td>'bkc(a)murkworks.com'</td></tr><tr valign="top" align="left"><th>zwiki_showlogo</th><td>'1'</td></tr><tr va 'root': <Application instance at 8c8efa8>, 'template': <ZopePageTemplate instance at 8c99148>, 'traverse_subpath': [], 'user': Anonymous User}) File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/TAL/TALInterpreter.py, line 158, in __call__ File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/TAL/TALInterpreter.py, line 191, in interpret File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/TAL/TALInterpreter.py, line 523, in do_useMacro File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/TAL/TALInterpreter.py, line 191, in interpret File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/TAL/TALInterpreter.py, line 476, in do_loop_tal File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/TAL/TALInterpreter.py, line 191, in interpret File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/TAL/TALInterpreter.py, line 345, in do_optTag_tal File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/TAL/TALInterpreter.py, line 330, in do_optTag File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/TAL/TALInterpreter.py, line 325, in no_tag File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/TAL/TALInterpreter.py, line 191, in interpret File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/TAL/TALInterpreter.py, line 491, in do_condition File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/TAL/TALInterpreter.py, line 191, in interpret File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/TAL/TALInterpreter.py, line 432, in do_insertStructure_tal File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/Products/PageTemplates/TALES.py, line 247, in evaluate File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/Products/PageTemplates/ZRPythonExpr.py, line 49, in __call__ (Info: dtimage.image.tag()) File Python expression "dtimage.image.tag()", line 2, in f File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/AccessControl/ZopeGuards.py, line 47, in guarded_getattr File /usr/local/Zope/Zope-2.5.1-linux2-x86/lib/python/ZODB/Connection.py, line 472, in setstate TALESError: (see above) Brad Clements, bkc(a)murkworks.com (315)268-1000 http://www.murkworks.com (315)268-9812 Fax AOL-IM: BKClements

1 0

What dos the second parameter of manage_addPropertySheet means?
by Rares Vernica 09 Dec '02

09 Dec '02

I found this: manage_addPropertySheet(id, ns) What does 'ns' means?! Thanks in advance, Rares

4 5

Re: [Zope] Large TextIndexNG problem
by David Mackay 09 Dec '02

09 Dec '02

PieterB wrote: >>I am having a problem creating an index for about 276,000 documents. I >>successfully created a TextIndexNG index for the documents by indexing >>50,000 at a time. >>[snip] > > >>However, the database >>size is now 2023.7M. > > Are you sure your Python has large file support (>2Gb). > Check out: > http://zope.org/Members/beacon/install_instructions > > To check if the Python version your using has large file support > enabled, start the intepreter and open a file [ f=open('temp', 'w') ] > and call [ f.tell() ] it should return a long [ 0L ] if LFS i enabled. > (otherwise it returns an integer [ 0 ]) > > Search for "LFS" or "Large file support" to get more info how to fix it. Thanks. I got an integer, so that's the problem. Dave

1 0

OT: International Friends.(Keyboard question)
by working4aliving 09 Dec '02

09 Dec '02

Hello, in the states, we (typically) use the qwerty keyboard. Is that the case in the rest of the world? Please email me direct (don't waste the list's time) Thank you all.

1 0

RE: [Zope] Large TextIndexNG Problem
by David Mackay 09 Dec '02

09 Dec '02

> The largest installation with TextIndexNG running I have seen was about > 70,000 documents large. It ran on a fast Athlon machine with 256 MB. > I assume you are running into some kind of memory problem. I suggest > to run "top" when you are trying to index the documents. Watch > the memory usage of the running Zope process and checkout your swap > process. Also you can try to play around with the number of subtransactions > uses. I have seen a memory explosion on a solaris system some > time ago but never seen such a memory problem under Linux i386. > > If you have further problems please contact me again. Thanks. I seem to have managed to scramble the database. I don't think that the python that I have loaded supports large files, so I probably exceeded the limit during the pack operation. I loaded a new zope instance, and imported components from the old instance. I've set the indexing script to commit the transaction every 20,000 documents, and that seems to have helped a lot. You were right about the memory. That was WHY we have 768MB on the machine. Dave

1 0

read() method
by Dmitry Matveev 09 Dec '02

09 Dec '02

Hello! For my way of solution the problem, i have to use the same code in PythonScript, not in External Method: import urllib params = urllib.urlencode({’spam’ : 1, ’eggs’ : 2, ’bacon’: 0}) f = urllib.urlopen( "http://www.musi-cal.com/") print f.read() I can allow the urllib module in /Products/GlobalScripts/__init__.py The problem is how allow the read() method from socket._fileobject class in my PythonScript??? M.b. there is another way to get the page's html-code from any server? -- Best regards, www.uralfirm.ru Dmitry Matveev mailto:matveev@uralfirm.ru

2 1

Re: [Zope] Large TextIndexNG problem (fwd)
by PieterB 09 Dec '02

09 Dec '02

> I am having a problem creating an index for about 276,000 documents. I > successfully created a TextIndexNG index for the documents by indexing > 50,000 at a time. > [snip] > However, the database > size is now 2023.7M. Are you sure your Python has large file support (>2Gb). Check out: http://zope.org/Members/beacon/install_instructions To check if the Python version your using has large file support enabled, start the intepreter and open a file [ f=open('temp', 'w') ] and call [ f.tell() ] it should return a long [ 0L ] if LFS i enabled. (otherwise it returns an integer [ 0 ]) Search for "LFS" or "Large file support" to get more info how to fix it. PieterB ----- End of forwarded message from PieterB -----

1 0

Large TextIndexNG problem
by David Mackay 09 Dec '02

09 Dec '02

I am having a problem creating an index for about 276,000 documents. I successfully created a TextIndexNG index for the documents by indexing 50,000 at a time. If I tried to do everything at one time, it would work for about six or seven hours, and then return with zer0 items indexed. Before I added the indexed items, my database size was about 360MB. Well, I decided that I needed to make a few changes, so I cleared the index. Then I packed the database. However, the database size is now 2023.7M. When I try to repack the database, I get an error message telling me that there have been no changes since the last pack. Also, I can't add any indices. I get a message that the connection was refused when attempting to contact the server. There are no messages in Z2.log indicating that there is a problem, and no tracebacks from python. The environment is: Zope 2.5.1 Python 2.1.3 Linux (Red Hat 8.0) kernel 2.4.18-3 The machine has 768MB of memory and 11GB of free hard drive space. Can anyone give me some pointers? Dave

2 1

Can I get the Find tab to work more like grep?
by Skip Montanaro 09 Dec '02

09 Dec '02

Is there a way to get the Find tab to work more like regular grep/egrep, that is, to display lines matching the search string instead of just the names of the objects? Alternatively, is there some product or external method I can install which will accomplish the same goal? -- Skip Montanaro - skip(a)pobox.com http://www.mojam.com/ http://www.musi-cal.com/

3 5

How to track the number of anonymous and authenticated users?
by treblid＠tpg.com.au 09 Dec '02

09 Dec '02

HI, I am a newbie to Zope and I can't figure out how to get Zope do the following: 1) Count total number of users on system 2) Keeps track of the total number of anonymous users and authenticated so that I could display them somewhere. I tried searching the web, on Zope.org, and the newsgroups and I can't seem to find any info on this. Has it got something to do with the session_data_manager? Best regards, Theng Kiat

2 1