September 2004 - Zope

standard_html_header & index_html
by John Poltorak 17 Sep '04

17 Sep '04

How do I include standard_html_header in index_html? It seems to get ignored when I use it.. -- John

5 9

Zope leaking memory?
by Dennis Allison 16 Sep '04

16 Sep '04

Pound-current (V 1.7+) Zope 2.6.4 Python 2.3.4 RH 7.3 Dual Athalon processors Occasionally, I see Zope gathering very large amounts of real memory. Normally buffering and the like seems to stabalize at around 500MB on a 4GB memory dual processor machine, but occasionally grows to 2.5GB or more. At the same time, CPU usage tends to fully occupy one of the processors for the threads holding all the memory. Clearly there is a problem. Any hints as to where to track down the problem?

7 26

ZEO / zdaemon.Daemon broken on Zope-2.7.1-0
by John Hunter 16 Sep '04

16 Sep '04

I am trying to use ZEO on Zope-2.7.1-0. I added /usr/local/Zope/lib/python to my PYTHONPATH and then tried to start ZEO with peds-pc311:/usr/local/Zope> sudo python lib/python/ZEO/start.py -p 7700 Traceback (most recent call last): File "lib/python/ZEO/start.py", line 362, in ? main(sys.argv) File "lib/python/ZEO/start.py", line 223, in main import zdaemon.Daemon ImportError: No module named Daemon peds-pc311:/usr/local/Zope> Looking in the zdaemon dir, indeed there is no Daemon class. There is a zdaemon.zdrun.Daemonizer.run method, but this has a different signature than zdaemon.Daemon.run(sys.argv, env.zeo_pid) which is what the ZEO start script is doing. Suggestions? Thanks, JDH

2 1

Browse E-mail with Zope and WAP/WML
by Segedunum 16 Sep '04

16 Sep '04

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi All, I've been playing around with the emil email client to provide e-mail access and reading via WAP/WML. Yes I know it has been unmaintained for a while (2002), but 0.6.1 seems to work fine with Zope 2.6. If anyone has any better suggestions, please feel free to share them. If not, I'll be looking into this more closely over the next few weeks and months. I've been playing around with e-mail access and this product for myself and for a client, not least because it provides a way of getting around the ludicrous restrictions UK (although not restricted to) mobile providers insist on putting on access to e-mail from your mobile phone. <potentially_useful_background_info> If anyone is interested, with providers such as O2 you cannot use the built in phone client that comes with many phones on the various Pay as You Go Tariffs. On those providers that do, you are restricted to signing up for an e-mail address with *them*, and even on the (ludicrously) expensive business options you are forced to send all mail through *their* smtp servers. If you are in a corporate environment and you manage your own e-mail, this isn't really an option. The providers try and insult you with all sorts of reasons as to why this is the case, but I think we can all guess. </potentially_useful_background_info> That aside, I've noticed an odd problem with login access to some DTML WML based documents for viewing on a mobile. I've been experimenting with a Sony Ericsson T630, and when I go to the main_wml page for viewing and put in the appropriate username and password it then continually asks me for them and then refuses to show it. Viewing it on a normal desktop browser such as Firefox or Konqueror is absolutely fine, and viewing on the mobile without authentication works fine, but this is of course, not an option. Has anyone noticed these quirks when working with WAP devices and WML pages (it's somewhat new territory for me), and how did you get around them? Any hints, general tips and technical pitfalls on using Zope and WAP/WML content would also be appreciated. Thanks in advance, David Legg -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) iD8DBQFBRMfS53OaWc7M8G0RAuXkAKCEgo91psT3PS6/u0964MhqMzwygwCfRV9E BgGES3P8fVOHLYhkq6dC4DM= =OhUE -----END PGP SIGNATURE-----

2 2

POSKeyErrors was Re: [Zope] Zope leaking memory?
by Chris McDonough 16 Sep '04

16 Sep '04

On Tue, 2004-09-14 at 19:18, Richard Jones wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Wed, 15 Sep 2004 08:51 am, you wrote: > > POSKeyErrors? Do you know which objects are going missing? > > BTrees of some sort in every case. > > Unfortunately, we make use of Transience outside of the regular SESSION stuff > (for wizards which include file uploads). > > Am still working through my daily morning POSKeyError cleanup :( I haven't seen any reports in the collector of POSKeyError resulting from Transience usage. I presume the BTrees that go missing are transience-related? Even if so I'm not sure if that indicates anything is wrong with Transience itself. Lately, Tim has made some fixes to ZODB 3.2 (aka Zope 2.7) that cause an exception to be raised if a connection is closed while there are pending modifications in that connection. Also, another bug has been fixed to make "begin()" to also do an "abort()" even if there are only modifications pending in subtransactions within a transaction ( see http://zope.org/Collectors/Zope/789 http://article.gmane.org/gmane.comp.web.zope.zodb/5364 ). Additionally, Zope's publisher has been changed to be more explicit about transaction management (required by the first ZODB fix, although its old behavior would have left it solved by the second ;-). Aapparently the "abort subtransactions when beginning the main transaction" bug can be a source of POSKeyErrors due to connection-cache/database desynchronization. I have no idea if this is what is biting you but it's worth a shot to find out I suppose. All of these fixes have happened recently enough that they haven't yet made it to any released Zope version and are only available in CVS (on the Zope-2_7-branch). Unrelatedly, new work has also been happening in Transience that will be merged into the 2.7 branch soon as well. This is currently in good shape on the 'chrism-pre273-branch' of Transience in CVS. See http://www.plope.com/Members/chrism/sessioning_redux for the list of features and bugfixes that have been happening to Transience and friends recently. - C

3 18

RDFSummary weirdness
by David Chandek-Stark 16 Sep '04

16 Sep '04

I am using RDFSummary 2.4p1 on Zope 2.7.0/CMF 1.4.7. I have noticed for along time this product exhibits strange behavior where -- without updating the RSS feed -- the data changes on successive calls. It seems to flip back and forth between the most recent data and the last (cached?) version. Any ideas? -- David Chandek-Stark Web Applications Developer Duke University - Perkins Library (919) 660-5859 dc(a)duke.edu

1 0

Job Posting - Plone/Zope/Python/XML Developer Wanted
by Paul Allen 16 Sep '04

16 Sep '04

(From http://birds.cornell.edu/About/jobs.html) Python Application Programmer Programmer/Analyst III - Posting #02932) DESCRIPTION: The Cornell Laboratory of Ornithology (CLO) is seeking a Programmer/Analyst to collaborate with team members to develop and maintain the Scientific Knowledge and Education Network (SKEN). SKEN will likely be created in the Plone/Zope content management system (CMS) environment, which was developed in the Python programming language. The goal of SKEN is to build an open-source infrastructure for creating knowledge and education networks centered on primary scientific references. Special attention will be given to integration with the National Science Digital Library system, of which the CLO has recently become a member. Team up with members of CLO's IT unit and the editors of the Birds of North America Online, a comprehensive reference covering the life histories of North American birds, to create SKEN. Conceptualize and design the SKEN infrastructure and develop an open-source platform from which scientific communities can be formed around primary references. This is a two year term appointment with possibility of renewal. REQUIREMENTS: Bachelors with 2-3 years experience or equivalent combination. A thorough understanding of object-oriented programming is essential, with knowledge and experience in Python and Java programming preferred. Working knowledge of XML technologies and programmatic XML document manipulation. Demonstrable skills in problem solving, critical thinking, and written and oral communication. Ability to communicate technical language in layman's terms. Illustrate proven record of successfully experience writing application requirements, architecting applications, and documenting code. Demonstrated ability to design and construct substantial computer software. Must work effectively on a team, and learn technical material quickly with the ability to prioritize multiple projects. Establish realistic goals and deliver work by scheduled deadlines. Ability to handle confidential information. Ability to work well with a diverse group in a professional, respectful work environment. Work independently while supporting the efforts of an overall team. Experience working in Plone/Zope environment is preferred. 5-10 years of experience working on substantial computer software and collaborating with a team of programmers on software projects is desired.

1 0

ZCatalog Performance problem
by Jonathan Hobbs 16 Sep '04

16 Sep '04

I am trying to install a new version of our database and have encountered some performance problems. We are using: Zope 2.6.1 Python 2.1 Linux2 ZCatalog 2.2.0 ZCTextIndex (with Lexicon) The 'old' Data.fs file contains 1,830,854 ('objects in the database'). Retrieval times, based on CallProfiler, are typically < 2 secs. The ZCatalog contains 671,873 entries, and the actual documents are stored in two standard Folders. The Lexicon contains 605,984 words. The 'new' version uses DBTab to mount a second .fs file which contains the ZCatalog and corresponding documents (which are now stored in two BTreeFolder2 type folders). The .fs file contains 7,670,410 ('objects in the database'). Retrieval times, based on CallProfiler, are are in the range 1 to 14 secs. The ZCatalog contains 782,216entries. The Lexicon contains 1,741,003 words. In both versions, the Lexicon 'Input Pipeline Stages' are the same (Splitter, CaseNormalizer, StopWordAndSingleCharRemover). In the 'old' version, the documents stored in the ZCatalog (zclass instances) are 'catalog-aware'. In the 'new' version they are not. Also, in the 'old' version the complete full text of the document was compressed and stored in the metadata table (this was eliminated in the 'new' version to reduce the size of the metadata table). Because of the very slow retrieval times, I installed ZopeProfiler and here is an extract of 'typical' results: ncalls tottime percall cumtime percall filename:lineno(function) 2 0.000 0.000 11.462 5.731 Catalog.py:717(searchResults) 2 2.069 1.035 11.461 5.730 Catalog.py:446(search) 1 0.216 0.216 9.320 9.320 OkapiIndex.py:120(_search_wids) 2 0.000 0.000 9.391 4.696 ZCTextIndex.py:167(_apply_index) 1 0.000 0.000 9.390 9.390 ParseTree.py:117(executeQuery) 1 0.000 0.000 9.390 9.390 BaseIndex.py:169(search) 1 0.216 0.216 9.320 9.320 OkapiIndex.py:120(_search_wids) 2085 0.239 0.000 9.174 0.004 Connection.py:499(setstate) 2085 0.045 0.000 8.761 0.004 FileStorage.py:647(load) 2085 8.510 0.004 8.716 0.004 FileStorage.py:620(_load) 2085 0.151 0.000 0.181 0.000 fsIndex.py:61(__getitem__) This profile was generated using a very simple script to access the zcatalog: <dtml-let res="Catalog.searchResults({'all_searchable_text' : all_searchable_text})"> <b>Done Search</b><br> </dtml-let> It appears that the delay is due to the time required to load the zctextindex into memory. This is supported by the fact that subsequent searches for the same search term take less than 1 second and do not generate any FileStorage '_load's, and running iostat during the search shows LOTS of disk activity (cpu usage is minimal). The cache 'target' is set at 100,000 and currently has about 59,000 objects (61,000 ghost objects). So, finally, the questions: 1) What would cause the Lexicon to jump from 605k words to 1741k words (a 187% increase), even though the zcatalog entries only grew from 671k to 782k (a 16.5% increase)? 2) What would cause the .fs file to jump from 1.8m entries to 7.6m entries (a 322% increase), even though the zcatalog entries only grew from 671k to 782k (a 16.5% increase)? 3) Is there any way to preload the entire zctextindex? Any and all ideas, comments and suggestions will be greatfully received! Jonathan

2 1

Call Zope objects using wget
by Paolo Dina 16 Sep '04

16 Sep '04

Hi all. I need to call a Python script (that simply creates a new folder object with a unique name) via wget. When I manually invoke the command using via shell, wget -d --spider --http-user=myuser --http-passwd=mypassword http://my.url.it/intranet/reset_portal_content/ZScriptHTML_tryForm the folder object isn't created, while calling the script via Test tab from the ZMI works correctly. The thing that makes me dubious against this misbehaviour, is the server response "302 Moved Temporarily". This is the debug message generated by wget -d: DEBUG output created by Wget 1.9.1 on linux-gnu. --12:48:15-- http://my.url.it/intranet/reset_portal_content/ZScriptHTML_tryForm => `ZScriptHTML_tryForm' Resolving my.url.it... www.xxx.yyy.zzz Caching my.url.it => www.xxx.yyy.zzz Connecting to my.url.it[www.xxx.yyy.zzz]:80... connected. Created socket 3. Releasing 0x8083c10 (new refcount 1). ---request begin--- HEAD /intranet/reset_portal_content/ZScriptHTML_tryForm HTTP/1.0 User-Agent: Wget/1.9.1 Host: my.url.it Accept: */* Connection: Keep-Alive Authorization: Basic YWRtaW46a29hbG9uZQ== Pragma: no-cache ---request end--- HTTP request sent, awaiting response... HTTP/1.1 302 Moved Temporarily Date: Wed, 15 Sep 2004 10:42:09 GMT Server: Zope/(unreleased version, python 2.3.3, linux2) ZServer/1.1 CPS/3.0 Bobo-Exception-Line: 56 Bobo-Exception-Value: See the server error log for details Bobo-Exception-File: DT_Raise.py Bobo-Exception-Type: Redirect Etag: Location: http://my.url.it/intranet/reset_portal_content Content-Type: text/html;charset=iso-8859-1 Connection: close Location: http://my.url.it/intranet/reset_portal_content [following] Closing fd 3 --12:48:15-- http://my.url.it/intranet/reset_portal_content => `reset_portal_content' Found my.url.it in host_name_addresses_map (0x8083c10) Connecting to my.url.it[www.xxx.yyy.zzz]:80... connected. Created socket 3. Releasing 0x8083c10 (new refcount 1). ---request begin--- HEAD /intranet/reset_portal_content HTTP/1.0 User-Agent: Wget/1.9.1 Host: my.url.it Accept: */* Connection: Keep-Alive Authorization: Basic YWRtaW46a29hbG9uZQ== Pragma: no-cache ---request end--- HTTP request sent, awaiting response... HTTP/1.1 200 OK Date: Wed, 15 Sep 2004 10:42:10 GMT Server: Zope/(unreleased version, python 2.3.3, linux2) ZServer/1.1 CPS/3.0 Accept-Ranges: none Ms-Author-Via: DAV Last-Modified: Wed, 15 Sep 2004 06:51:45 GMT Etag: Content-Type: application/octet-stream Connection: close Length: unspecified [application/octet-stream] Closing fd 3 200 OK From Zope mailing lists I have seen that many Zope users have never had problems using wget with Zope.. even if someone has suggested to use zeo + wget in case of problems. Any pointer to solve this issue would be highly appreciated. TIA, Paolo

3 3

Re: [Zope] passing parameter problem
by Alexis Roda 16 Sep '04

16 Sep '04

Laura McCord wrote: > I tried what you suggested and that did not work either. actStatus will > not be null. > tal:define="actStatus result/Active_Inactive" returns either active or > inactive and that is working correctly. From what I know from your page template it is equivalent to: <div tal:define="actStatus template/title; colorStat python:actStatus + ' and this is foo'" tal:content="colorStat"> </div> get a value from somewhere and calculate a second value based on first value. That works ok so your problem should be with actStatus initialization or with statChk script. > This is what my python script looks like which includes a parameter, > actStatus: > if actStatus == 'active': > return "#000" > if actStatus == 'inactive': > return "#b7b7b7" If actStatus is not 'active' nor 'inactive' the script will return None. IIRC None is tal Nothing and you're using colorStat in a tal:attributes, which interprets Nothing as "remove the attribute". Try adding a raise ValueError('actStatus="%s"' % repr(actStatus)) statement at the end of the script, just to be sure that the script gets a valid actStatus and returns a color. If statChk is just the four lines you send it could be replaced by something like: colorStat python:test(actStatus == 'active', '#000', '#b7b7b7') HTH -- //// (@ @) ----------------------------oOO----(_)----OOo-------------------------- <> Ojo por ojo y el mundo acabara ciego /\ Alexis Roda - Universitat Rovira i Virgili - Reus, Tarragona (Spain) -----------------------------------------------------------------------

1 0