September 2004 - Zope

zcatalog query sends zope spinning
by nwingfield＠dixon-hughes.com 03 Sep '04

03 Sep '04

I have a ZCatalog containing some 400,000+ objects. I recently rebuilt the catalog due to corruption, so it should be in pristine condition. Today I issued a simple query (one condition) of the ZCatalog which should have returned a very large number of objects. My intention was to generate a data export that I could then import into another system. The query ran for 2 hours, at which point I discovered that Zope was hung and completely unresponsive. I had to break out the 'kill -9' to get it to die. Does anyone have any hints on getting large amounts of data out of the ZODB using the ZCatalog (or otherwise)? My efforts thus far have proven futile. ID:[{20040902144153.711.468943402-12.6.18.86}]

3 2

"zope.org" incredibly slow
by Dieter Maurer 03 Sep '04

03 Sep '04

I have just waited for minutes to see the "collector_issue_followup_form". Such a bad response time of the Zope community site is very bad advertising for Zope... Fortunately, I know that Zope can do better! -- Dieter

4 6

RE: [Zope] Newbie Zope Question
by Jean-Francois.Doyon＠CCRS.NRCan.gc.ca 03 Sep '04

03 Sep '04

Looks simple enough to me, You'll need to post more details, such as code and tracebacks! J.F. -----Original Message----- From: zope-bounces(a)zope.org [mailto:zope-bounces@zope.org]On Behalf Of Darren Ehlers Sent: September 3, 2004 1:32 PM To: zope(a)zope.org Subject: [Zope] Newbie Zope Question I am trying to build a table of properties for each object located in a folder. I can get the objects OK, but it doesn't seem to be able to pick the user-defined properties (although it does pick up the title property just fine). Here is the code. Why doesn't this work Thanks, Darren <tr tal:repeat="item container/objectValues"> <td class="style2" tal:content="item/targeted_release">targeted_release</td> <td class="style2" tal:content="item/category">category</td> <td class="style2" tal:content="item/title">title</td> <td class="style2" tal:content="item/dev_days">dev_days</td> <td class="style2" tal:content="item/developer">devloper</td> <td class="style2" tal:content="item/biz_design">business_desing</td> <td class="style2" tal:content="item/app_design">app_design</td> <td class="style2" tal:content="item/tech_design">tech_design</td> <td class="style2" tal:content="item/test_plan">test_plan</td> <td class="style2" tal:content="item/coding">coding</td> <td class="style2" tal:content="item/testing">testing</td> </tr>

1 0

RE: [Zope] Integrating Zope with Excel.
by Jean-Francois.Doyon＠CCRS.NRCan.gc.ca 03 Sep '04

03 Sep '04

Joe, Hmmm, well yes, Zope is multi-threaded ... But this only means that one thread does one piece of work at a time ... In a production environment, there's nothing preventing 2 threads from processing 2 similar requests for an Excel conversion, in which ase you'd run into the issues I mentionned. Apart from that I can't help much more ... Your best bet would be with ActiveState support or the win32all mailing lists, if any ... J.F. -----Original Message----- From: zope-bounces+jean-francois.doyon=ccrs.nrcan.gc.ca(a)zope.org [mailto:zope-bounces+jean-francois.doyon=ccrs.nrcan.gc.ca@zope.org]On Behalf Of Joe Goldthwaite Sent: August 25, 2004 6:25 PM To: zope(a)zope.org Subject: RE: [Zope] Integrating Zope with Excel. Thanks for the reply J.F. I can use all the input I cat get. Although I might run into problems with re-entrancy when this goes into production, I don't think that's the case here. I'm the only one using the machine and I'm not submitting concurrent requests. I also thought Zope was multi-threaded so it has to finish a request before it can start another one. I could be wrong on that though. Joe Goldthwaite -----Original Message----- From: Jean-Francois.Doyon(a)CCRS.NRCan.gc.ca [mailto:Jean-Francois.Doyon@CCRS.NRCan.gc.ca] Sent: Wednesday, August 25, 2004 3:06 PM To: joe(a)goldthwaites.com; zope(a)zope.org Subject: RE: [Zope] Integrating Zope with Excel. I don't know much of anything about coding on windows, but I had looked at doing something smiliar once ... To convert PPT's on demand to HTML in my case, but I looked at using Python's COM features to do it (Regular Python + win32all, not ActiveState's). One thing that stopped me dead in my tracks was many comments form Microsoft that all the Office related COM objects are NOT re-entrant. Therefore Excel.Application can only exist once and so on, and you have to take care with such issues ... This is why nonbody's written an MS Office based Web Service :) You're not really using it as a web service, but the same concecpt applies I think? Basically, I'd look at object re-entrancy and concurency issues on the MS Office side. I ended up never needing mine, but I think I'd had it down to the fact that the best was to have the Application instance running and use Document instances to do the work ... Thing is I really didn't want to have PowerPoint instanciated permanently on the server. Anyways, just some thoughts, though I'm not sure they're of any help :) J.F. -----Original Message----- From: zope-bounces(a)zope.org [mailto:zope-bounces@zope.org]On Behalf Of Joe Goldthwaite Sent: August 25, 2004 5:40 PM To: zope(a)zope.org Subject: [Zope] Integrating Zope with Excel. Hello Again, I've been working on a Zope application and one of the tasks is to integrate the web reports with Excel. I've got some financial reports that the user can browse through using a web interface. What I'm trying to do is have a link on the page that will download the report they're viewing into a local Excel worksheet. I decided to use com because I want the worksheet to have colors, print settings, formatting, etc. I already have working code in VB. I've looked into XML and Pyxlwriter but it looks like a new learning curve compared to just translating my VB com commands into Python. I'm using Visual Studio .Net 2003 with Active Python as my development environment. When I submit the URL that creates the XL worksheet in development, it works like a top. In testing, I've generated 50 different worksheets without any problems. When I try to run the same code in Zope I have problems. Under Zope, I take the submitted URL and create a worksheet that I save to a file. Then I return a meta command to redirect the browser to the newly created file which is downloaded to the users computer. The good news is that it works perfectly - sometimes. Other times I get this error message; Module ExcelReport, line 48, in __init__ Module win32com.client, line 95, in Dispatch Module win32com.client.dynamic, line 91, in _GetGoodDispatchAndUserName Module win32com.client.dynamic, line 79, in _GetGoodDispatch com_error: (-2147221008, 'CoInitialize has not been called.', None, None) The error message is coming back from the line; XLApp = Dispatch("Excel.Application"). The really strange thing is that the error seems to be random. Sometimes it works, sometimes it doesn't. I haven't been able to determine any rhyme or reason to it. When I restart Zope, it will work one to as many as 10 times. Then I'll get the error message. If I keep trying the same URL, it might eventually work or I may have to restart Zope to get it working again. Does anyone have any idea as to what's going on here? Thanks, Joe Goldthwaite _______________________________________________ Zope maillist - Zope(a)zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) _______________________________________________ Zope maillist - Zope(a)zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )

5 12

[Warning] Potential serious Session/TemporaryStorage problem
by Dieter Maurer 03 Sep '04

03 Sep '04

Recently, strange session related "Conflict Errors" have been reported. The strange thing about these conflicts is that the timestamp when the object was read from the ZODB is *AFTER* the timestamp the object has at commit time. Apparently, the object state jumped backward in time. When the object is modified, this results in a "ConflictError". However, when the object is only read, then old data is used -- probably without anyone noticing this. This might be a reason for the many inconsistencies Zope's Session implementation is plagued with. Here is the respective collector entry: http://collector.zope.org/Zope/1488 I will try to localize the place where this jumping backward in time happens. -- Dieter

3 6

Re: including arbitrary files in zope products
by Josef Meile 03 Sep '04

03 Sep '04

> I borrowed the "product_home = Globals..." line from your code. Thanks! > > But this is not optimal because if I change applet.jar (which happens a > lot during development), it is not automatically reflected within Zope. Yes, you are right. Perhaps you should try what Max suggested. > I also suspect that some of these products might be of help: > > http://sourceforge.net/projects/localfs > > http://zope.org/Members/shh/ExtFile I think this would be a good solution for you. You could perhaps add a localfs folder or an ExtFile calling the manage_add method of such classes. I also considered them for my solution, but actually I didn't use them because I don't want to add more extra dependencies to my Product and my applet doesn't change. In case it changes some day, then I will have to think about an update method, but I think it isn't difficult to do it.

1 0

V . I . C . O . D . I . N snow
by delcie Dressler 03 Sep '04

03 Sep '04

3 3

Re: including arbitrary files in zope products
by Josef Meile 03 Sep '04

03 Sep '04

Hi Raja, some few month ago, I was trying to do the same: Store a java applet class on the file system and load it dynamically from each instance of my product, but I didn't succeeded. You can see my attempt on: http://mail.zope.org/pipermail/zope/2004-May/149813.html Anyway, I solved it storing the java applet on the Zope DB when initializing my product. The code is attached to the message. After adding the file to the Zope DB, then you can use its path on the <applet> tag and it should work. Regards, Josef --- In zope(a)yahoogroups.com, Raja Subramanian <rsubr@p...> wrote: > Hi, > > I am facing a basic problem with Zope that I'm unable to solve. > > I want to embed a java applet (jar) in my zope product. The > jar file lives as "<Product Basedir>/www/foo.jar" on the filesystem. > I want to make this available as "<Product Instance Dir>/foo.jar" > inside zope. > > To include ZPT files, I'm able to do - > > from Products.PageTemplates.PageTemplateFile import PageTemplateFile > ... > class Foo(Folder): > ... > index_html = PageTemplateFile('zpt/index_html', globals()) > > and make zpt files available inside zope. How can I do the same for > arbitrary files such as java archives and others? > > I don't know how to use the "File" object for this purpose. Can > someone please help? > > Thanks in advance. > > - Raja > > _______________________________________________ > Zope maillist - Zope@z... > http://mail.zope.org/mailman/listinfo/zope > ** No cross posts or HTML encoding! ** > (Related lists - > http://mail.zope.org/mailman/listinfo/zope-announce > http://mail.zope.org/mailman/listinfo/zope-dev ) import Globals, os from OFS.Image import manage_addFile from zLOG import LOG, INFO, BLATHER, \ PROBLEM, WARNING #Get's the physical location of your product PRODUCT_HOME=Globals.package_home(globals()) def initialize(context): #Do the normal initialization stuff here #... #... #... #Get's the ZMI Context appContext=context._ProductContext__app #On my case the java class was stored on #PRODUCT_HOME/java, here you have to change #it for your location absolutePath=os.path.join(PRODUCT_HOME,'java') """ ColorPicker.class is the name of the class (here you can use your jar file) /java is the path where you want to store it on the ZMI """ def _createFileInFolder(context, filePath, fileName, zopePath, title='', binary=0): """Creates a the folder <zopePath> and puts the file <filePath>/<fileName> on it """ pathLen=len(zopePath) #Remove leading slash if (zopePath[-1]=='/') and (pathLen>1): zopePath=zopePath[0:pathLen-1] if zopePath[0]!='/': #The product will only work with relative paths #if the instances can adquire them. _log('The path "%s" is relative.' %zopePath, severity=WARNING, detail='For better results use absolute paths.', product='createFileInFolder') zopePath='/'+zopePath pathParts=os.path.split(zopePath) basePath=pathParts[0] folder=pathParts[1] #This only happens when no slash is given at #the beginning of the path, so the root will #be assumed fileContainer=context.unrestrictedTraverse(basePath,None) if fileContainer==None: _log('The path "%s" doesn\'t exist on the Zope DB.' %basePath, severity=PROBLEM,detail='Create it first.', product='createFileInFolder') return fileFolder=fileContainer #If the given path wasn't the zope root if folder!='': _tryAdd(fileContainer,folder,manage_addFolder,'Utils') fileFolder=getattr(fileContainer,folder) absolutePath=os.path.join(filePath,fileName) if binary: data=file(absolutePath,'rb') else: data=file(absolutePath) _tryAdd(fileFolder,fileName,manage_addFile, title,file=data) def _tryAdd(container, id, addMethod, title='', file=None): """If an object doesn't exist on the Zope DB, then it will be added; otherwise, the method will return 0 """ try: if file!=None: #It is a file object addMethod(container,id,title=title, file=file) else: addMethod(container,id,title=title) except BadRequest: result=0 return 1 def _log(msg, severity=INFO, detail='', error=None, product='', exit=0): """Logs a message to the zope log""" if not NOISY_DEBUG and severity == BLATHER: return if type(msg) is UnicodeType: msg = msg.encode(sys.getdefaultencoding(), 'replace') if type(detail) is UnicodeType: detail = detail.encode(sys.getdefaultencoding(), 'replace') LOG(product, severity, msg, detail, error) if exit: Lifetime.shutdown(0)

3 2

Adding to Zsql method result object
by Peter Millar 03 Sep '04

03 Sep '04

I have a query returning data from the zsql method. How can I append to the results object that I receive. Lets say I select a,b,c from mytable I get a results object . from my use/reading of this it is a list of dictionaries ... I thought that I could do this to append a new row myrow = dict([('a','1'),('b','2'),('c','3')]) aData=aData,myrow But alas I get an error: AttributeError: 'tuple' object has no attribute 'dictionaries' Could anyone shed some light on this please, not so much the error as I understand that ... But how I can do append a row to this data structure. ____________________________________________ Peter Millar

2 1

Re: including arbitrary files in zope products
by Josef Meile 03 Sep '04

03 Sep '04

Ops, I realiced that the function called is missing. You have to call it on the initialize method like this: _createFileInFolder(appContext,absolutePath,'yourFile.jar', '/your/zope/path',title='utils') > Josef Meile wrote: > >> Hi Raja, >> >> some few month ago, I was trying to do the same: Store a java applet >> class on the file system and load it dynamically from each instance of >> my product, but I didn't succeeded. You can see my attempt on: >> >> http://mail.zope.org/pipermail/zope/2004-May/149813.html >> >> Anyway, I solved it storing the java applet on the Zope DB when >> initializing my product. The code is attached to the message. >> >> After adding the file to the Zope DB, then you can use its path on the >> <applet> tag and it should work. >> >> Regards, >> Josef >> >> >> --- In zope(a)yahoogroups.com, Raja Subramanian <rsubr@p...> wrote: >> > Hi, >> > >> > I am facing a basic problem with Zope that I'm unable to solve. >> > >> > I want to embed a java applet (jar) in my zope product. The >> > jar file lives as "<Product Basedir>/www/foo.jar" on the filesystem. >> > I want to make this available as "<Product Instance Dir>/foo.jar" >> > inside zope. >> > >> > To include ZPT files, I'm able to do - >> > >> > from Products.PageTemplates.PageTemplateFile import >> PageTemplateFile >> > ... >> > class Foo(Folder): >> > ... >> > index_html = PageTemplateFile('zpt/index_html', globals()) >> > >> > and make zpt files available inside zope. How can I do the same for >> > arbitrary files such as java archives and others? >> > >> > I don't know how to use the "File" object for this purpose. Can >> > someone please help? >> > >> > Thanks in advance. >> > >> > - Raja >> > >> > _______________________________________________ >> > Zope maillist - Zope@z... >> > http://mail.zope.org/mailman/listinfo/zope >> > ** No cross posts or HTML encoding! ** >> > (Related lists - >> > http://mail.zope.org/mailman/listinfo/zope-announce >> > http://mail.zope.org/mailman/listinfo/zope-dev ) >> >> >> ------------------------------------------------------------------------ >> >> import Globals, os >> from OFS.Image import manage_addFile >> from zLOG import LOG, INFO, BLATHER, \ >> PROBLEM, WARNING >> >> #Get's the physical location of your product >> PRODUCT_HOME=Globals.package_home(globals()) >> >> def initialize(context): >> #Do the normal initialization stuff here >> #... >> #... >> #... >> #Get's the ZMI Context >> appContext=context._ProductContext__app >> >> #On my case the java class was stored on >> #PRODUCT_HOME/java, here you have to change >> #it for your location >> absolutePath=os.path.join(PRODUCT_HOME,'java') >> """ >> ColorPicker.class is the name of the class >> (here you can use your jar file) >> /java is the path where you want to store >> it on the ZMI """ >> >> def _createFileInFolder(context, filePath, fileName, >> zopePath, title='', binary=0): >> """Creates a the folder <zopePath> and puts the >> file <filePath>/<fileName> on it >> """ >> pathLen=len(zopePath) >> #Remove leading slash >> if (zopePath[-1]=='/') and (pathLen>1): >> zopePath=zopePath[0:pathLen-1] >> >> if zopePath[0]!='/': >> #The product will only work with relative paths >> #if the instances can adquire them. >> _log('The path "%s" is relative.' %zopePath, >> severity=WARNING, >> detail='For better results use absolute paths.', >> product='createFileInFolder') >> zopePath='/'+zopePath >> pathParts=os.path.split(zopePath) >> basePath=pathParts[0] >> folder=pathParts[1] >> #This only happens when no slash is given at >> #the beginning of the path, so the root will >> #be assumed >> fileContainer=context.unrestrictedTraverse(basePath,None) >> if fileContainer==None: >> _log('The path "%s" doesn\'t exist on the Zope DB.' %basePath, >> severity=PROBLEM,detail='Create it first.', >> product='createFileInFolder') >> return >> >> fileFolder=fileContainer >> #If the given path wasn't the zope root >> if folder!='': >> _tryAdd(fileContainer,folder,manage_addFolder,'Utils') >> fileFolder=getattr(fileContainer,folder) >> absolutePath=os.path.join(filePath,fileName) >> if binary: >> data=file(absolutePath,'rb') >> else: >> data=file(absolutePath) >> _tryAdd(fileFolder,fileName,manage_addFile, >> title,file=data) >> >> def _tryAdd(container, id, addMethod, title='', >> file=None): >> """If an object doesn't exist on the Zope DB, >> then it will be added; otherwise, the >> method will return 0 >> """ >> try: >> if file!=None: >> #It is a file object >> addMethod(container,id,title=title, >> file=file) >> else: >> addMethod(container,id,title=title) >> except BadRequest: >> result=0 >> return 1 >> >> def _log(msg, severity=INFO, detail='', >> error=None, product='', exit=0): >> """Logs a message to the zope log""" >> if not NOISY_DEBUG and severity == BLATHER: >> return >> if type(msg) is UnicodeType: >> msg = msg.encode(sys.getdefaultencoding(), 'replace') >> if type(detail) is UnicodeType: >> detail = detail.encode(sys.getdefaultencoding(), 'replace') >> LOG(product, severity, msg, detail, error) >> if exit: >> Lifetime.shutdown(0)

1 1