April 2006 - Zope - Zope lists

Is ExtFile right for me?
by Kirk Strauser 11 Apr '06

11 Apr '06

My Zope site generates many reports in PDF format, and stores them in a folder that's only accessible to customers via a password-protected form. I currently store each report's password as a property of its File object. My problem is that I'm creating several hundred megabyte-sized file objects in my ZODB each day, and even though I expire them after five days the performance hit when storing and fetching the files is painful. I think, then, that I should be looking at an external filesystem store. LocalFS looks very nice, but it doesn't seem to support properties (which I currently need). ExtFile looks very nice and does support properties, and that's the way I'm heading. Does that seem reasonable given my usage? Also, I'm currently inserting the file uploads into a folder named "files" (ain't I creative?) with this method call: context.files.manage_addFile(id=filename, file=filecontents, content_type=filetype) I tried what I thought would be the ExtFile equivalent: context.files.manage_addProduct['ExtFile'].manage_addExtFile(id=filename, file=filecontents, content_type=filetype) but that only dumps the contents of the file to the client and doesn't actually store that data. Clients see a long page of gibberish like: Shipment Search | View Selected Images %PDF-1.2 %âãÏÓ 1 0 obj << /Type /Catalog /Pages 3 0 R >> endobj 2 0 obj << /CreationDate (D:20060410110742) /ModDate (D:20060410110742) /Producer (libtiff / tiff2pdf - 20060313 / d) >> endobj 3 0 obj << /Type /Pages /Kids [ 4 0 R 9 0 R ] /Count 2 >> endobj 4 0 obj << /Type /Page /Parent 3 0 R /MediaBox [0.0000 0.0000 608.2560 792.0000] /Contents 5 0 R /Resources << /XObject << /Im1 7 0 R >> /ProcSet [ /ImageB ] >> >> endobj 5 0 obj << /Length 6 0 R >> stream q 608.2560 0.0000 0.0000 792.0000 0.0000 0.0000 cm /Im1 Do Q endstream endobj 6 0 obj 62 endobj 7 0 obj << /Length 8 0 R /Type /XObject /Subtype In other words, the normal page header, plus the raw data. I'm not sure why that's happening. -- Kirk Strauser The Day Companies

2 2

Any to do product
by Vishal Kashyap 10 Apr '06

10 Apr '06

Hi, Can Anyone please suggest a simple stand alone To Do list product for Zope. -- With Best Regards, Vishal Kashyap. http://www.vishal.net.in

1 0

[Ann] Incremental filtering: a new way for catalog query optimization
by Dieter Maurer 09 Apr '06

09 Apr '06

Catalog searches are often slow. IncrementalSearch[2] can lead to drastic gains in query time. However, there is a large class of queries that cannot be significantly sped up: queries that contain huge Or subqueries involving bushy indexes, usually resulting from a time based subquery checking effectiveness and expiration (or other similar use cases) The newest versions of the companions "AdvancedQuery", "ManagableIndex" and "IncrementalSearch[2]" support incremental filtering to execute most such queries efficiently. With incremental filtering, the index is not use in the usual way. Instead, the remaining query parts determine a set of document candidates with is then filtered by the filtering subqueries, dropping documents not matched by these subqueries. Lets look at an example: Suppose you search for news containing 'AdvancedQuery' which are effective and not expired. The standard query would look like Eq('portal_type','News') & Eq('SearchableText', 'AdvancedQuery') & Le('effective', now) & Ge('expires', now) Internally, the "Le('effective', now)" subquery is expanded into "Or(*[Eq('effective', t) for t in 'effective' and t<=now]) which usually is huge. The "Ge" subquery is similarly expanded. The filtering query has the form Eq('portal_type','News') & Eq('SearchableText', 'AdvancedQuery') & Ge('expires', now, filter=True) & Le('effective', now, filter=True) When this query is executed, "Eq('portal_type','News') & Eq('SearchableText', 'AdvancedQuery')" determines a set of candidate objects. From this (probably small) set, objects not satisfying "expires >= now" and (then) "effective <= now" are filtered out. This way, we avoid the construction of huge Or subqueries. Of course, filtering is only efficient in some circumstances: usually, when the other query parts already garantee a small set of candidates and the filtering can avoid the construction of large intermediaries. Otherwise, filtering may not improve the query speed but increase it (maybe even drastically). Incremental filtering is a powerful optimazation tool, which need careful usage... You need the complete bundle ("AdvancedQuery", "IncrementalSearch[2]" and "ManagableIndex") when you want to use incremental filtering. More information and download: <http://www.dieter.handshake.de/pyprojects/zope> -- Dieter

1 0

how to include a local html file
by Chris Purves 09 Apr '06

09 Apr '06

Hello, How can I include an html file on my local filesystem into a DTML method? -- Good day, eh. Chris

2 2

filtering results
by gf 08 Apr '06

08 Apr '06

Hi, I am a Zope newbie. I apologize if the terminology I use below is not correct, but I hope that you get the gist of what I am about to ask. I would like to be able to 'filter' results before returning them to the user. In other words, if a user requests a given object, it is passed through a filter before being passed on to him or her. As a simple example, I may want to tidy the html before the user sees it, or eliminate or add certain tags or words. I have put together a simple 'html tidy' external method that takes raw html and produces a tidied version, but I don't know how I can apply it. Thank you for any comments, suggestions, or simple examples you may be able to provide. Best Regards, gyro

3 3

TextIndexNG and ExtFile
by Palermo, Tom 08 Apr '06

08 Apr '06

All, I've got a site with a couple hundred ExtFile objects. I now need to index the content of the various files (word, pdf, excel, ppt). I see that ExtFile doesn't have a PrincipiaSearchSource method so I created one. It returns self.index_html(). When I go back into my catalog and reindex my PrincipiaSearchSource index (a textindexng index), clicking the "Reindex" button opens the ExtFile I'm trying to index. Is there something more to the process of trying to index ExtFiles with TextIndexNG. I've googled around and tried a few things but either they don't work or I get the same results I've just described. Any ideas? Thanks, Tom Palermo

3 3

RE: [Zope] TextIndexNG question
by Palermo, Tom 07 Apr '06

07 Apr '06

I figured it out. -Tom -----Original Message----- From: Andreas Jung [mailto:lists@zopyx.com] Sent: Friday, April 07, 2006 1:33 PM To: Palermo, Tom; 'zope(a)zope.org' Subject: Re: [Zope] TextIndexNG question --On 7. April 2006 13:18:52 -0400 "Palermo, Tom" <TPalermo(a)constellagroup.com> wrote: > > Why do none of the typical Microsoft converters show up in the above > table? Because you did not read the TXNG documentation? The documentation states clearly that you must have the related converters installed (the docs documents the converts including their sources) + they have to be installed in the $PATH so they can be started from with Zope. > This is on a Linux box and I'm not sure if things like wvWare are > installed (eg. for converting and indexing msword files). see above > I can index a > pdf file object no problem, but word docs are ignored - they > catalogged but their text/content doesn't get indexed. because the pdftotext converter is installed and could be found by TXNG. > > It shows all of the microsoft converters. I don't have the > dependencies installed on this machine so I know the actual index > won't work. I'm just curious as to why the two converter listings I've provided are different. > Why should the installatation of your Windows box be the same as your Linux box? This question really makes no sense to me. The reasons are obvious (see above) and the solution is straight forward (see above): installed the required converters and make them available through $PATH as requested and documented. -aj ----------------------------------------------------------------------- - Andreas Jung ZOPYX Ltd. & Co KG - - E-mail: info(a)zopyx.com Web: www.zopyx.com, www.zopyx.de - -----------------------------------------------------------------------

1 0

TextIndexNG question
by Palermo, Tom 07 Apr '06

07 Apr '06

All, I've just installed TextIndexNG2. I created a TextIndexNG2 index called PrincipiaSearchSource in my Catalog. In the converters tab for this index, I see this: Mime-Types Description depends on application/pdf Adobe Acrobat PDF pdftotext application/vnd.sun.xml.writer OpenOffice text/html Converter HTML to ASCII text/plain Null converter text/sgml, text/xml Converter SGML to ASCII Why do none of the typical Microsoft converters show up in the above table? This is on a Linux box and I'm not sure if things like wvWare are installed (eg. for converting and indexing msword files). I can index a pdf file object no problem, but word docs are ignored - they catalogged but their text/content doesn't get indexed. The converters shows this on my Win2K machine for the same converters tab: application/msexcel, application/ms-excel, application/vnd.ms-excel Microsoft Excel xls2csv application/mspowerpoint, application/ms-powerpoint, application/vnd.ms-powerpoint Microsoft PowerPoint pptHtml application/msword, application/ms-word, application/vnd.ms-word Microsoft Word wvWare application/pdf Adobe Acrobat PDF pdftotext application/postscript Adobe Postscript Document ps2ascii application/vnd.sun.xml.writer OpenOffice text/html Converter HTML to ASCII text/plain Null converter text/sgml, text/xml Converter SGML to ASCII It shows all of the microsoft converters. I don't have the dependencies installed on this machine so I know the actual index won't work. I'm just curious as to why the two converter listings I've provided are different. Thanks, Tom Palermo

2 1

single sign-on
by Fernando Martins 07 Apr '06

07 Apr '06

Hi, I'm doing single sign-on using Apache+mod_ntlm+FastCGI. Since the last is deprecated, is there any alternative? TIA, Fernando Martins

12 19

accessing zope admin on suse 10.1
by Dwayne Hottinger 06 Apr '06

06 Apr '06

I downloaded suse 10.1 beta 4 and installed zope during the install. It seems to run fine, but I cant access the admin interface. Is there a file that I can edit as root and set the username and password? I tried the zpassword.py command and still cant access it. thanks, ddh -- Dwayne Hottinger Network Administrator Harrisonburg City Public Schools

4 3