January 2002 - Zope - Zope lists

Re: [Zope] Adding a Factory results in a new ZCatalog index type
by Maik Jablonski 26 Jan '02

26 Jan '02

dear lennart, dear dieter, thank you for help, but i think there is some kind of misunderstanding... maybe my english is not so well... sorry for that. in short-terms: when i add a simple new product with a "add list name" like TEST (as described in the Zope Book), then 'TEST' will be displayed in the list of INDEX-Types in a ZCatalog ('TextIndex', 'KeyWordIndex','PathIndex','FieldIndex' and 'TEST'!!!!!!!!!). that's what i don't want. in long-terms: what do i do (i would be glad if someone could test this on his own machine and report what happens, would take only a minute..:-)) (there's no use in this product, it's only to reproduce the effect very quick...;-)): 1. Take a fresh install of Zope 2.4.3 or 2.5.0b4 (you can surely use your configured install, this is only to avoid mistakes caused by other products or so.) 2. Go to the Product-Management and add a new Product (call it TheTest or something) 3. Enter the new product 'TheTest' 4. Add a DTML-Method with id = hello (don't change the source) in the product 'TheTest' 5. Add a Zope Factory with id = TheFactory, Add list name = TEST, Initial method = hello 6. Now create a ZCatalog somewhere in your Root-Folder or elsewhere or enter an existing ZCatalog 7. Enter the ZCatalog and CLICK ON THE INDEXES-Tab (!!!!!!!!!!!!) 8. Now pull down the "Select type to add..."-Menu. In my Menu are now: FieldIndex KeywordIndex PathIndex TEST <-- That's what i don't want here TextIndex My question is: what do i have to do, that this does not happen, because it's a little bit confusing, because TEST is not an Indexer at all... I've tested this on 2.4.3 and 2.5.0b4... fresh installs, same results. thank you for your further help in advance. greetings, maik jablonski >From: "Maik Jablonski" <maik.jablonski(a)uni-bielefeld.de> > >>Adding the new product metatype to the "Select type to add..." in the >>Management Interface is that what I want and what i get!!! but I DON'T >> >want an > >>entry in the "Add list name" in a ZCATALOG's INDEX-Tab. What to do? >> > >Explanation: The "Select type to add"-list in a folder and the "Select type >to add"-list in a catalog are one and the same. It is a list of availiable >objects that you can add at that point. When you create a new object, it >will appear in those lists. > >Question: What problem does it cause you that it appears in both list? >

3 4

RE: [Zope] Folder with one million Documents?
by sean.upton＠uniontrib.com 26 Jan '02

26 Jan '02

This will be taxing on Zope, so you need to be willing to be patient enough to optimize your application a bit. BTreeFolder works well for this, provided you are willing to consider bypassing use of the ObjectManager APIs and read/write to BTreeFolder._tree directly or use BTreeFolder._setOb() and BTreeFolder._getOb() instead of ObjectManager._getObject()... You also will REALLY need some nice hardware. I would suggest the fastest box you can get with LOTS of RAM. I would look at something along the lines of a Dual Athlon 2000+ (P-rated, not MHz) box with 3-4 GB RAM, and a striped RAID volume of fast disks. I have a BTreeFolder-derived folder and have populated it with about a third-of-a-million Cataloged objects, with each object using an underlying relational datastore, and about 8 Cataloged indexes, mostly field indexes index the result of a relational query; bulk adding these objects from a RDB datasource with cataloging takes about 2-3 hours on a P4 1.4GHz, and right now with my application, the Catalog is broken until a bulk-reindex on the advanced Tab of the Catalog - another 2 hours. I don't think BTreeFolder is a problem, but I would suspect that reindexing a Catalog with 3 million documents with full-text search setups would take you over 10-15 hours on a fast computer, longer if there is a complex amount of filtering document formats involved. Sean -----Original Message----- From: Thomas Guettler [mailto:zopestoller@thomas-guettler.de] Sent: Friday, January 25, 2002 6:56 AM To: zope(a)zope.org Subject: [Zope] Folder with one million Documents? Hi! I am developing a simple DMS. Up to now I use a python product with a BTreeFolder which contains all the documents. Every document gets an ID with DateTime().millis(). There will be up to 50 users working at the same time. And in the end I will have up to 3 million documents. Is there a better class than BTreeFolder for such mass storage? For the curious, here is one result of the benchmarks: I benchmark it with httperf: +5 requests per second +10MBit connection between client and server +every request creates a document Anzahl der Dokumente: 2159 httperf.exe: warning: open file limit > FD_SETSIZE; limiting max. # of open files to FD_SETSIZE httperf.exe --timeout=5 --client=0/1 --server=prophet --port=8080 --uri=/a/benchmarks/create_new_doc --rate=5 --send-buffer=4096 --recv-buffer=16384 --add-header='Authorization: Basic em9wZTp6b3Bl\n' --num-conns=1000 --num-calls=1 Maximum connect burst length: 1 Total: connections 1000 requests 1000 replies 571 test-duration 204.814 s Connection rate: 4.9 conn/s (204.8 ms/conn, <=26 concurrent connections) Connection time [ms]: min 110.0 avg 977.2 max 5257.0 median 289.5 stddev 1315.9 Connection time [ms]: connect 0.5 Connection length [replies/conn]: 1.000 Request rate: 4.9 req/s (204.8 ms/req) Request size [B]: 120.0 Reply rate [replies/s]: min 0.0 avg 2.9 max 5.2 stddev 2.2 (40 samples) Reply time [ms]: response 976.4 transfer 0.0 Reply size [B]: header 216.0 content 79.0 footer 0.0 (total 295.0) Reply status: 1xx=0 2xx=562 3xx=0 4xx=0 5xx=9 CPU time [s]: user 83.96 system 117.15 (user 41.0% system 57.2% total 98.2%) Net I/O: 1.4 KB/s (0.0*10^6 bps) Errors: total 429 client-timo 429 socket-timo 0 connrefused 0 connreset 0 Errors: fd-unavail 0 addrunavail 0 ftab-full 0 other 0 Anzahl der Dokumente: 3075 --end: Fri Jan 25 14:25:34 2002 _______________________________________________ Zope maillist - Zope(a)zope.org http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )

2 1

Zope 2.5.0b4 trouble
by Oleg Broytmann 26 Jan '02

26 Jan '02

Hi! I just installed 2.5.0b4 and tried to import my sites. One site works fine, but the other give an error on almost all requests. What's that? Error Type: KeyError Error Value: meta_index Traceback (innermost last): File /usr/local/Zope-2.5.0b4-src/lib/python/ZPublisher/Publish.py, line 150, in publish_module File /usr/local/Zope-2.5.0b4-src/lib/python/ZPublisher/Publish.py, line 114, in publish File /usr/local/Zope-2.5.0b4-src/lib/python/Zope/__init__.py, line 158, in zpublisher_exception_hook (Object: zope.net.ru) File /usr/local/Zope-2.5.0b4-src/lib/python/ZPublisher/Publish.py, line 98, in publish File /usr/local/Zope-2.5.0b4-src/lib/python/ZPublisher/mapply.py, line 88, in mapply (Object: index_html) File /usr/local/Zope-2.5.0b4-src/lib/python/ZPublisher/Publish.py, line 39, in call_object (Object: index_html) File /usr/local/Zope-2.5.0b4-src/lib/python/OFS/DTMLMethod.py, line 127, in __call__ (Object: index_html) File /usr/local/Zope-2.5.0b4-src/lib/python/DocumentTemplate/DT_String.py, line 473, in __call__ (Object: index_html) File /usr/local/Zope-2.5.0b4-src/lib/python/DocumentTemplate/DT_With.py, line 76, in render (Object: Python) File /usr/local/Zope-2.5.0b4-src/lib/python/OFS/DTMLDocument.py, line 120, in __call__ (Object: python_news_short) File /usr/local/Zope-2.5.0b4-src/lib/python/DocumentTemplate/DT_String.py, line 473, in __call__ (Object: python_news_short) File /usr/local/Zope-2.5.0b4-src/lib/python/DocumentTemplate/DT_With.py, line 76, in render (Object: Python.News.parnassus_news) File /usr/local/Zope-2.5.0b4-src/lib/python/DocumentTemplate/DT_Util.py, line 159, in eval (Object: _['meta_index']=='') (Info: _) File <string>, line 2, in f File /usr/local/Zope-2.5.0b4-src/lib/python/AccessControl/DTML.py, line 32, in guarded_getitem (Object: index_html) File /usr/local/Zope-2.5.0b4-src/lib/python/AccessControl/ZopeGuards.py, line 90, in guarded_getitem KeyError: (see above) Oleg. -- Oleg Broytmann http://www.zope.org/Members/phd/ phd(a)phd.pp.ru Programmers don't die, they just GOSUB without RETURN.

2 2

Re: [Zope] ZPatterns grief!
by Bak ＠ kedai 26 Jan '02

26 Jan '02

>Can anybody throw any light on this, or better still, point me towards >something else a little more straight-forward (well I can wish can't I?) >that will allow me to have some form of "register and login" procedure (I >want browsers to be able to enter their own messages and following that edit >them, but only once they've logged in. All this runs off a MySQL Db okay, >that was the easy part :) exUserFolder has basic membership that works well. i;m using it at www.my-zope.org. no complaints, yet ;) >

1 0

How to keep an object in the zodb cache?
by Gregory Popovitch 26 Jan '02

26 Jan '02

Hi, I am writing a product which loads a large python structure object from disk. I don't wish to store this structure in the Zope Zodb, so I load it in a variable prefixed with _v_. The code looks like: class myproduct(SimpleItem.SimpleItem): def __init__(self, id, title): " initializes a new instance of a myproduct" self.id = id self.title = title self._v_store = None def load(self): " load store into memory " import mystore if not hasattr(self, '_v_store') or not self._v_store: print "loading store" self._v_store = mystore.LoadStore(self.title) The function self.load() is called from each URL method to ensure that the store is loaded. My problem is that after some inactivity in the web server, I can see that the store is reloaded when a web page is requested, and it takes some time. I assume that the object self._v_store has been flushed from the zodb cache and discarded. How can I prevent self._v_store from ever beeing flushed from the cache? Any idea? Thanks, Gregory Popovitch

3 2

Python Product Coding Guidelines? (constructor=...)
by Thomas Guettler 25 Jan '02

25 Jan '02

Hi! When I "grep" for the constructor syntax in the __init__.py files I see that most people use the form constructor(manage_addFoo, ...) The DevGuide just calls the function addForm. I think it would be good if there would be a guideline. This makes it easier to read other peoples code. What do you think?

4 3

basic search engine (product?) for a working website in zope
by cruciatuz 25 Jan '02

25 Jan '02

i integrated a website in zope (uploaded the complete site with ftp and splitted navigation into one single file, which is dynamically inserted in every single page with dtml-var, and did some other things ...). BUT: now i want to add a simple search-form in the file with the navigation which outputs the search-results on a new page with hyperlinks, like many modern websites do (it's called a "site search", i think). i didn't find a product which can do that AND i am not able to code this thing myself (i played around with zcatalog, but it's a very complex thing and my book about zope doesn't tell how to code a site search, it just says what i 'could' do with a zcatalog). i also had a look at cmf, but i wont be able to use it with the zope installation at work. any suggestions? any products i missed? a tutorial which discribes how to solve my problem? some hints? -- many thx in advance Stefan Antoni ---------------------------- Fre Jan 25 17:53:46 EST 2002

2 1

Newbie Unable to Login to Management Interface on RH 7.2
by CLIFFORD ILKAY 25 Jan '02

25 Jan '02

Hi, I just installed Zope 2.4.3 onto Red Hat 7.2 from the RPMs that I found at <http://www.taupro.com/Downloads/Zope/Zope243> and followed the directions on that site. I elected to use Zope.cgi so that I could continue using Apache. I had to change all references to /var/www from /home/http but other than that, no problems. I can see the Zope welcome page and browse the on-line docs but I cannot login when I browse to <http://localhost/manage>. Since I was not absolutely certain what the default superuser password was out of the box, I changed it by executing the following: # /usr/bin/python /usr/share/zope/utilities/zpasswd.py /var/zope/access After I changed the password, I restarted both Apache and Zope to no avail. I ran the password program again and set the password to CLEARTEXT to ensure I had it right, restarted both Apache and Zope, and again, no luck. There is nothing obvious in the Zope and the Apache logs. Any ideas? Regards, Clifford Ilkay Dinamis Corporation 3266 Yonge Street, Suite 1419 Toronto, Ontario Canada M4N 3P6 Tel: 416-410-3326 mailto:clifford_ilkay@dinamis.com Only 44 days to AmerOmnis 2002. Register at <http://www.AmerOmnis.com>.

2 1

problem with security
by Robert Rottermann 25 Jan '02

25 Jan '02

Hi there, I have a script that is called TTW by anonymous. Its job is to copy a folder that is in one of its parent folders. The script has "Manager" as proxy role. When it is trying to paste the folder it gets a security assertion saying that it is not allowed to do so. When I give anonymous the right "create folder" every thing works fine. Why is that so?? Robert

2 1

Problems with setting Local Managers
by Agata Cruickshank 25 Jan '02

25 Jan '02

I am a zope 'newbee' and would really appreciate if someone could help me. I am trying to set up a manager on a sub-folder 'Sales' in Sales/acl_users folder, but when I try to log on http://mydomain.com:8888/../../Sales/manage and enter the password in the prompt box I do get unauthorised error (I can get in by entering the password of the manager whose details are stored under root/acl_users folder. I don't get this error on a local installation of Zope, so I assume it has something to do with the way Zope is installed on the server. I run Zope 2.4.3 binary release on sunos5 operating system, with Apache front end connected via PCGI. It runs on 8888 port. It seems that Zope cannot find a path to acl_user folder in any subfolder, and only manages to get to acl_users under Root Folder. Any suggestions? Many thanks, Agata

3 2