[Zope-dev] ZCatalog, REQUEST, misc.

[Chris McDonough]

> I have a 1GHz Pentium with 1GB RAM and 1GB swap.  After
> I added all the objects with a little script (that took about 12 hours), I
> was going to index them to the Catalog I have. (I had to uncomment the
> index_object method's innards in CatalogAwareness.py because of a problem
> I mention further down.)
>
> After using up all my RAM and swap, taking five hours and making nothing
> happen, I gave up trying to index the objects.  (This was _with_
> subtransactions enabled; set to "10000".)
>
> This is Zope 2.3.1b1, by the way.  I'm changing to Zope 2.3.2b2 as we
> speak, but I don't think it will improve performance that much.

Have you read
http://www.zope.org/Members/mcdonc/HowTos/UpgradeToNewCatalog/index_html ?
I suspect there will be improvement.

>
> So, to the origin of one of the problems.
>
> This is part of my script to add all these objects:
>
> """
> from AccessControl.SecurityManagement import newSecurityManager
> from ZPublisher.Request import Request
> from ZPublisher.Response import Response
>
> import sys
> import Zope
> import os
>
> os.environ['SERVER_NAME'] = 'localhost'
> os.environ['SERVER_PORT'] = '80'
>
> response=Response()
> REQUEST = Request(sys.stdin, os.environ, response)
>
> applic=Zope.app()
> #applic.REQUEST = REQUEST
>
> uf=applic.acl_users
> user=uf.getUser('sysadm').__of__(uf)
> newSecurityManager(None, user)
> print applic
> """
>
> Nice, that works.  What happends from now on in the script is quite
> boring; it just adds object.  Kinda like:
>
>   applic['object'].manage_add_something()
>
> If I do not uncomment those things in CatalogAwareness.py I get an error
> on SERVER_URL.  I think it is resolve_url() that tries to make a run for
> it.

What are you referring to when you say "those things" in the sentence above?

Have you seen the "makerequest.py" script inside of the lib/python/Testing
directory?  It does what you're trying to do when you set
applic.REQUEST=REQUEST (the major difference being that it works ;-).  If
you read the Zope Developer's Guide testing and debugging chapter from its
CVS repository, I believe it's documented in there.  If you don't have the
lib/python/Testing/makerequest.py file, look on cvs.zope.org for it.

> Clever as I thought I was, I added the "applic.REQUEST = REQUEST" line
> (which, as you can see, is now commented out) and thought that it would
> fix my problems.  Initially, it looked good.  Adding the objects (with the
> CatalogAwareness.py stuff not commented out) was working.
>
> However, when I tried to access any of the newly added objects, Zope died,
> restarted and pretended like nothing had happend.  This is reproducable.

I don't know why this is, but I'd suggest you use makerequest.

> What I would really like, is to avoid this after-add-reindex procedure
> (since I can't make it happen).  Is there any way I can produce a "good
> enough" REQUEST for Zope?  So that it doesn't do that die-repeat cycle on
> me?
>
> Any hints greatly appretiated! :-)
>
> (I haven't got much confidence that the Catalog will be able to respond to
> any queries with over a million objects indexed, but time will show I
> guess.  I tried a 2.3.1b1 Catalog with 250.000 odd objects, and it wasn't
> very talkative.)

I'll be curious to see the results.  Hopefully you'll have better luck under
2.3.1b2.